Method, device, equipment and storage medium for assisting reading

Abstract

Embodiments of the present application disclose a method, device and equipment for assisting reading, and a storage medium, belonging to the technical field of computers. The method comprises: determining at least one set of positions corresponding to each of a plurality of text texts included in a target document, each set of positions comprising a start position and an end position, and the same set of positions being used to indicate a key text segment in the corresponding text text; constructing an element graph corresponding to each of the text texts based on each of the text texts and the at least one set of positions corresponding to each of the text texts, the element graph being used to indicate elements in the key text segment in the corresponding text text and relationships between the elements; and generating questions and answers corresponding to each of the text texts based on each of the text texts and the element graph corresponding to each of the text texts, so as to assist reading. The present application introduces an outline and a table of contents to generate the questions and the answers, so that the generated questions and answers are targeted.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for assisting reading. Background Technology

[0002] Supplemental reading, using computer technology, addresses the lack of teacher guidance during self-study of educational materials. Supplemental reading technology includes question generation and answer generation. Question generation involves the computer automatically generating questions related to the target document's content. Answer generation involves the computer automatically generating the corresponding answers to these questions. By automatically generating a large number of high-quality questions and their corresponding answers, students can assess their understanding of the target document and refine their information needs.

[0003] In related technologies, a method for question generation has been proposed. This method first identifies the text type of the target document based on its text structure, including structured and semi-structured text types, natural language text types, etc. Second, it selects a question generation model from multiple models that corresponds to the text type of the target document. Finally, it inputs the target document into the selected question generation model to generate the question.

[0004] However, in the methods described above, the questions generated by the question generation model are relatively simple, and in some cases, the generated questions are not very relevant to the content of the target document, and the generated questions also lack specificity. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for assisting reading, which can solve the problems of related technologies generating answers that lack specificity. The technical solution is as follows:

[0006] On the one hand, a method for assisting reading is provided, the method comprising:

[0007] Identify at least one set of positions corresponding to each of the multiple body texts in the target document. Each set of positions includes a start position and an end position. The same set of positions is used to indicate a key text segment in the corresponding body text.

[0008] Based on each text text and at least one set of positions corresponding to each text text, an element map is constructed for each text text. The element map is used to indicate the elements in the key text segments of the corresponding text text and the relationships between the elements.

[0009] Based on each text document and its corresponding feature map, questions and answers are generated for each text document to aid reading.

[0010] Optionally, the target document has an outline and a table of contents, the table of contents including multiple headings, each heading corresponding to one of the multiple body texts; determining at least one set of positions corresponding to each body text in the multiple body texts included in the target document includes:

[0011] The outline and table of contents are used as input to the first recurrent neural network model to obtain multiple first dense vectors output by the first recurrent neural network model. The multiple first dense vectors correspond one-to-one with the multiple titles. The first dense vectors are used to indicate the key knowledge points in the body text of the corresponding title.

[0012] Determine the text semantic information of each text, wherein the text semantic information is used to indicate the semantics of each word in the corresponding text.

[0013] Based on the textual semantic information of the plurality of first dense vectors and the plurality of main texts, at least one set of positions corresponding to each main text is determined.

[0014] Optionally, determining at least one set of positions corresponding to each piece of text based on the textual semantic information of the plurality of first dense vectors and the plurality of text texts includes:

[0015] The semantic information of the plurality of first dense vectors and the plurality of main texts is used as input to the annotation generation network model to obtain at least one set of positions corresponding to each main text output by the annotation generation network model.

[0016] Optionally, the step of constructing an element map corresponding to each text text based on each text text and at least one set of positions corresponding to each text text includes:

[0017] Select one text from the plurality of texts, and construct the feature map corresponding to the selected text according to the following operations, until the feature map corresponding to each text is constructed:

[0018] Based on the start and end positions of at least one set of positions corresponding to the selected main text, at least one key text fragment is obtained from the selected main text.

[0019] The at least one key text fragment is used as input to the feature extraction network model to obtain multiple quintuples output by the feature extraction network model. The quintuples include subject, subject category, predicate, object, and object category.

[0020] Based on the relationships between the multiple quintuples, an element map corresponding to the selected text is constructed.

[0021] Optionally, the step of generating questions and answers corresponding to each text text based on each text text and its corresponding feature map includes:

[0022] Select one text from the plurality of texts as the target text, and generate the questions and answers corresponding to the target text according to the following operations, until questions and answers corresponding to each text are generated:

[0023] Determine the first semantic relation vector corresponding to each element in the target element map, and the second semantic relation vector corresponding to each target quintuple in multiple target quintuples;

[0024] Wherein, the first semantic relationship vector is used to indicate the semantic relationship between the corresponding element and other elements in the target element map, the second semantic relationship vector is used to indicate the semantic relationship between the corresponding target quintuple and other target quintuples, the target element map is the element map corresponding to the target text, and the plurality of target quintuples are quintuples used to construct the target element map;

[0025] Based on the target element map, the first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the target text, the questions and answers corresponding to the target text are generated.

[0026] Optionally, determining the first semantic relation vector corresponding to each element in the target element map, and the second semantic relation vector corresponding to each target quintuple in the plurality of target quintuples, includes:

[0027] The target element map is used as input to the graph convolutional network model to obtain the first semantic relation vector corresponding to each element output by the graph convolutional network model.

[0028] The multiple target quintuples are used as input to the second recurrent neural network model to obtain the second semantic relation vector corresponding to each target quintuple output by the second recurrent neural network model.

[0029] Optionally, generating the questions and answers corresponding to the target text based on the target element map, the first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the target text includes:

[0030] The first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the textual semantic information of the target text are used as inputs to the attention network model to obtain the second dense vector output by the attention network model. The second dense vector is used to indicate the key knowledge points about the target element map.

[0031] Select at least one element from the target element map as the target element;

[0032] Based on the second dense vector and the target element, generate the questions and answers corresponding to the target text.

[0033] Optionally, generating the question and answer corresponding to the target text based on the second dense vector and the target element includes:

[0034] The second dense vector and the target element are used as inputs to the question generation network model to obtain the question output by the question generation network model.

[0035] The second dense vector and the question are used as inputs to the answer generation network model to obtain the answer output by the answer generation network model.

[0036] On the other hand, an auxiliary reading device is provided, the device comprising:

[0037] The determination module is used to determine at least one set of positions corresponding to each of the multiple body texts in the target document. Each set of positions includes a start position and an end position. The same set of positions is used to indicate a key text segment in the corresponding body text.

[0038] A construction module is used to construct an element map corresponding to each text text based on each text text and at least one set of positions corresponding to each text text. The element map is used to indicate the elements in the key text segments of the corresponding text text and the relationships between the elements.

[0039] The generation module is used to generate questions and answers for each text text based on each text text and its corresponding feature map, in order to assist in reading.

[0040] Optionally, the target document has an outline and a table of contents, the table of contents including multiple headings, each heading corresponding to a specific body text; the determining module includes:

[0041] The vector determination unit is used to take the outline and table of contents as input to the first recurrent neural network model to obtain multiple first dense vectors output by the first recurrent neural network model. The multiple first dense vectors correspond one-to-one with the multiple titles. The first dense vectors are used to indicate the key knowledge points in the body text of the corresponding title.

[0042] A semantic information determination unit is used to determine the text semantic information of each text, wherein the text semantic information is used to indicate the semantics of each word in the corresponding text.

[0043] The location determination unit is used to determine at least one set of locations corresponding to each text text based on the textual semantic information of the plurality of first dense vectors and the plurality of text texts.

[0044] Optionally, the position determination unit is specifically used for:

[0045] The semantic information of the plurality of first dense vectors and the plurality of main texts is used as input to the annotation generation network model to obtain at least one set of positions corresponding to each main text output by the annotation generation network model.

[0046] Optionally, the building module is specifically used for:

[0047] Select one text from the plurality of texts, and construct the feature map corresponding to the selected text according to the following operations, until the feature map corresponding to each text is constructed:

[0048] Based on the start and end positions of at least one set of positions corresponding to the selected main text, at least one key text fragment is obtained from the selected main text.

[0049] The at least one key text fragment is used as input to the feature extraction network model to obtain multiple quintuples output by the feature extraction network model. The quintuples include subject, subject category, predicate, object, and object category.

[0050] Based on the relationships between the multiple quintuples, an element map corresponding to the selected text is constructed.

[0051] Optionally, the generation module is specifically used for:

[0052] Select one text from the plurality of texts as the target text, and generate the questions and answers corresponding to the target text according to the following operations, until questions and answers corresponding to each text are generated:

[0053] Determine the first semantic relation vector corresponding to each element in the target element map, and the second semantic relation vector corresponding to each target quintuple in multiple target quintuples;

[0054] Wherein, the first semantic relationship vector is used to indicate the semantic relationship between the corresponding element and other elements in the target element map, the second semantic relationship vector is used to indicate the semantic relationship between the corresponding target quintuple and other target quintuples, the target element map is the element map corresponding to the target text, and the plurality of target quintuples are quintuples used to construct the target element map;

[0055] Based on the target element map, the first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the target text, the questions and answers corresponding to the target text are generated.

[0056] Optionally, the generation module is specifically used for:

[0057] The target element map is used as input to the graph convolutional network model to obtain the first semantic relation vector corresponding to each element output by the graph convolutional network model.

[0058] The multiple target quintuples are used as input to the second recurrent neural network model to obtain the second semantic relation vector corresponding to each target quintuple output by the second recurrent neural network model.

[0059] Optionally, the generation module is specifically used for:

[0060] The first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the textual semantic information of the target text are used as inputs to the attention network model to obtain the second dense vector output by the attention network model. The second dense vector is used to indicate the key knowledge points about the target element map.

[0061] Select at least one element from the target element map as the target element;

[0062] Based on the second dense vector and the target element, generate the questions and answers corresponding to the target text.

[0063] Optionally, the generation module is specifically used for:

[0064] The second dense vector and the target element are used as inputs to the question generation network model to obtain the question output by the question generation network model.

[0065] The second dense vector and the question are used as inputs to the answer generation network model to obtain the answer output by the answer generation network model.

[0066] On the other hand, a computer device is provided, the computer device including a memory and a processor, the memory for storing computer programs, and the processor for executing the computer programs stored in the memory to implement the steps of the above-described assisted reading method.

[0067] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the above-described assisted reading method.

[0068] On the other hand, a computer program product containing instructions is provided, which, when run on a computer, cause the computer to perform the steps of the aforementioned assisted reading method.

[0069] The technical solutions provided in this application can bring at least the following beneficial effects:

[0070] By identifying key text fragments in the target document and then constructing an element map for each body text based on these key text fragments, the generated questions and answers can be more relevant to the target document and thus more targeted. Furthermore, based on the body text and its corresponding element map, complex questions and answers involving multiple elements and their relationships can be generated, helping students improve their reasoning and associative abilities. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 This is a schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0073] Figure 2 This is a flowchart of a method for assisting reading provided in an embodiment of this application;

[0074] Figure 3 This is a schematic diagram of the structure of an auxiliary reading device provided in an embodiment of this application;

[0075] Figure 4 This is a schematic diagram of the structure of an auxiliary reading device provided in an embodiment of this application;

[0076] Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0078] Before providing a detailed explanation of the methods for assisting reading provided in the embodiments of this application, the terminology and application scenarios involved in the embodiments of this application will be introduced first.

[0079] First, the terms used in the embodiments of this application will be explained.

[0080] Key text snippets: Key sentences or paragraphs in the main text that contain key knowledge points.

[0081] Element extraction: Element extraction is an important technique for extracting key knowledge points. The main text includes descriptions of a certain event or phenomenon. Element extraction refers to extracting the elements from these descriptions to form a data format that is convenient for device storage. An element refers to the subject, predicate, object, subject category, or object category included in the description of a certain event or phenomenon in the main text.

[0082] Element map: A map composed of elements and the relationships between elements.

[0083] Word masking prediction: Word masking prediction is a commonly used unsupervised learning method, and also a typical self-supervised learning method. By masking some words in the main text and predicting the masked words, it is possible to learn the relationship between the context of the main text and the masked words.

[0084] Question generation technology: In education, asking questions helps students exchange information. Similarly, computer knowledge bases, acting as "learners," can efficiently build their knowledge bases through proactive questioning. That is, questioning helps different knowledge bases exchange information and accept each other's knowledge. Therefore, question generation technology involves selecting at least one element from an element graph corresponding to a given text as the target element and generating a question based on the knowledge points of that element graph. In other words, given a text, it generates a question related to the content of the text.

[0085] Answer generation technology: This is a typical reading comprehension task. It is mainly responsible for generating answers based on natural language descriptions for the main text and questions related to the content of the main text. Unlike multiple-choice questions and true / false questions, the answers generated by answer generation technology are often fragments of the main text or a combination of multiple text fragments.

[0086] Next, the application scenarios of the embodiments of this application will be introduced.

[0087] The assisted reading method provided in this application can be applied to various scenarios. For example, in current online self-study scenarios, there is a lack of teachers providing on-site instruction, guidance, and Q&A. Therefore, during the process of students learning a target document using online resources, the assisted reading device can generate questions related to the content of the target document. After studying the target document, students can answer the questions generated by the assisted reading device and compare their own answers with the answers generated by the device to check their understanding of the target document and improve their information needs.

[0088] Please refer to Figure 1 Assisted reading devices can extract key text fragments, such as important paragraphs or sentences, from the main text of a target document based on its outline and table of contents. Furthermore, based on these key text fragments, the device can construct an element map for each main text fragment, indicating the elements within those fragments and the relationships between them. Then, based on the main text and its corresponding element map, questions and their answers can be generated. Figure 1 As shown, the generated question is: Why are residual networks more useful than stacked networks? The answer is: Residual networks divide the training data into multiple data blocks. When training through these multiple data blocks, the error of each data block is minimized, ultimately achieving the goal of minimizing the overall error. Furthermore, residual networks can solve the problems of vanishing or exploding gradients that occur in stacked networks.

[0089] Among them, the assistive reading device can be any device that can interact with the user through one or more methods such as a keyboard, touchpad, touch screen or handwriting device, and generate questions and corresponding answers to the questions, such as PC (Personal Computer), mobile phone, smartphone, PDA (Personal Digital Assistant), PPC (Pocket PC), tablet computer, smart TV, etc.

[0090] Those skilled in the art should understand that the above application scenarios and auxiliary reading devices are merely examples. Other existing or future application scenarios and auxiliary reading devices that are applicable to the embodiments of this application should also be included within the scope of protection of the embodiments of this application, and are hereby incorporated by reference.

[0091] The method for assisting reading provided in the embodiments of this application will be explained in detail below.

[0092] Figure 2 This is a flowchart illustrating a method for assisting reading provided in an embodiment of this application. This method can be applied to assistive reading devices. Please refer to... Figure 2 The method includes the following steps.

[0093] Step 201: Determine at least one set of positions corresponding to each of the multiple body texts in the target document. Each set of positions includes a start position and an end position. The same set of positions is used to indicate a key text segment in the corresponding body text.

[0094] In some embodiments, the target document has an outline and a table of contents, the table of contents including multiple headings that correspond one-to-one with multiple body texts. The assisted reading device may determine at least one set of positions corresponding to each of the multiple body texts included in the target document by following steps (1)-(3).

[0095] (1) The outline and table of contents of the target document are used as input to the first recurrent neural network model to obtain multiple first dense vectors output by the first recurrent neural network model. These multiple first dense vectors correspond one-to-one with the multiple titles included in the target document's table of contents. The first dense vectors are used to indicate the key knowledge points in the body text of the corresponding title.

[0096] It's important to note that the outline of a target document is a summary of its key knowledge points, providing an overview of the document's main points. Typically, a document includes an outline; therefore, the target document also includes an outline. The table of contents of a target document is formed by arranging its various headings according to the document's structure. These headings are summaries and extracts of the content from the main text within the target document. Typically, a document includes one table of contents; therefore, the target document also includes one table of contents. A table of contents contains multiple headings, each corresponding to a piece of main text.

[0097] In addition, the first recurrent neural network model is pre-trained and used to determine the first dense vector corresponding to the multiple headings included in the table of contents of the target document.

[0098] (2) Determine the text semantic information of each text in the multiple texts, which is used to indicate the semantics of each word in the corresponding text.

[0099] There are several ways to determine the semantic information of each text element. As an example, each text element can be used as input to a word-masking prediction network model to obtain the semantic information of each text element output by the word-masking prediction network model.

[0100] It's important to note that the word-masking prediction network model is pre-trained based on multiple sample texts. That is, some words in each sample text are masked, and then these masked sample texts are used as input to the untrained word-masking prediction network model. The masked words in each sample text are used as the output of this model, which is then used to train the network to learn the relationship between the context of the sample text and the masked words. After training, any text can be used as input to the word-masking prediction network model to obtain the semantic information of that text as output.

[0101] Placeholders can be used to cover some words in the sample text. Of course, other methods can also be used to cover some words in the sample text. This application does not limit this method.

[0102] (3) Based on the multiple first dense vectors and the text semantic information of the multiple texts, determine at least one set of positions corresponding to each text.

[0103] In some embodiments, the plurality of first dense vectors and the textual semantic information of the plurality of main texts are used as input to the annotation generation network model to obtain at least one set of positions corresponding to each main text output by the annotation generation network model.

[0104] That is, for any title among the multiple titles included in the table of contents of the target document, the first dense vector corresponding to the title and the text semantic information of the body text corresponding to the title are used as input to the annotation generation network model to obtain at least one set of positions output by the annotation generation network model.

[0105] The annotation-generating network model is pre-trained to determine a set of positions corresponding to at least one key text segment in each main text. Each set of positions includes a start position and an end position. The start position is the position of the first character of the key text segment, and the end position is the position of the last character of the key text segment. The same set of positions is used to indicate a key text segment in the corresponding main text. Here, a key text segment refers to an important sentence or paragraph in the main text that contains key knowledge points.

[0106] Optionally, after inputting the first dense vector corresponding to the title and the semantic information of the body text corresponding to the title into the annotation generation network model, the annotation generation network model may output multiple sets of positions. For ease of description, the multiple sets of positions output by the annotation generation network model are referred to as multiple sets of candidate positions. At this time, the annotation generation network model can also output the confidence scores corresponding to the start and end positions in each set of positions. In this way, based on the confidence scores corresponding to the start and end positions in each set of positions, at least one set of positions can be selected from the multiple sets of candidate positions, thereby using the text fragments corresponding to the at least one set of selected positions as key text fragments.

[0107] In other words, the annotation-generated network model may identify multiple candidate text segments in the body text corresponding to the title, and the model will also output the confidence scores of the start and end positions for each candidate text segment. Then, based on the confidence scores of the start and end positions for each candidate text segment, at least one candidate text segment can be selected as the key text segment.

[0108] The methods for selecting at least one set of candidate positions from the multiple sets of candidate positions can include various methods, or in other words, the methods for selecting at least one candidate text segment as the key text segment from the multiple candidate text segments can include various methods. Two of these methods will be introduced below.

[0109] In the first implementation, at least one set of candidate positions is selected from the multiple sets of candidate positions, where the confidence level of the starting position is greater than a first confidence threshold and the confidence level of the ending position is greater than a second confidence threshold. The selected at least one set of candidate positions is used to indicate the key text fragments in the body text corresponding to the title.

[0110] The first confidence threshold and the second confidence threshold can be preset, and they can also be adjusted according to different needs. The first confidence threshold and the second confidence threshold can be the same or different.

[0111] The second implementation method is to determine the confidence level of each candidate position based on the confidence level of the start position and the confidence level of the end position in each candidate position group, and select at least one candidate position group with a confidence level greater than the third confidence level threshold from the multiple candidate positions. The selected at least one candidate position group is used to indicate the key text segment in the body text corresponding to the title.

[0112] As an example, for any set of candidate positions, the confidence level of the starting position is multiplied by its weight to obtain the first confidence level. The confidence level of the ending position is multiplied by its weight to obtain the second confidence level. The first and second confidence levels are then added together to obtain the overall confidence level for that set of candidate positions.

[0113] The third confidence threshold can be preset, and it can also be adjusted according to different needs.

[0114] It should be noted that in the above method, the annotation-generating network model outputs multiple sets of positions, along with the confidence scores of the start and end positions within each set. The positions corresponding to key text fragments are then selected based on these confidence scores. In other embodiments, the multiple sets of positions output by the annotation-generating network model have already been filtered based on confidence scores; that is, the selection process is performed internally within the annotation-generating network model based on the confidence scores of the start and end positions within each set. In this case, the multiple sets of positions output by the annotation-generating network model can be directly identified as the positions corresponding to the key text fragments.

[0115] Step 202: Based on each text text and at least one set of positions corresponding to each text text, construct an element map corresponding to each text text. This element map is used to indicate the elements in the key text segments of the corresponding text text and the relationships between the elements.

[0116] In some embodiments, one text text may be selected from the plurality of text texts, and the feature map corresponding to the selected text text may be constructed according to the following steps (1)-(3) until the feature map corresponding to each text text is constructed.

[0117] (1) Based on the start and end positions of at least one set of positions corresponding to the selected text, obtain at least one key text fragment from the selected text.

[0118] For any given set of positions, since the set includes a start position and an end position (the start position being the position of the first character of the key text segment and the end position being the position of the last character of the key text segment), the text segment located between the start and end positions in the selected text can be extracted as the key text segment. Once the key text segments corresponding to each set of positions are obtained, at least one key text segment can be acquired.

[0119] (2) Use at least one key text fragment as input to the feature extraction network model to obtain multiple quintuples output by the feature extraction network model, which include subject, subject category, predicate, object and object category.

[0120] Since a key text fragment may include one sentence or multiple sentences, and usually one sentence is sufficient to identify a quintuple, by using at least one key text fragment as input to the feature extraction network model, the feature extraction network model can extract the elements of each sentence from the at least one key text fragment, thereby obtaining multiple quintuples.

[0121] The feature extraction network model is pre-trained and used to extract features from key text fragments, thereby determining the subject category and object category of the extracted features, thus forming a quintuple.

[0122] The subject category includes nominal subjects and predicate subjects. A nominal subject is a noun, numeral, or noun phrase that is the subject of a statement, meaning it describes a person or thing. A predicate subject is a verb, adjective, or verb phrase that is the subject of a statement, meaning it describes an action, state, or thing. The predicate indicates the external manifestation or state of the subject's action. The object indicates the receiver or object of the action; object categories include verb objects and prepositional objects.

[0123] (3) Based on the relationship between the multiple quintuples, construct the element map corresponding to the selected text.

[0124] For any two quintuples, if they contain the same elements, then a relationship exists between them. If they do not contain the same elements, then no relationship exists between them. After processing each pair of quintuples using the above method, the quintuples with relationships can be identified. Based on these relationships, an element map corresponding to the selected text can be formed.

[0125] Because quintuples with relationships share common elements, and elements within the same quintuple also have certain relationships, deduplicating the elements from multiple quintuples yields multiple distinct elements in the element graph. For any given element, it is connected to the other elements in its quintuple. After processing each element, a element graph is formed. In other words, the element graph describes the elements and the relationships between them.

[0126] Step 203: Based on each text and its corresponding feature map, generate questions and answers for each text to aid reading.

[0127] In some embodiments, one text is selected from the plurality of texts as the target text, and questions and answers corresponding to the target text are generated according to the following steps (1)-(2) until questions and answers corresponding to each text are generated.

[0128] (1) Determine the first semantic relation vector corresponding to each element in the target element map, and the second semantic relation vector corresponding to each target quintuple in multiple target quintuples.

[0129] The first semantic relation vector is used to indicate the semantic relationship between the corresponding element and other elements in the target element map, the second semantic relation vector is used to indicate the semantic relationship between the corresponding target quintuple and other target quintuples, the target element map is the element map corresponding to the target text, and the multiple target quintuples are quintuples used to construct the target element map.

[0130] In some embodiments, the target feature map is used as input to a graph convolutional network model to obtain a first semantic relation vector corresponding to each feature output by the graph convolutional network model. These multiple target quintuples are then used as input to a second recurrent neural network model to obtain a second semantic relation vector corresponding to each target quintuple output by the second recurrent neural network model.

[0131] The graph convolutional network model is pre-trained and used to determine the first semantic relation vector corresponding to each element in the feature graph. The second recurrent neural network model is pre-trained and used to determine the second semantic relation vector corresponding to each quintuple.

[0132] (2) Based on the target element map, the first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the target text, generate the questions and answers corresponding to the target text.

[0133] In some embodiments, the first semantic relation vector corresponding to each element in the target feature graph, the second semantic relation vector corresponding to each target quintuple, and the textual semantic information of the target text are used as inputs to the attention network model to obtain the second dense vector output by the attention network model. At least one element is selected from the target feature graph as the target element, and based on the second dense vector and the target element, the question and answer corresponding to the target text are generated.

[0134] It should be noted that the attention network model is pre-trained and used to determine the second dense vector corresponding to the main text. The second dense vector is used to indicate the key knowledge points in the main text regarding that element graph.

[0135] The process of generating questions and answers corresponding to the target text based on the second dense vector and target elements includes: using the second dense vector and target elements as input to a question generation network model to obtain the question output by the model; and using the second dense vector and the generated question as input to an answer generation network model to obtain the answer output by the model. In other words, the question and answer corresponding to the target text are obtained.

[0136] The above describes selecting at least one element from the target element map as the target element to generate questions and answers. Of course, other elements can also be selected from the target element map to generate questions and answers. In other words, for any given text, one or more questions and corresponding answers may be generated.

[0137] The question generation network model and the answer generation network model are pre-trained. The question generation network model is used to extract content related to the target element from the second dense vector to generate a question, and the answer generation network model is used to extract content related to the question from the second dense vector to generate an answer.

[0138] The question generation network model and the answer generation network model can be trained in various ways. This application embodiment introduces one such method. That is, the assisted reading device can train the question generation network model and the answer generation network model to be trained using multiple dense vectors of samples, multiple sample elements, multiple sample questions, and sample answers corresponding to each sample question. Next, we will take one dense vector of samples, sample elements corresponding to a sample question, and sample answers corresponding to this sample question as an example. For example, training can be performed by iterative loop. The i-th iteration of this iterative loop includes the following steps (1)-(5), where i is a positive integer:

[0139] (1) Use the dense vector of the sample and the sample elements as input to the problem generation network model in the i-th iteration to obtain the i-th problem output by the problem generation network model in the i-th iteration.

[0140] (2) Use the dense vector of the sample and the i-th question as input to the answer generation network model of the i-th iteration to obtain the i-th answer output by the answer generation network model of the i-th iteration.

[0141] (3) If the i-th answer does not match the sample answer and the question generation network model processed in the i-th loop does not meet the first convergence condition, then based on the sample answer and the i-th answer, the question generation network model processed in the i-th loop is updated to obtain the question generation network model processed in the (i+1)-th loop. The answer generation network model processed in the i-th loop is used as the answer generation network model processed in the (i+1)-th loop, and the (i+1)-th loop of the loop iteration method is executed.

[0142] If the i-th answer does not match the sample answer, and the question generation network model processed in the i-th iteration does not meet the first convergence condition, it indicates that the question generated by the question generation network model processed in the i-th iteration is not accurate enough. In this case, the question generation network model processed in the i-th iteration can be updated based on the sample answer and the i-th answer to obtain the question generation network model processed in the (i+1)-th iteration. Then, the answer generation network model processed in the i-th iteration is used as the answer generation network model processed in the (i+1)-th iteration, and the (i+1)-th iteration is executed.

[0143] The process of determining whether the i-th answer matches the sample answer includes: determining the similarity between the i-th answer and the sample answer; if the similarity is greater than the similarity threshold, then the i-th answer matches the sample answer; if the similarity is not greater than the similarity threshold, then the i-th answer does not match the sample answer.

[0144] The process of updating the question generation network model for the i-th iteration based on the sample answer and the i-th answer includes: determining the loss value for the i-th iteration based on the sample answer and the i-th answer, and updating the question generation network model for the i-th iteration based on the loss value for the i-th iteration.

[0145] The method for determining the loss value for the i-th iteration based on the sample answer and the i-th answer can include several approaches. As an example, the number of characters between the first and second positions can be used to determine the loss value for the i-th iteration, where the first position is the starting position of the sample answer in the sample text, and the second position is the starting position of the i-th answer in the sample text. Alternatively, the first position can be the ending position of the sample answer in the sample text, and the second position is the ending position of the i-th answer in the sample text.

[0146] As another example, determine the number of identical characters in the sample answer and the i-th answer, and use the ratio of the number of identical characters to the number of characters in the sample answer as the loss value for the i-th time.

[0147] It should be noted that the first convergence condition can be set in advance according to different needs. For example, the first convergence condition may include: the number of iterations of the problem generation network model processed in the i-th loop reaches a first specified number of iterations. The first specified number of iterations can be a first maximum number of iterations or a first preset number of iterations.

[0148] (4) If the i-th answer does not match the sample answer, and the question generation network model processed in the i-th loop has met the first convergence condition, then the question generation network model processed in the i-th loop is used as the question generation network model processed in the (i+1)-th loop. Based on the sample answer and the i-th answer, the answer generation network model processed in the i-th loop is updated to obtain the answer generation network model processed in the (i+1)-th loop, and the (i+1)-th loop is executed in a loop iteration manner.

[0149] If the i-th answer does not match the sample answer, and the question generation network model in the i-th iteration has met the first convergence condition, it indicates that the question generated by the question generation network model in the i-th iteration is relatively accurate, but the answer generated by the answer generation network model in the i-th iteration is not accurate enough. In this case, the question generation network model in the i-th iteration can be used as the question generation network model in the (i+1)-th iteration. Then, based on the sample answer and the i-th answer, the answer generation network model in the i-th iteration is updated to obtain the answer generation network model in the (i+1)-th iteration, and the (i+1)-th iteration is executed.

[0150] The process of updating the answer generation network model for the i-th iteration based on the sample answer and the i-th answer is similar to the process of updating the question generation network model for the i-th iteration described above, and will not be repeated here in this embodiment.

[0151] (5) If the i-th answer matches the sample answer, or the question generation network model in the i-th iteration has met the first convergence condition and the answer generation network model in the i-th iteration has met the second convergence condition, then the trained question generation network model and answer generation network model are determined.

[0152] If the i-th answer matches the sample answer, or if the question generation network model in the i-th iteration has met the first convergence condition and the answer generation network model in the i-th iteration has met the second convergence condition, it indicates that the question generated by the question generation network model in the i-th iteration is relatively accurate, and the answer generated by the answer generation network model in the i-th iteration is also relatively accurate. In this case, the question generation network model in the i-th iteration can be directly identified as the trained question generation network model, and the answer generation network model in the i-th iteration can be identified as the trained answer generation network model.

[0153] It should be noted that the second convergence condition can be preset according to different needs. For example, the second convergence condition may include: the number of iterations of the answer generation network model in the i-th iteration reaches a second specified number of iterations. The second specified number of iterations can be a second maximum number of iterations or a second preset number of iterations. The second specified number of iterations can be the same as or different from the first specified number of iterations. That is, the second maximum number of iterations and the first maximum number of iterations can be the same as or different from the second preset number of iterations and the first preset number of iterations.

[0154] For example, please refer to Figure 3 The assisted reading device includes a label generation network model, a word masking prediction network model, a feature extraction network model, a graph convolutional network model, a second recurrent neural network model, a question generation network model, and an answer generation network model.

[0155] The process involves training the annotation generation network model using multiple sample outlines, such as outlines 1, 2, and 3. A word-masking prediction network model is trained by partially masking words in multiple sample texts, using the masked text as input and the masked words as output. Similarly, a feature extraction network model is trained using each sample text. A graph convolutional network model is trained using multiple sample feature graphs, and a second recurrent neural network model is trained using multiple sample quintuples. The question generation network model and answer generation network model are trained using the methods described above. After training, these network models can be used to determine the question and answer corresponding to each text in the target document.

[0156] In this embodiment, the outline and table of contents of the target document are introduced to generate questions and answers. This ensures that the questions and answers generated around the outline are targeted, enabling intensive review of key knowledge. Furthermore, element extraction from key text fragments ensures that the generated questions and answers contain more information related to the target document. Moreover, by fusing related quintuples (i.e., fusing related elements), complex questions and answers involving multiple elements can be generated, helping students improve their reasoning and associative abilities. In addition, the answers and target elements output by the answer generation network model are used as input to the question generation network model to obtain fluent questions output by the question generation network model, enhancing the interaction between the question generation network model and the answer generation network model while generating questions and answers.

[0157] Figure 4 This is a schematic diagram of the structure of an auxiliary reading device provided in an embodiment of this application. The auxiliary reading device can be implemented as part or all of an auxiliary reading equipment by software, hardware, or a combination of both. The auxiliary reading equipment can be... Figure 1 The assistive reading device shown. Please refer to... Figure 4 The device includes: a determination module 401, a construction module 402, and a generation module 403.

[0158] The determining module 401 is used to determine at least one set of positions corresponding to each of the multiple body texts included in the target document. Each set of positions includes a start position and an end position, and the same set of positions is used to indicate a key text segment in the corresponding body text. For detailed implementation processes, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0159] The construction module 402 is used to construct an element map corresponding to each text text based on each text text and at least one set of positions corresponding to each text text. This element map is used to indicate the elements in the key text segments of the corresponding text text and the relationships between the elements. For detailed implementation process, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0160] The generation module 403 is used to generate questions and answers for each text text based on each text text and its corresponding feature map, to aid reading. Detailed implementation processes are described in the corresponding embodiments above and will not be repeated here.

[0161] Optionally, the target document has an outline and a table of contents, the table of contents including multiple headings, each corresponding to a specific body text; the determination module 401 includes:

[0162] The vector determination unit is used to take the outline and table of contents as input to the first recurrent neural network model to obtain multiple first dense vectors output by the first recurrent neural network model. These multiple first dense vectors correspond one-to-one with the multiple titles. The first dense vectors are used to indicate the key knowledge points in the body text of the corresponding title.

[0163] The semantic information determination unit is used to determine the text semantic information of each text, which is used to indicate the semantics of each word in the corresponding text.

[0164] The location determination unit is used to determine at least one set of locations corresponding to each text based on the multiple first dense vectors and the text semantic information of the multiple texts.

[0165] Optionally, the position determination unit is specifically used for:

[0166] The multiple first dense vectors and the textual semantic information of the multiple main texts are used as input to the annotation generation network model to obtain at least one set of positions corresponding to each main text output by the annotation generation network model.

[0167] Optionally, module 402 is specifically used for:

[0168] Select one text from the multiple text sources, and construct the feature map corresponding to the selected text by following these steps, until the feature map corresponding to each text is constructed:

[0169] Based on the start and end positions of at least one set of positions corresponding to the selected main text, at least one key text fragment is obtained from the selected main text.

[0170] The key text fragment is used as input to the feature extraction network model to obtain multiple quintuples output by the feature extraction network model, which include subject, subject category, predicate, object, and object category.

[0171] Based on the relationships between these multiple quintuples, an element map corresponding to the selected text is constructed.

[0172] Optionally, the generation module 403 is specifically used for:

[0173] Select one text from the multiple text files as the target text, and generate the questions and answers corresponding to the target text by following these steps, until questions and answers for each text file are generated:

[0174] Determine the first semantic relation vector corresponding to each element in the target element map, and the second semantic relation vector corresponding to each target quintuple in multiple target quintuples;

[0175] The first semantic relation vector is used to indicate the semantic relationship between the corresponding element and other elements in the target element map, the second semantic relation vector is used to indicate the semantic relationship between the corresponding target quintuple and other target quintuples, the target element map is the element map corresponding to the target text, and the multiple target quintuples are quintuples used to construct the target element map;

[0176] Based on the target element map, the first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the target text, the questions and answers corresponding to the target text are generated.

[0177] Optionally, the generation module 403 is specifically used for:

[0178] The target feature map is used as the input to the graph convolutional network model to obtain the first semantic relation vector corresponding to each feature output by the graph convolutional network model.

[0179] The multiple target quintuples are used as input to the second recurrent neural network model to obtain the second semantic relation vector corresponding to each target quintuple output by the second recurrent neural network model.

[0180] Optionally, the generation module 403 is specifically used for:

[0181] The first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the textual semantic information of the target text are used as inputs to the attention network model to obtain the second dense vector output by the attention network model. The second dense vector is used to indicate the key knowledge points about the target element map.

[0182] Select at least one element from the target feature map as the target element;

[0183] Based on the second dense vector and the target elements, generate the questions and answers corresponding to the target text.

[0184] Optionally, the generation module 403 is specifically used for:

[0185] The second dense vector and the target elements are used as inputs to the question generation network model to obtain the question output by the question generation network model.

[0186] The second dense vector and the generated question are used as input to the answer generation network model to obtain the answer output by the answer generation network model.

[0187] In this embodiment, the outline and table of contents of the target document are introduced to generate questions and answers. This ensures that the questions and answers generated around the outline are targeted, enabling intensive review of key knowledge. Furthermore, element extraction from key text fragments ensures that the generated questions and answers contain more information related to the target document. Moreover, by fusing related quintuples (i.e., fusing related elements), complex questions and answers involving multiple elements can be generated, helping students improve their reasoning and associative abilities. In addition, the answers and target elements output by the answer generation network model are used as input to the question generation network model to obtain fluent questions output by the question generation network model, enhancing the interaction between the question generation network model and the answer generation network model while generating questions and answers.

[0188] It should be noted that the auxiliary reading device provided in the above embodiments is only illustrated by the division of the above functional modules when assisting in reading the target document. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the auxiliary reading device and the auxiliary reading method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0189] Figure 5 This is a structural block diagram of a terminal 500 provided in an embodiment of this application. The terminal can be the aforementioned auxiliary reading device. The terminal 500 can be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 500 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0190] Typically, terminal 500 includes a processor 501 and a memory 502.

[0191] Processor 501 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 501 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 501 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0192] Memory 502 may include one or more computer-readable storage media, which may be non-transitory. Memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 502 is used to store at least one instruction, which is executed by processor 501 to implement the assisted reading method provided in the method embodiments of this application.

[0193] In some embodiments, the terminal 500 may also optionally include a peripheral device interface and at least one peripheral device. The processor 501, memory 502, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: radio frequency circuitry 504, touch display screen 505, camera assembly 506, audio circuitry 507, positioning assembly 508, and power supply 509.

[0194] The peripheral device interface can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 501 and the memory 502. In some embodiments, the processor 501, memory 502 and peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 501, memory 502 and peripheral device interface can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0195] The radio frequency (RF) circuit 504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 504 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 504 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 504 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 504 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application embodiment.

[0196] Display screen 505 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 505 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 501 for processing. In this case, display screen 505 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 505, serving as the front panel of terminal 500; in other embodiments, there may be at least two display screens, respectively disposed on different surfaces of terminal 500 or in a folded design; in still other embodiments, display screen 505 may be a flexible display screen, disposed on a curved or folded surface of terminal 500. Furthermore, display screen 505 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 505 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0197] The camera assembly 506 is used to acquire images or videos. Optionally, the camera assembly 506 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 506 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0198] The audio circuit 507 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 501 for processing, or input to the radio frequency circuit 504 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the terminal 500. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 501 or the radio frequency circuit 504 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 507 may also include a headphone jack.

[0199] The positioning component 508 is used to determine the current geographic location of the terminal 500 in order to enable navigation or LBS (Location Based Service). The positioning component 508 can be a positioning component based on the US GPS (Global Positioning System), China's BeiDou system, or Russia's Galileo system.

[0200] Power supply 509 is used to power the various components in terminal 500. Power supply 509 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 509 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0201] In some embodiments, the terminal 500 further includes one or more sensors 510. The one or more sensors 510 include, but are not limited to: an accelerometer 511, a gyroscope 512, a pressure sensor 513, a fingerprint sensor 514, an optical sensor 515, and a proximity sensor 516.

[0202] Accelerometer 511 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established by terminal 500. For example, accelerometer 511 can be used to detect the components of gravitational acceleration on the three coordinate axes. Processor 501 can control touch screen 505 to display the user interface in landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 511. Accelerometer 511 can also be used for games or for acquiring user motion data.

[0203] The gyroscope sensor 512 can detect the orientation and rotation angle of the terminal 500. The gyroscope sensor 512, in conjunction with the accelerometer sensor 511, can collect 3D motion data from the user on the terminal 500. Based on the data collected by the gyroscope sensor 512, the processor 501 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0204] The pressure sensor 513 can be disposed on the side bezel of the terminal 500 and / or on the lower layer of the touch display screen 505. When the pressure sensor 513 is disposed on the side bezel of the terminal 500, it can detect the user's grip signal on the terminal 500, and the processor 501 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 513. When the pressure sensor 513 is disposed on the lower layer of the touch display screen 505, the processor 501 can control the operable controls on the UI interface based on the user's pressure operation on the touch display screen 505. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0205] The fingerprint sensor 514 is used to collect the user's fingerprint. The processor 501 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 514, or the fingerprint sensor 514 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as trusted, the processor 501 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 514 can be located on the front, back, or side of the terminal 500. When the terminal 500 has physical buttons or a manufacturer's logo, the fingerprint sensor 514 can be integrated with the physical buttons or manufacturer's logo.

[0206] An optical sensor 515 is used to collect ambient light intensity. In one embodiment, the processor 501 can control the display brightness of the touch screen 505 based on the ambient light intensity collected by the optical sensor 515. Specifically, when the ambient light intensity is high, the display brightness of the touch screen 505 is increased; when the ambient light intensity is low, the display brightness of the touch screen 505 is decreased. In another embodiment, the processor 501 can also dynamically adjust the shooting parameters of the camera assembly 506 based on the ambient light intensity collected by the optical sensor 515.

[0207] The proximity sensor 516, also known as a distance sensor, is typically located on the front panel of the terminal 500. The proximity sensor 516 is used to detect the distance between the user and the front of the terminal 500. In one embodiment, when the proximity sensor 516 detects that the distance between the user and the front of the terminal 500 is gradually decreasing, the processor 501 controls the touchscreen display 505 to switch from a screen-on state to a screen-off state; when the proximity sensor 516 detects that the distance between the user and the front of the terminal 500 is gradually increasing, the processor 501 controls the touchscreen display 505 to switch from a screen-off state to a screen-on state.

[0208] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on terminal 500, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0209] In some embodiments, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the assisted reading method described above. For example, the computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0210] It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium, in other words, it can be a non-transient storage medium.

[0211] It should be understood that all or part of the steps of the above embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-described computer-readable storage medium.

[0212] That is, in some embodiments, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of the above-described method for assisting reading.

[0213] It should be understood that "at least one" as mentioned herein refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., are not necessarily different.

[0214] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for assisting reading, characterized in that, The method includes: The outline and table of contents of the target document are used as input to the first recurrent neural network model to obtain multiple first dense vectors output by the first recurrent neural network model. The table of contents includes multiple titles, and the multiple titles correspond one-to-one with the multiple body texts included in the target document. The multiple first dense vectors correspond one-to-one with the multiple titles, and the first dense vectors are used to indicate the key knowledge points in the body text of the corresponding title. Determine the text semantic information of each text, wherein the text semantic information is used to indicate the semantics of each word in the corresponding text. Based on the textual semantic information of the plurality of first dense vectors and the plurality of main texts, at least one set of positions corresponding to each main text is determined. Each set of positions includes a start position and an end position. The same set of positions is used to indicate a key text segment in the corresponding main text. Based on each text text and at least one set of positions corresponding to each text text, an element map is constructed for each text text. The element map is used to indicate the elements in the key text segments of the corresponding text text and the relationships between the elements. Based on each text document and its corresponding feature map, questions and answers are generated for each text document to aid reading.

2. The method as described in claim 1, characterized in that, The step of determining at least one set of positions corresponding to each piece of text based on the textual semantic information of the plurality of first dense vectors and the plurality of main texts includes: The semantic information of the plurality of first dense vectors and the plurality of main texts is used as input to the annotation generation network model to obtain at least one set of positions corresponding to each main text output by the annotation generation network model.

3. The method as described in claim 1, characterized in that, The construction of an element map corresponding to each text text based on each text text and at least one set of positions corresponding to each text text includes: Select one text from the plurality of texts, and construct the feature map corresponding to the selected text according to the following operations, until the feature map corresponding to each text is constructed: Based on the start and end positions of at least one set of positions corresponding to the selected main text, at least one key text fragment is obtained from the selected main text. The at least one key text fragment is used as input to the feature extraction network model to obtain multiple quintuples output by the feature extraction network model. The quintuples include subject, subject category, predicate, object, and object category. Based on the relationships between the multiple quintuples, an element map corresponding to the selected text is constructed.

4. The method as described in claim 1, characterized in that, The process of generating questions and answers for each text text based on each text text and its corresponding feature map includes: Select one text from the plurality of texts as the target text, and generate the questions and answers corresponding to the target text according to the following operations, until questions and answers corresponding to each text are generated: Determine the first semantic relation vector corresponding to each element in the target element map, and the second semantic relation vector corresponding to each target quintuple in multiple target quintuples; Wherein, the first semantic relationship vector is used to indicate the semantic relationship between the corresponding element and other elements in the target element map, the second semantic relationship vector is used to indicate the semantic relationship between the corresponding target quintuple and other target quintuples, the target element map is the element map corresponding to the target text, and the plurality of target quintuples are quintuples used to construct the target element map; Based on the target element map, the first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the target text, the questions and answers corresponding to the target text are generated.

5. The method as described in claim 4, characterized in that, The determination of the first semantic relation vector corresponding to each element in the target element map, and the second semantic relation vector corresponding to each target quintuple in the multiple target quintuples, includes: The target element map is used as input to the graph convolutional network model to obtain the first semantic relation vector corresponding to each element output by the graph convolutional network model. The multiple target quintuples are used as input to the second recurrent neural network model to obtain the second semantic relation vector corresponding to each target quintuple output by the second recurrent neural network model.

6. The method as described in claim 4, characterized in that, The step of generating the question and answer corresponding to the target text based on the target element map, the first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the target text includes: The first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the textual semantic information of the target text are used as inputs to the attention network model to obtain the second dense vector output by the attention network model. The second dense vector is used to indicate the key knowledge points about the target element map. Select at least one element from the target element map as the target element; Based on the second dense vector and the target element, generate the questions and answers corresponding to the target text.

7. The method as described in claim 6, characterized in that, The step of generating the questions and answers corresponding to the target text based on the second dense vector and the target elements includes: The second dense vector and the target element are used as inputs to the question generation network model to obtain the question output by the question generation network model. The second dense vector and the question are used as inputs to the answer generation network model to obtain the answer output by the answer generation network model.

8. A device for assisting reading, characterized in that, The device includes: The vector determination unit is used to take the outline and table of contents of the target document as input to the first recurrent neural network model to obtain multiple first dense vectors output by the first recurrent neural network model. The table of contents includes multiple titles, and the multiple titles correspond one-to-one with the multiple body texts included in the target document. The multiple first dense vectors correspond one-to-one with the multiple titles, and the first dense vectors are used to indicate the key knowledge points in the body text of the corresponding title. A semantic information determination unit is used to determine the text semantic information of each text, wherein the text semantic information is used to indicate the semantics of each word in the corresponding text. The position determination unit is used to determine at least one set of positions corresponding to each text based on the text semantic information of the plurality of first dense vectors and the plurality of texts. Each set of positions includes a start position and an end position. The same set of positions is used to indicate a key text segment in the corresponding text. A construction module is used to construct an element map corresponding to each text text based on each text text and at least one set of positions corresponding to each text text. The element map is used to indicate the elements in the key text segments of the corresponding text text and the relationships between the elements. The generation module is used to generate questions and answers for each text text based on each text text and its corresponding feature map, in order to assist in reading.

9. The apparatus as claimed in claim 8, characterized in that, The location determination unit is specifically used for: The multiple first dense vectors and the textual semantic information of the multiple main texts are used as inputs to the annotation generation network model to obtain at least one set of positions corresponding to each main text output by the annotation generation network model. Specifically, the building module is used for: Select one text from the plurality of texts, and construct the feature map corresponding to the selected text according to the following operations, until the feature map corresponding to each text is constructed: Based on the start and end positions of at least one set of positions corresponding to the selected main text, at least one key text fragment is obtained from the selected main text. The at least one key text fragment is used as input to the feature extraction network model to obtain multiple quintuples output by the feature extraction network model. The quintuples include subject, subject category, predicate, object, and object category. Based on the relationships between the multiple quintuples, construct the element map corresponding to the selected text. Specifically, the generation module is used for: Select one text from the plurality of texts as the target text, and generate the questions and answers corresponding to the target text according to the following operations, until questions and answers corresponding to each text are generated: Determine the first semantic relation vector corresponding to each element in the target element map, and the second semantic relation vector corresponding to each target quintuple in multiple target quintuples; Wherein, the first semantic relationship vector is used to indicate the semantic relationship between the corresponding element and other elements in the target element map, the second semantic relationship vector is used to indicate the semantic relationship between the corresponding target quintuple and other target quintuples, the target element map is the element map corresponding to the target text, and the plurality of target quintuples are quintuples used to construct the target element map; Based on the target element map, the first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the target text, generate the questions and answers corresponding to the target text. Specifically, the generation module is used for: The target element map is used as input to the graph convolutional network model to obtain the first semantic relation vector corresponding to each element output by the graph convolutional network model. The multiple target quintuples are used as input to the second recurrent neural network model to obtain the second semantic relation vector corresponding to each target quintuple output by the second recurrent neural network model. Specifically, the generation module is used for: The first semantic relation vector corresponding to each element in the target element map, the second semantic relation vector corresponding to each target quintuple, and the textual semantic information of the target text are used as inputs to the attention network model to obtain the second dense vector output by the attention network model. The second dense vector is used to indicate the key knowledge points about the target element map. Select at least one element from the target element map as the target element; Based on the second dense vector and the target element, generate the question and answer corresponding to the target text. Specifically, the generation module is used for: The second dense vector and the target element are used as inputs to the question generation network model to obtain the question output by the question generation network model. The second dense vector and the question are used as inputs to the answer generation network model to obtain the answer output by the answer generation network model.

10. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-7.

11. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

Images