Key problem generation method and system based on attention mechanism and storage medium
By using attention-based language model training and clustering methods, the problem of low accuracy of pre-trained models in specialized domains is solved, and efficient generation and classification of key questions are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 63963 TROOP OF THE PLA
- Filing Date
- 2023-12-11
- Publication Date
- 2026-05-19
AI Technical Summary
When existing pre-trained language models are directly applied in specialized domains, the accuracy of summarizing key issues is relatively low.
By establishing a fixed vocabulary, training a language model using an attention mechanism, extracting multiple key questions, and generating the final key questions through clustering and expert group additions, deletions, and modifications, the training process is optimized by using a self-competitive neural network for clustering and random masking.
It improved the efficiency and accuracy of summarizing key issues and enhanced the scientific nature and efficiency of classifying multiple key issues.
Smart Images
Figure CN117688179B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method, system, and storage medium for generating key questions based on an attention mechanism, belonging to the field of artificial intelligence technology. Background Technology
[0002] With the rapid development of deep learning and natural language processing technologies, language models trained with large amounts of labeled data can quickly and efficiently understand and analyze text data. They possess capabilities such as sentiment analysis, language translation, text classification, information extraction, and text generation. Through data-driven models, they can understand, analyze, and summarize document content and intelligently generate key questions.
[0003] However, there are domain differences between currently popular pre-trained language models and specific domains. Directly summarizing the key questions raised by various experts using existing language models results in low accuracy. Summary of the Invention
[0004] To overcome the shortcomings of existing technologies, this invention provides a key question generation method, system, and storage medium based on an attention mechanism. It uses key questions proposed by experts in various fields as training words to retrain existing language models, thereby improving the efficiency and accuracy of aggregation.
[0005] To achieve the aforementioned objective, this invention provides a key question generation method based on an attention mechanism, characterized in that...
[0006] S01: Establish a fixed vocabulary, and use the words in the fixed vocabulary to train the language model containing the attention mechanism to obtain the initial training language model;
[0007] S02: Extract several key questions from the predicted mask words through initial training of the language model;
[0008] S03: Cluster the multiple first key questions using a clusterer, distribute the clustering results to each member of the expert group, and have each member of the expert group add, delete, and modify the clustered key questions to obtain multiple second key questions; use the added, deleted, and modified multiple second key questions as training statements; generate the encoding features of the training statements based on the first mask of the training statements containing words not recorded in the fixed vocabulary, and generate prediction mask words based on the encoding features;
[0009] S04: Repeat steps S02 and S03 above until the first critical problem converges to generate the final critical problem;
[0010] The clusterer includes an embedding layer, a hidden layer, and a classifier. The embedding layer performs word embedding, position embedding, and paragraph embedding on each word of the multiple first key questions to obtain a data matrix. The hidden layer converts the data matrix into hidden layer vectors for each word. The hidden layer adopts a multi-layer transformer structure that includes an attention mechanism model. The classifier classifies the words into multiple classes based on their hidden layer vectors.
[0011] To achieve the aforementioned objective, this invention also provides a key question generation method based on an attention mechanism, comprising:
[0012] S01: Establish a fixed vocabulary and train the language model using the words in the fixed vocabulary to obtain the initial trained language model;
[0013] S02: Extract several key questions from the predicted mask words through initial training of the language model;
[0014] S03: Cluster multiple first key questions, distribute the clustering results to each member of the expert group, and have each member of the expert group add, delete, and modify the clustered key questions to obtain multiple second key questions; use the added, deleted, and modified second key questions as training statements; generate the encoding features of the training statements based on the first mask of the training statements containing words not recorded in the fixed vocabulary; generate prediction mask words based on the encoding features;
[0015] S04: Repeat steps S02 and S03 above until the first critical problem converges to generate the final critical problem;
[0016] Specifically, the clustering of several key issues includes:
[0017] S1-1: Let there be N primary critical issues;
[0018] S1-2 will focus on one of the N primary critical issues related to equipment application through the hard attention mechanism model module;
[0019] S1-3: Learn the vector of the first key question into the two-dimensional intermediate layer of the self-competitive neural network;
[0020] S1-4 inputs the vectors of N-1 problems other than one of the N first key problems into the input layer of the self-competitive neural network. The self-competitive neural network clusters the N-1 problems other than one of the N first key problems. The k problems whose distance from the vector of the N-1 key problems other than one of the N first key problems to the vector of one problem in the intermediate layer is less than or equal to a set value are grouped into the same class as that one problem. The k key problems grouped into that one problem are removed from the N-1 problems to obtain Nk-1 problems that have not yet been clustered.
[0021] S1-5: Determine if Nk-1 is zero. If yes, end the classification; otherwise, proceed to step S1-6.
[0022] S1-6; N←Nk-1, and return to step S1-2.
[0023] To achieve the aforementioned objective, this invention also provides a key question generation method based on an attention mechanism, characterized by comprising:
[0024] S01: Establish a fixed vocabulary and train the language model using the words in the fixed vocabulary to obtain the initial trained language model;
[0025] S02: Extract several key questions from the predicted mask words through initial training of the language model;
[0026] S03: Cluster multiple first key questions, distribute the clustering results to each member of the expert group, and have each member of the expert group add, delete, and modify the clustered key questions to obtain multiple second key questions; use the added, deleted, and modified second key questions as training statements; generate the encoding features of the training statements based on the first mask of the training statements containing words not recorded in the fixed vocabulary; generate prediction mask words based on the encoding features;
[0027] S04: Repeat steps S02 and S03 above until the first critical problem converges to generate the final critical problem;
[0028] Specifically, the clustering of several key issues includes:
[0029] S2-1: Let there be N primary critical issues;
[0030] S2-2: Focus on k questions related to equipment application from multiple primary key questions through a hard attention mechanism model module;
[0031] S2-3: Learn the vectors of k problems into the two-dimensional intermediate layer of the self-competitive neural network;
[0032] S2-4 inputs the vectors of Nk problems other than k problems out of N first key problems into the input layer of the self-competitive neural network in turn. The self-competitive neural network performs clustering. Problems whose distances from the vectors of the Nk first key problems to the vectors of the k problems in the intermediate layer are less than or equal to a set value are respectively classified into the same class of the k problems. Then, m problems that can be classified into the k problems are removed from the Nk problems to obtain Nkm problems that have not yet been clustered.
[0033] S2-5: Determine if Nkm is zero. If yes, end the classification; otherwise, proceed to step S1-6.
[0034] S2-6; N←Nkm, and return to step S2-2.
[0035] To achieve the aforementioned objective, the present invention also provides a key question generation system based on an attention mechanism, comprising: a storage medium and a processor, wherein a computer program is stored on the storage medium; and the processor invokes the computer program to implement the above-described method.
[0036] To achieve the aforementioned objective, the present invention also provides a storage medium, characterized in that it stores a computer program thereon, the computer program being invoked by a processor to implement the above-described method.
[0037] Compared with the prior art, the present invention has the following beneficial effects:
[0038] The language model initially trained in this invention, incorporating a graph attention mechanism, extracts multiple first key questions from a prediction mask. These first key questions are then clustered, and the clustering results are distributed to each member of an expert group. Each member of the expert group adds, deletes, and modifies the clustered key questions to obtain multiple second key questions. These added, deleted, and modified second key questions are used as training statements. Based on the first mask, which randomly masks words not recorded in a fixed vocabulary within the training statements, encoded features of the training statements are generated. Predictive mask words are generated based on these encoded features. The initially trained language model is then adjusted based on these predicted mask words, thereby improving aggregation efficiency and accuracy. Attached Figure Description
[0039] Figure 1 This is a flowchart of the key question generation method based on the attention mechanism provided in the first embodiment of the present invention;
[0040] Figure 2 This is a block diagram of the clusterer provided in the first embodiment of the present invention;
[0041] Figure 3 yes Figure 2 The block diagram of the encoder in the transformer. Detailed Implementation
[0042] The preferred embodiments for implementing the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that these embodiments are merely illustrative and do not limit the scope of the invention. The computer program code for performing the operations of the embodiments of the present invention can be written in any combination of one or more programming languages. These programming languages may include object-oriented programming languages, such as Java, Smalltalk, C++, etc., or conventional procedural programming languages, such as C or similar programming languages. The method according to the present invention can be implemented by a circuit with corresponding functional configurations. The circuit includes a memory and a processor.
[0043] First Embodiment
[0044] Figure 1 This is a flowchart of the key question generation method based on the attention mechanism provided in the first embodiment of the present invention, as shown below. Figure 1 The method provided in the first embodiment of the present invention includes:
[0045] S01: Establish a fixed vocabulary and train a language model with an attention mechanism using words from the fixed vocabulary;
[0046] S02: Several key questions extracted from predicted mask words through initial training of the language model;
[0047] S03: Multiple first key questions are clustered using a classifier, and the clustering results are distributed to each member of the expert group. Each member of the expert group adds, deletes, and modifies the clustered key questions to obtain multiple second key questions. These added, deleted, and modified second key questions are used as training statements. Based on a first mask that randomly masks words not recorded in a fixed vocabulary within the training statements, encoded features of the training statements are generated. Predictive mask words are generated based on these encoded features. The clusterer includes an embedding layer, a hidden layer, and a classifier. The embedding layer performs word embedding, position embedding, and paragraph embedding on each word of the multiple first key questions to obtain a data matrix. The hidden layer converts the data matrix into hidden layer vectors for each word. The hidden layer adopts a multi-layer transformer structure including an attention mechanism model. The classifier classifies the words according to their hidden layer vectors to obtain multiple classes.
[0048] S04: Repeat steps S02 and S03 above until the first critical problem converges and generates the critical problem.
[0049] Figure 2 This is a block diagram of the clusterer provided in the first embodiment of the present invention, as shown below. Figure 2The clusterer provided in the first embodiment includes an embedding layer, a hidden layer, and a classifier. The embedding layer performs word encoding embedding, position embedding, and multiple key question embedding on each word pair (represented as x1, x2, ..., xn) in multiple first key questions to obtain a data matrix. The hidden layer converts the data matrix into hidden layer vectors h1, h2, ..., hn for each word. The hidden layer can adopt a multi-layer transformer structure including an attention mechanism model. Then, the classifier classifies the hidden layer vectors h1, h2, ..., hn for each word into classes y1, y2, ..., ym, where m is a positive integer.
[0050] The Transformer includes an encoder and a decoder. The encoder includes an input encoder, a position encoder, an attention mechanism model, a first acceleration convergence unit, a feedforward neural network, and a second acceleration convergence unit. The input encoder encodes the words that make up each first key question; the position encoder encodes the position of each word in each first key question; the adder adds the information provided by the input encoder and the information provided by the position encoder to obtain a two-dimensional data matrix X. The first column of this two-dimensional data X represents the sentence order encoding of the information to be transmitted; the second column represents the word count encoding of each sentence; and the third to Nth columns represent the word encoding and the word position encoding in each sentence. The attention mechanism model maps this two-dimensional data matrix into three lookup matrices with the same dimensions: a query matrix Q, a key matrix K, and a result matrix V. The elements of the attention matrix A are calculated according to the following formula:
[0051] In the formula, Q i To query the element in the i-th row of matrix Q; K T j K is the transpose of the key matrix. T The j-th column element, where J is an integer; the first accelerator performs residual transformation and normalization on the X+AV matrix to obtain the first normalized matrix; the neural network activates the first normalized matrix using an activation function, then performs residual transformation and normalization again through the second accelerator to obtain the second normalized matrix.
[0052] The second normalized matrix is used by the decoder to obtain the hidden layer vectors h1, h2, ..., hn encoded by each word.
[0053] In this invention, generating the encoded features includes: generating an identifier sequence for the training statement based on a fixed vocabulary and an unrecorded vocabulary in the training statement that does not overlap with the fixed vocabulary; generating a first identifier sequence that does not contain the unrecorded words and a second identifier sequence that contains the unrecorded words in the identifier sequence based on a first mask; generating an embedding vector for the first identifier sequence through a first embedding layer of the pre-trained initial training language model; generating an embedding vector for the second identifier sequence through a second embedding layer; and generating the encoded features based on the embedding vectors of the first identifier sequence and the second identifier sequence.
[0054] In this invention, generating the encoded feature based on the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence includes: merging the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence to generate the embedding vector of the identifier sequence; and generating the encoded feature based on the embedding vector of the identifier sequence through an encoding layer.
[0055] The key question generation method based on attention mechanism provided by the present invention further includes: standardizing the initial pre-trained language model after training into a key question fusion model including a single embedding layer.
[0056] In this invention, merging the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence to generate the embedding vector of the identifier sequence includes: expanding the first mask to obtain a second mask; and merging the embedding vector of the recorded identifier sequence with the embedding vector of the unrecorded identifier sequence based on the second mask to generate the embedding vector of the identifier sequence.
[0057] The first embodiment uses a clusterer for classification, thereby improving the scientific rigor and efficiency of classifying multiple key issues.
[0058] Second Embodiment
[0059] The key question generation method based on the attention mechanism provided in the second embodiment of the present invention will not repeat the same parts as in the first embodiment; only the different parts will be described. In the second embodiment, clustering multiple first key questions specifically includes:
[0060] S1-1: Let there be N primary critical issues;
[0061] S1-2 will focus on one of the N primary critical issues related to equipment application through the hard attention mechanism model module;
[0062] S1-3: Learn the vector of the first key question into the two-dimensional intermediate layer of the self-competitive neural network;
[0063] S1-4 inputs the vectors of N-1 problems other than one of the N first key problems into the input layer of the self-competitive neural network. The self-competitive neural network clusters the N-1 problems other than one of the N first key problems. The k problems whose distance from the vector of the N-1 key problems other than one of the N first key problems to the vector of one problem in the intermediate layer is less than or equal to a set value are grouped into the same class as that one problem. The k key problems grouped into that one problem are removed from the N-1 problems to obtain Nk-1 problems that have not yet been clustered.
[0064] S1-5: Determine if Nk-1 is zero. If yes, end the classification; otherwise, proceed to step S1-6.
[0065] S1-6; N←Nk-1, and return to step S1-2.
[0066] The first embodiment focuses on key issues related to equipment application among multiple first key issues through a hard attention mechanism model module, and sequentially clusters them through a self-competitive neural network, thereby improving the scientificity and efficiency of classifying multiple key issues.
[0067] The second embodiment focuses on key issues related to equipment application among multiple first key issues through a hard attention mechanism model module, and sequentially clusters them through a self-competitive neural network, thereby improving the scientificity and efficiency of classifying multiple key issues.
[0068] Third Embodiment
[0069] The key question generation method based on the attention mechanism provided in the second embodiment of the present invention will not repeat the same parts as in the first embodiment; only the different parts will be described. In the second embodiment, clustering multiple first key questions specifically includes:
[0070] S2-1: Let there be N primary critical issues;
[0071] S2-2: Focus on k questions related to equipment application from multiple primary key questions through a hard attention mechanism model module;
[0072] S2-3: Learn the vectors of k problems into the two-dimensional intermediate layer of the self-competitive neural network;
[0073] S2-4 inputs the vectors of Nk problems other than k problems out of N first key problems into the input layer of the self-competitive neural network in turn. The self-competitive neural network performs clustering. Problems whose distances from the vectors of the Nk first key problems to the vectors of the k problems in the intermediate layer are less than or equal to a set value are respectively classified into the same class of the k problems. Then, m problems that can be classified into the k problems are removed from the Nk problems to obtain Nkm problems that have not yet been clustered.
[0074] S2-5: Determine if Nkm is zero. If yes, end the classification; otherwise, proceed to step S1-6.
[0075] S2-6; N←Nkm, and return to step S2-2.
[0076] The third embodiment focuses on the k key issues related to equipment application among multiple first key issues through a hard attention mechanism model module, and further improves the scientificity and efficiency of classifying multiple key issues through self-competitive neural network clustering.
[0077] Fourth embodiment
[0078] The key question generation method based on the attention mechanism provided in the third embodiment of the present invention will not be described again for the parts that are the same as those in the first embodiment; only the differences will be described. In the third embodiment, clustering the N first key questions specifically includes:
[0079] S3-1 Randomly select k first key questions as initial cluster centers;
[0080] S3-2 calculates the semantic similarity of each first key question to the k cluster centers and assigns it to the class corresponding to the cluster center with the highest semantic similarity.
[0081] S3-3 For each category, recalculate its cluster center by calculating the mean of the semantic vectors of all questions in each category, and use it as its new cluster center;
[0082] S3-4: Repeat steps S3-2 to S3-3 above until the problems in each category no longer change.
[0083] The clustering method provided in the fourth embodiment is simple, computationally inefficient, and requires low hardware costs.
[0084] Fifth embodiment
[0085] This invention also provides a key question generation system based on an attention mechanism, comprising: a processor, a ROM, and RAM. The processor executes various processes based on a program stored in the ROM or a program sent from a storage unit to the RAM. The storage unit includes a storage medium storing a computer program. The processor invokes the computer program to implement the methods provided in the first to fourth embodiments. The RAM may also store data required for the processor to execute various processes, as needed. The processor, ROM, and RAM are interconnected via a bus. The system also includes an input / output interface, which is also connected to the bus. Additionally, the following components are connected to the input / output interface: a keyboard, a display and speakers, a hard disk, a network interface card, a modem, etc. The modem performs communication processing, for example, via a network such as the Internet or a LAN. The system also includes a driver, which can be connected to the input / output interface as needed. A removable medium, such as a semiconductor memory, can be installed on the driver as needed to install a computer program read therefrom into the storage unit.
[0086] Sixth Embodiment
[0087] The present invention also provides a storage medium having a computer program stored thereon, the computer program being invoked by a processor to implement the methods of the first to fourth embodiments. The aforementioned storage medium may include, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., but is not limited thereto.
[0088] The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments. All modifications made to the present invention without departing from the spirit of the present invention are within the technical scope of the present invention.
Claims
1. A key question generation method based on an attention mechanism, characterized in that, include: S01: Establish a fixed vocabulary, and use the words in the fixed vocabulary to train the language model containing the attention mechanism to obtain the initial training language model; S02: Extract several key questions from the predicted mask words through initial training of the language model; S03: Cluster the multiple primary key questions using a clusterer, distribute the clustering results to each member of the expert group, and have each member of the expert group add, delete, and modify the clustered key questions to obtain multiple secondary key questions; use the added, deleted, and modified secondary key questions as training statements; The encoding features of the training statement are generated based on a first mask that randomly masks words not recorded in a fixed vocabulary that are included in the training statement. Generate predicted mask words based on the encoded features; Generating the encoded features includes: generating an identifier sequence for the training statement based on a fixed vocabulary and an unrecorded vocabulary in the training statement that does not overlap with the fixed vocabulary; generating a first identifier sequence that does not contain the unrecorded words and a second identifier sequence that contains the unrecorded words based on a first mask; generating an embedding vector for the first identifier sequence through a first embedding layer of the initially trained language model; generating an embedding vector for the second identifier sequence through a second embedding layer; and generating the encoded features based on the embedding vectors of the first and second identifier sequences. S04: Repeat steps S02 and S03 above until the first critical problem converges to generate the final critical problem; The clusterer includes an embedding layer, a hidden layer, and a classifier. The embedding layer performs word embedding, position embedding, and paragraph embedding on each word of the multiple first key questions to obtain a data matrix. The hidden layer converts the data matrix into hidden layer vectors for each word. The hidden layer adopts a multi-layer transformer structure that includes an attention mechanism model. The classifier classifies the words into multiple classes based on their hidden layer vectors.
2. The key question generation method based on attention mechanism according to claim 1, wherein, Generating the encoded feature based on the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence includes: merging the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence to generate the embedding vector of the identifier sequence; and generating the encoded feature based on the embedding vector of the identifier sequence through an encoding layer.
3. The key question generation method based on attention mechanism according to claim 2, characterized in that, Also includes: The initial trained language model is standardized into a key question generation model that includes a single embedding layer.
4. The key question generation method based on attention mechanism according to claim 2, characterized in that, The process of merging the embedding vectors of the first and second identifier sequences to generate the embedding vector of the identifier sequence includes: expanding the first mask to obtain a second mask; and merging the embedding vectors of the first and second identifier sequences based on the second mask to generate the embedding vector of the identifier sequence.
5. A key question generation method based on an attention mechanism, characterized in that, include: S01: Establish a fixed vocabulary and train the language model using the words in the fixed vocabulary to obtain the initial trained language model; S02: Extract several key questions from the predicted mask words through initial training of the language model; S03: Cluster the multiple primary key questions, distribute the clustering results to each member of the expert group, and have each member of the expert group add, delete, and modify the clustered key questions to obtain multiple secondary key questions; use the added, deleted, and modified secondary key questions as training statements; The encoding features of the training statement are generated based on a first mask that randomly masks words not recorded in a fixed vocabulary that are included in the training statement. Generate predicted mask words based on the encoded features; Generating the encoded features includes: generating an identifier sequence for the training statement based on a fixed vocabulary and an unrecorded vocabulary in the training statement that does not overlap with the fixed vocabulary; generating a first identifier sequence that does not contain the unrecorded words and a second identifier sequence that contains the unrecorded words based on a first mask; generating an embedding vector for the first identifier sequence through a first embedding layer of the initially trained language model; generating an embedding vector for the second identifier sequence through a second embedding layer; and generating the encoded features based on the embedding vectors of the first and second identifier sequences. S04: Repeat steps S02 and S03 above until the first critical problem converges to generate the final critical problem; Specifically, the clustering of several key issues includes: S1-1: Let there be N primary critical issues; S1-2 will focus on one of the N primary critical issues related to equipment application through the hard attention mechanism model module; S1-3: Learn the vector of the first key question into the two-dimensional intermediate layer of the self-competitive neural network; S1-4 inputs the vectors of N-1 questions (excluding one of the N first key questions) into the input layer of the self-competitive neural network. The self-competitive neural network clusters the N-1 questions (excluding one of the N first key questions). The k questions whose distance from the vector of the N-1 key questions (excluding one of the N first key questions) to the vector of one question in the intermediate layer is less than or equal to a set value are grouped into the same class as that one question. The k key questions grouped into that one question are then removed from the N-1 questions to obtain Nk-1 questions that have not yet been clustered. S1-5: Determine if the value is Nk-1 or zero. If yes, end the classification; otherwise, proceed to step S1-6. S1-6; N←Nk-1, and return to step S1-2.
6. A key question generation method based on an attention mechanism, characterized in that, include: S01: Establish a fixed vocabulary, and use the words in the fixed vocabulary to train the language model containing the attention mechanism to obtain the initial training language model; S02: Extract several key questions from the predicted mask words through initial training of the language model; S03: Cluster the multiple primary key questions, distribute the clustering results to each member of the expert group, and have each member of the expert group add, delete, and modify the clustered key questions to obtain multiple secondary key questions; use the added, deleted, and modified secondary key questions as training statements; The encoding features of the training statement are generated based on a first mask that randomly masks words not recorded in a fixed vocabulary that are included in the training statement. Generate predicted mask words based on the encoded features; Generating the encoded features includes: generating an identifier sequence for the training statement based on a fixed vocabulary and an unrecorded vocabulary in the training statement that does not overlap with the fixed vocabulary; generating a first identifier sequence that does not contain the unrecorded words and a second identifier sequence that contains the unrecorded words based on a first mask; generating an embedding vector for the first identifier sequence through a first embedding layer of the initially trained language model; generating an embedding vector for the second identifier sequence through a second embedding layer; and generating the encoded features based on the embedding vectors of the first and second identifier sequences. S04: Repeat steps S02 and S03 above until the first critical problem converges to generate the final critical problem; Specifically, the clustering of several key issues includes: S2-1: Let there be N primary critical issues; S2-2: Focus on k questions related to equipment application from multiple primary key questions through a hard attention mechanism model module; S2-3: The vectors of k problems are learned into the two-dimensional intermediate layer of the self-competitive neural network; S2-4 inputs the vectors of Nk problems other than k problems out of N first key problems into the input layer of the self-competitive neural network in turn. The self-competitive neural network performs clustering. Problems whose distances from the vectors of the Nk first key problems to the vectors of the k problems in the intermediate layer are less than or equal to a set value are respectively classified into the same class of the k problems. Then, m problems that can be classified into the k problems are removed from the Nk problems to obtain Nkm problems that have not yet been clustered. S2-5: Determine if Nkm is zero. If yes, end the classification; otherwise, proceed to step S1-6. S2-6; N←Nkm, and return to step S2-2.
7. A key question generation system based on an attention mechanism, comprising: A storage medium and a processor, wherein a computer program is stored on the storage medium; the processor invokes the computer program to implement the key question generation method based on the attention mechanism as described in any one of claims 1-6.
8. A storage medium, characterized in that, It stores a computer program that is invoked by a processor to implement the attention-based key question generation method according to any one of claims 1-6.