Artificial intelligence-based key problem intelligent fusion method and system, and storage medium
By combining fixed vocabulary encoding and self-competitive neural network clustering with expert correction, a key question fusion model was formed, which solved the problem of low accuracy of pre-trained language models in special domains and improved the accuracy and efficiency of expert question aggregation in the military field.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 63963 TROOP OF THE PLA
- Filing Date
- 2023-12-11
- Publication Date
- 2026-05-19
AI Technical Summary
Existing pre-trained language models have low accuracy in summarizing key issues in specific domains.
By training a language model with a fixed vocabulary, generating encoded features and predicted mask words, clustering is performed using a self-competitive neural network, and expert correction is combined to form a fusion model for key issues.
It improves the accuracy and efficiency of summarizing key issues and is suitable for summarizing questions raised by experts in the military field.
Smart Images
Figure CN117688178B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method, system, and storage medium for intelligent fusion of key issues based on artificial intelligence, belonging to the field of artificial intelligence technology. Background Technology
[0002] With the rapid development of deep learning and natural language processing technologies, language models trained with large amounts of labeled data can quickly and efficiently understand and analyze text data. They possess capabilities such as sentiment analysis, language translation, text classification, information extraction, and text generation. Through data-driven models, they can understand, analyze, and summarize document content and intelligently generate key questions.
[0003] However, there are domain differences between currently popular pre-trained language models and specific domains. Directly summarizing key questions raised by experts in specific domains using existing language models results in low accuracy. Summary of the Invention
[0004] To overcome the shortcomings of existing technologies, this invention provides a key issue intelligent fusion method, system, and storage medium based on artificial intelligence. It uses the key issues modified by various experts as training statements to train the initial training model, thereby improving the efficiency and accuracy of the aggregation.
[0005] To achieve the aforementioned objective, this invention provides a key problem fusion method based on artificial intelligence, comprising:
[0006] S01: Train the initial training language model by encoding words in a fixed vocabulary; extract training sentences from the equipment demonstration report, generate encoding features of the training sentences based on a first mask that randomly masks words not recorded in the fixed vocabulary that are included in the training sentences; generate predicted mask words based on the encoding features.
[0007] S02: Several key questions extracted from predicted mask words through initial training of the language model;
[0008] S03: Cluster multiple first key questions, distribute the clustering results to each member of the expert group, and have each member of the expert group correct the clustered key questions to obtain multiple second key questions; use the corrected multiple second key questions as training statements; generate the encoding features of the training statements based on the first mask of the training statements containing words not recorded in the fixed vocabulary, and generate predicted mask words based on the encoding features;
[0009] S04: Repeat steps S02 and S03 above until the first critical problem converges and the critical problem fusion model is obtained.
[0010] Preferably, generating the encoded features includes: generating an identifier sequence of the training statement based on a fixed vocabulary and an unrecorded vocabulary in the training statement that does not overlap with the fixed vocabulary; generating a first identifier sequence that does not contain the unrecorded words and a second identifier sequence that contains the unrecorded words in the identifier sequence based on the first mask; generating an embedding vector of the first identifier sequence through the first embedding layer of the pre-trained initial training language model; generating an embedding vector of the second identifier sequence through the second embedding layer of the pre-trained initial training language model; and generating the encoded features based on the embedding vectors of the first identifier sequence and the second identifier sequence.
[0011] Preferably, generating the encoded feature based on the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence includes: merging the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence to generate the embedding vector of the identifier sequence; and generating the encoded feature based on the embedding vector of the identifier sequence through an encoding layer.
[0012] Preferably, the key issue fusion method based on artificial intelligence further includes: standardizing the initial pre-trained language model after training into a key issue fusion model including a single embedding layer.
[0013] Preferably, merging the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence to generate the embedding vector of the identifier sequence includes: expanding the first mask to obtain a second mask; and merging the embedding vector of the recorded identifier sequence and the embedding vector of the unrecorded identifier sequence based on the second mask to generate the embedding vector of the identifier sequence.
[0014] Preferably, the language model is the T5 language model.
[0015] Preferably, clustering multiple primary key issues specifically includes:
[0016] S1-1: Let there be N primary critical issues;
[0017] S1-2 will focus on one of the N primary critical issues related to equipment application through the hard attention mechanism model module;
[0018] S1-3: Learn the vector of the first key question into the two-dimensional intermediate layer of the self-competitive neural network;
[0019] S1-4 inputs the vectors of N-1 questions (excluding one of the N first key questions) into the input layer of the self-competitive neural network. The self-competitive neural network clusters the N-1 questions (excluding one of the N first key questions). The k questions whose distance from the vector of the N-1 key questions (excluding one of the N first key questions) to the vector of one question in the intermediate layer is less than or equal to a set value are grouped into the same class as that one question. The k key questions grouped into that one question are then removed from the N-1 questions to obtain Nk-1 questions that have not yet been clustered.
[0020] S1-5: Determine if the value is Nk-1 or zero. If yes, end the classification; otherwise, proceed to step S1-6.
[0021] S1-6; N←Nk-1, and return to step S1-2.
[0022] Preferably, clustering multiple primary key issues specifically includes:
[0023] S2-1: Let there be N primary critical issues;
[0024] S2-2: Focus on k questions related to equipment application from multiple primary key questions through a hard attention mechanism model module;
[0025] S2-3: The vectors of k problems are learned into the two-dimensional intermediate layer of the self-competitive neural network;
[0026] S2-4 inputs the vectors of Nk problems other than k problems out of N first key problems into the input layer of the self-competitive neural network in turn. The self-competitive neural network performs clustering. Problems whose distances from the vectors of the Nk first key problems to the vectors of the k problems in the intermediate layer are less than or equal to a set value are respectively classified into the same class of the k problems. Then, m problems that can be classified into the k problems are removed from the Nk problems to obtain Nkm problems that have not yet been clustered.
[0027] S2-5: Determine if Nkm is zero. If yes, end the classification; otherwise, proceed to step S1-6.
[0028] S2-6; N←Nkm, and return to step S2-2.
[0029] To achieve the aforementioned objective, the present invention also provides a system for integrating key issues in artificial intelligence, comprising: a storage medium and a processor, wherein a computer program is stored on the storage medium; and the processor invokes the computer program to implement the above-described method.
[0030] To achieve the aforementioned objective, the present invention also provides a storage medium, characterized in that it stores a computer program thereon, the computer program being invoked by a processor to implement the above-described method.
[0031] Compared with the prior art, the present invention has the following beneficial effects:
[0032] This invention extracts multiple first key questions from statements contained in equipment demonstration reports using an initially trained language model. These first key questions are then clustered, and the clustering results are distributed to each member of an expert group. Each member of the expert group modifies the clustered key questions to obtain multiple second key questions. These modified second key questions are used as training statements. Encoding features of the training statements are generated based on a first mask that randomly masks words not recorded in a fixed vocabulary. Predictive mask words are generated based on these encoding features. The initially trained language model is then adjusted based on these predicted mask words to achieve the integration of technical issues raised by various experts through the language model, thus improving the efficiency and accuracy of the aggregation. Attached Figure Description
[0033] Figure 1 This is a flowchart of the intelligent fusion method for key issues based on artificial intelligence provided in the first embodiment of the present invention;
[0034] Figure 2 This is a flowchart of the clustering method provided in the first embodiment of the present invention;
[0035] Figure 3 This is a flowchart of the clustering method provided in the second embodiment of the present invention. Implementation
[0036] The preferred embodiments for implementing the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that these embodiments are merely illustrative and do not limit the scope of the invention. The computer program code for performing the operations of the embodiments of the present invention can be written in any combination of one or more programming languages. These programming languages may include object-oriented programming languages, such as Java, Smalltalk, C++, etc., or conventional procedural programming languages, such as C or similar programming languages. The method according to the present invention can be implemented by a circuit with corresponding functional configurations. The circuit includes a memory and a processor. Example
[0037] Figure 1 This is a flowchart of the intelligent fusion method for key issues based on artificial intelligence provided in the first embodiment of the present invention, as shown below. Figure 1 The method provided by the present invention includes:
[0038] S01: Train the initial training language model by encoding words in a fixed vocabulary; extract training sentences from the equipment demonstration report, generate encoding features of the training sentences based on a first mask that randomly masks words not recorded in the fixed vocabulary that are included in the training sentences; generate predicted mask words based on the encoding features, wherein the language model is a T5 model.
[0039] S02: Several key questions extracted from predicted mask words through initial training of the language model;
[0040] S03: Cluster multiple first key questions, distribute the clustering results to each member of the expert group, and have each member of the expert group correct the clustered key questions to obtain multiple second key questions; use the corrected multiple second key questions as training statements; generate the encoding features of the training statements based on the first mask of the training statements containing words not recorded in the fixed vocabulary, and generate predicted mask words based on the encoding features;
[0041] S04: Repeat steps S02 and S03 above until the model accuracy is achieved and the key issue fusion model is obtained.
[0042] The first embodiment improves the accuracy and efficiency of the language model in summarizing key questions provided by military experts by repeatedly training the initially trained language model with multiple modified second key questions as training statements.
[0043] In this invention, extracting training sentences from equipment demonstration reports includes: segmenting the equipment demonstration report into multiple sentences, segmenting each sentence into multiple words, encoding the words, their positions, and the sentences to generate a two-dimensional data matrix X. The first column of the two-dimensional data matrix X represents the sentence order encoding of the equipment demonstration report, the second column represents the word count encoding of each sentence, and the third to Nth columns represent the word encoding of each sentence and the word position encoding within the sentence. The two-dimensional data matrix is then input into a T5 model.
[0044] In this invention, generating the encoded features includes: generating an identifier sequence for the training statement based on a fixed vocabulary and an unrecorded vocabulary in the training statement that does not overlap with the fixed vocabulary; generating a first identifier sequence that does not contain the unrecorded words and a second identifier sequence that contains the unrecorded words in the identifier sequence based on a first mask; generating an embedding vector for the first identifier sequence through a first embedding layer of the pre-trained initial training language model; generating an embedding vector for the second identifier sequence through a second embedding layer; and generating the encoded features based on the embedding vectors of the first identifier sequence and the second identifier sequence.
[0045] In this invention, generating the encoded feature based on the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence includes: merging the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence to generate the embedding vector of the identifier sequence; and generating the encoded feature based on the embedding vector of the identifier sequence through an encoding layer.
[0046] The key issue fusion method based on artificial intelligence provided by the present invention further includes: standardizing the initial pre-trained language model after training into a key issue fusion model including a single embedding layer.
[0047] In this invention, merging the embedding vector of the first identifier sequence and the embedding vector of the second identifier sequence to generate the embedding vector of the identifier sequence includes: expanding the first mask to obtain a second mask; and merging the embedding vector of the recorded identifier sequence with the embedding vector of the unrecorded identifier sequence based on the second mask to generate the embedding vector of the identifier sequence.
[0048] Figure 2 This is a flowchart of the clustering method provided in the first embodiment of the present invention, as follows: Figure 2 As shown, in the first embodiment, clustering multiple key issues specifically includes:
[0049] S1-1: Let there be N primary critical issues;
[0050] S1-2 will focus on one of the N primary critical issues related to equipment application through the hard attention mechanism model module;
[0051] S1-3: Learn the vector of the first key question into the two-dimensional intermediate layer of the self-competitive neural network;
[0052] S1-4 inputs the vectors of N-1 questions (excluding one of the N first key questions) into the input layer of the self-competitive neural network. The self-competitive neural network clusters the N-1 questions (excluding one of the N first key questions). The k questions whose distance from the vector of the N-1 key questions (excluding one of the N first key questions) to the vector of one question in the intermediate layer is less than or equal to a set value are grouped into the same class as that one question. The k key questions grouped into that one question are then removed from the N-1 questions to obtain Nk-1 questions that have not yet been clustered.
[0053] S1-5: Determine if the value is Nk-1 or zero. If yes, end the classification; otherwise, proceed to step S1-6.
[0054] S1-6; N←Nk-1, and return to step S1-2.
[0055] The first embodiment focuses on key issues related to equipment application among multiple first key issues through a hard attention mechanism model module, and sequentially clusters them through a self-competitive neural network, thereby improving the scientificity and efficiency of classifying multiple key issues. Example
[0056] The parts of the key problem fusion method based on artificial intelligence provided in the second embodiment of the present invention that are the same as those in the first embodiment will not be described again; only the parts that are different will be described.
[0057] Figure 3 This is a flowchart of the clustering method provided in the second embodiment of the present invention, as follows: Figure 3 As shown, in the third embodiment, clustering multiple first key issues specifically includes:
[0058] S2-1: Let there be N primary critical issues;
[0059] S2-2: Focus on k questions related to equipment application from multiple primary key questions through a hard attention mechanism model module;
[0060] S2-3: The vectors of k problems are learned into the two-dimensional intermediate layer of the self-competitive neural network;
[0061] S2-4 inputs the vectors of Nk problems other than k problems out of N first key problems into the input layer of the self-competitive neural network in turn. The self-competitive neural network performs clustering. Problems whose distances from the vectors of the Nk first key problems to the vectors of the k problems in the intermediate layer are less than or equal to a set value are respectively classified into the same class of the k problems. Then, m problems that can be classified into the k problems are removed from the Nk problems to obtain Nkm problems that have not yet been clustered.
[0062] S2-5: Determine if Nkm is zero. If yes, end the classification; otherwise, proceed to step S1-6.
[0063] S2-6; N←Nkm, and return to step S2-2.
[0064] The second embodiment focuses on the k key issues related to equipment application among multiple first key issues through a hard attention mechanism model module, and further improves the scientificity and efficiency of classifying multiple key issues through self-competitive neural network clustering. Example
[0065] The parts of the key problem fusion method based on artificial intelligence provided in the third embodiment of the present invention that are the same as those in the first embodiment will not be described again; only the differences will be described. In the third embodiment, clustering N first key problems specifically includes:
[0066] S3-1 Randomly select k first key questions as initial cluster centers;
[0067] S3-2 calculates the semantic similarity of each primary key question to its k cluster centers and assigns it to the class corresponding to the cluster center with the highest semantic similarity.
[0068] S3-3 For each category, recalculate its cluster center by calculating the mean of the semantic vectors of all questions in each category, and use it as its new cluster center;
[0069] S3-4: Repeat steps S3-2 to S3-3 above until the problems in each category no longer change.
[0070] The clustering method provided in the third embodiment is simple, computationally inefficient, and requires low hardware costs. Example
[0071] The present invention also provides a key problem fusion system based on artificial intelligence, comprising: a processor, a ROM, a RAM, and a storage medium, wherein the processor performs various processes based on a program stored in the ROM or a program sent from the storage medium to the RAM, the program including a computer program for implementing the methods of the first to third embodiments.
[0072] In the fourth embodiment, the key issue fusion system further includes a communication unit and an expert recognition unit. The communication unit is configured to allow the processor and the expert's terminal to communicate via a network. The expert recognition unit includes a CNN and a stitching unit. The CNN includes multiple layers of interleaved convolutional layers from level 1 to level N and pooling layers from level 1 to level N. The CNN input contains an image of the expert's eyes. The stitching unit obtains the outline of the expert's eyes from the image extracted from the image of level N pooling layer, and stitches together the expert's eyeball image extracted from level n pooling layer to form the expert's authentication identifier, where n is greater than or equal to 1 and less than N.
[0073] The fourth embodiment obtains a low-resolution outline image of the expert's eye through the expert recognition unit, while simultaneously acquiring a high-resolution image of the eyeball. In this way, the expert can be identified through the expert's eyes, preventing the leakage of the expert's identity through facial authentication. Example
[0074] The present invention also provides a storage medium characterized in that it stores a computer program thereon, the computer program being invoked by a processor to implement the methods of the first to third embodiments. The aforementioned storage medium may include, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., but is not limited thereto.
[0075] The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments. All modifications made to the present invention without departing from the spirit of the present invention are within the technical scope of the present invention.
Claims
1. A key problem fusion method based on artificial intelligence, characterized in that, S01: Train the initial training language model by encoding words in a fixed vocabulary; extract training sentences from the equipment demonstration report, and generate the encoding features of the training sentences based on a first mask that randomly masks words not recorded in the fixed vocabulary that are included in the training sentences. Generate predicted mask words based on the encoded features; S02: Extracting several key questions from predicted masked words through initial training of the language model; S03: Cluster the multiple primary key issues, distribute the clustering results to each member of the expert group, and have each member of the expert group revise the clustered key issues to obtain multiple secondary key issues; The revised second key questions were used as training statements; The encoding features of the training statement are generated based on a first mask that randomly masks words not recorded in a fixed vocabulary that are included in the training statement. Generate predicted mask words based on the encoded features; S04: Repeat steps S02 and S03 above until the model accuracy is achieved and the key issue fusion model is obtained; Specifically, the clustering of several key issues includes: S2-1: Let there be N primary critical issues; S2-2: Focus on k questions related to equipment application from multiple primary key questions through a hard attention mechanism model module; S2-3: The vectors of k problems are learned into the two-dimensional intermediate layer of the self-competitive neural network; S2-4 inputs the vectors of Nk problems other than k problems out of N first key problems into the output layer of the self-competitive neural network in sequence. The self-competitive neural network performs clustering. Problems whose distances from the vectors of the Nk first key problems to the vectors of the k problems in the intermediate layer are less than or equal to a set value are respectively classified into the same class of the k problems. Then, m problems that can be classified into the k problems are removed from the Nk problems to obtain Nkm problems that have not yet been clustered. S2-5: Determine if Nkm is zero. If yes, end the clustering; otherwise, proceed to step S2-6. S2-6; N←Nkm, and return to step S2-2.
2. The key issue fusion method based on artificial intelligence according to claim 1, characterized in that, The generation of the encoding features of the training statement includes: generating an identifier sequence of the training statement based on a fixed vocabulary and an unrecorded vocabulary in the training statement that does not overlap with the fixed vocabulary; generating a first identifier sequence that does not contain the unrecorded words and a second identifier sequence that contains the unrecorded words in the identifier sequence based on the first mask; generating an embedding vector of the first identifier sequence through the first embedding layer of the initial training language model; generating an embedding vector of the second identifier sequence through the second embedding layer of the initial training language model; and generating the encoding features based on the embedding vectors of the first identifier sequence and the second identifier sequence.
3. The key issue fusion method based on artificial intelligence according to claim 2, wherein, Generating the encoded features based on the embedding vectors of the first and second identifier sequences includes: merging the embedding vectors of the first and second identifier sequences to generate the embedding vector of the identifier sequence of the training statement; and generating the encoded features based on the embedding vector of the identifier sequence of the training statement through an encoding layer.
4. The key issue fusion method based on artificial intelligence according to claim 3, characterized in that, Also includes: The initial trained language model is standardized into a key question fusion model including a single embedding layer after training is completed.
5. The key problem fusion method based on artificial intelligence according to any one of claims 1-4, characterized in that, The language model is the T5 language model.
6. A key problem fusion method based on artificial intelligence, characterized in that, include: S01: Train the initial language model by encoding the words in the fixed vocabulary; Training statements are extracted from the equipment demonstration report, and the encoding features of the training statements are generated based on a first mask that randomly masks words not recorded in a fixed vocabulary that are included in the training statements. Generate predicted mask words based on the encoded features; S02: Extracting several key questions from predicted masked words through initial training of the language model; S03: Cluster the multiple primary key issues, distribute the clustering results to each member of the expert group, and have each member of the expert group revise the clustered key issues to obtain multiple secondary key issues; The revised second key questions were used as training statements; The encoding features of the training statement are generated based on a first mask that randomly masks words not recorded in a fixed vocabulary that are included in the training statement. Generate predicted mask words based on the encoded features; S04: Repeat steps S02 and S03 above until the model accuracy is achieved and the key issue fusion model is obtained; Specifically, the clustering of several key issues includes: S1-1: Let there be N primary critical issues; S1-2 will focus on one of the N primary critical issues related to equipment application through the hard attention mechanism model module; S1-3: Learn the vector of the first key question into the two-dimensional intermediate layer of the self-competitive neural network; S1-4 inputs the vectors of the N-1 questions (excluding one of the N first key questions) sequentially into the output layer of the self-competitive neural network. The self-competitive neural network clusters the N-1 questions (excluding one of the N first key questions). The k questions whose distance from the vector of the N-1 key questions (excluding one of the N first key questions) to the vector of one question in the intermediate layer is less than or equal to a set value are grouped into the same category as one question. The k key questions that are grouped into one question are then removed from the N-1 questions to obtain the Nk-1 questions that have not yet been clustered. S1-5: Determine if Nk-1 is zero. If yes, end the clustering; otherwise, proceed to step S1-6. S1-6; N←Nk-1, and return to step S1-2.
7. A key problem fusion system based on artificial intelligence, comprising: A storage medium and a processor, wherein a computer program is stored on the storage medium; the processor invokes the computer program to implement the method described in any one of claims 1-6.
8. A storage medium, characterized in that, It contains a computer program that is invoked by a processor to implement the method described in any one of claims 1-6.