A large model text reverse reasoning enhancement method based on mask mechanism

Abstract

The present invention discloses a large-model text reverse reasoning enhancement method based on a masking mechanism, which relates to the field of text reasoning technology. The method comprises: using the ModernBERT model as the underlying architecture for fine-tuning training, constructing a large knowledge extraction model, performing fine-grained knowledge extraction, and determining the extracted knowledge; constructing a reverse reasoning instruction dataset based on the extracted knowledge; introducing a masking mechanism to construct a prompt word engineering based on reverse reasoning; and performing enhanced training on the large knowledge extraction model based on the reverse reasoning instruction dataset and the prompt word engineering to obtain a knowledge extraction enhancement model, wherein the knowledge extraction enhancement model outputs reverse reasoning after masking the text. The present invention solves the technical problem that the prior art lacks effective reverse reasoning verification capabilities in knowledge extraction and text reasoning tasks, resulting in low accuracy of reasoning results, thereby achieving the technical effect of improving the reverse reasoning verification capabilities of the model in text reasoning and improving the accuracy of the reasoning results.

Description

Technical Field

[0001] The present invention relates to the field of text reasoning technology, and in particular to a large-model text reverse reasoning enhancement method based on a mask mechanism. Background Art

[0002] With the rapid development of natural language processing technology, text reasoning and knowledge extraction based on large-scale pre-trained language models have been widely used in many fields, especially in the automated analysis of scientific and technological literature, greatly improving work efficiency. However, existing large-model reasoning methods still face significant challenges when dealing with complex causal relationships and reasoning tasks in scientific and technological literature. Current reasoning technologies generally rely on one-way causal inference, that is, one-way inference from known input conditions to reasoning results. This method lacks effective reverse reasoning verification capabilities. Therefore, when the model generates an inference result, it is impossible to backtrack and verify its rationality through the existing inference results, resulting in low accuracy of the inference result. Summary of the Invention

[0003] This application provides a large-model text reverse reasoning enhancement method based on a mask mechanism, which is used to solve the technical problem that the existing technology lacks effective reverse reasoning verification capabilities in knowledge extraction and text reasoning tasks, resulting in low accuracy of reasoning results.

[0004] In view of the above problems, this application provides a large model text reverse reasoning enhancement method based on a mask mechanism.

[0005] The present application provides a large-model text reverse reasoning enhancement method based on a mask mechanism, the method comprising:

[0006] Fine-tuning training is performed using the ModernBERT model as the underlying architecture to construct a large knowledge extraction model, and fine-grained knowledge extraction is performed on scientific and technological literature to determine the extracted knowledge. Based on the extracted knowledge, a reverse reasoning instruction dataset is constructed, wherein the reverse reasoning instruction dataset is composed of triples. A masking mechanism is introduced to construct a prompt word engineering based on reverse reasoning, wherein the masking mechanism is cause masking and result retention, and the prompt word engineering is a mapping of masked sentences before and after masking to original sentences. Based on the reverse reasoning instruction dataset and the prompt word engineering, the large knowledge extraction model is enhanced trained to obtain a knowledge extraction enhancement model, wherein the knowledge extraction enhancement model outputs reverse reasoning after text masking.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0008] This application uses the ModernBERT model as the underlying architecture for fine-tuning training, constructs a large model for knowledge extraction, performs fine-grained knowledge extraction on scientific and technological literature, and determines the extracted knowledge; based on the extracted knowledge, constructs a reverse reasoning instruction dataset, wherein the reverse reasoning instruction dataset is composed of triples; introduces a masking mechanism to construct a prompt word engineering based on reverse reasoning, wherein the masking mechanism is cause masking and result retention, and the prompt word engineering is a mapping of masked sentences and original sentences before and after masking; based on the reverse reasoning instruction dataset and the prompt word engineering, the large model for knowledge extraction is enhanced and trained to obtain a knowledge extraction enhanced model, wherein the knowledge extraction enhanced model outputs reverse reasoning after text masking. The present invention solves the technical problem that the existing technology lacks effective reverse reasoning verification capabilities in knowledge extraction and text reasoning tasks, resulting in low accuracy of reasoning results. By introducing a masking mechanism and the construction and enhanced training of a reverse reasoning instruction dataset, the reverse reasoning verification capability of the model in text reasoning is improved, and the accuracy of the reasoning results is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flowchart of a large-model text reverse reasoning enhancement method based on a mask mechanism provided in an embodiment of the present application;

[0011] Figure 2 A flowchart of constructing a large knowledge extraction model in a large model text reverse reasoning enhancement method based on a mask mechanism provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] This application provides a large-model text reverse reasoning enhancement method based on a mask mechanism, which is used to solve the technical problem that the existing technology lacks effective reverse reasoning verification capabilities in knowledge extraction and text reasoning tasks, resulting in low accuracy of reasoning results. By introducing a mask mechanism and the construction and enhanced training of a reverse reasoning instruction data set, the technical effect of improving the model's reverse reasoning verification capabilities in text reasoning and improving the accuracy of reasoning results is achieved.

[0013] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0014] It should be noted that any variations of the terms "include" and "have" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products or devices.

[0015] Examples, such as Figure 1 As shown, the present application provides a large model text reverse reasoning enhancement method based on a mask mechanism, the method comprising:

[0016] Step S100: Fine-tune the ModernBERT model as the underlying architecture, build a large knowledge extraction model, perform fine-grained knowledge extraction on scientific literature, and determine the extracted knowledge.

[0017] In an embodiment of the present application, the ModernBERT model is used as the underlying architecture for fine-tuning training. When constructing a large model for knowledge extraction, manually labeled samples containing multiple corpus types are first obtained. The ModernBERT model is fine-tuned and trained using these samples to generate a classification model for distinguishing different corpus types in scientific and technological literature. Then, based on the classification model, the scientific and technological literature is semi-automatically classified and labeled, and high-quality fine-tuning corpus is determined through manual verification and iterative optimization. Finally, the classification model is further trained with high-quality fine-tuning corpus to construct an accurate large model for knowledge extraction that can effectively identify and extract different types of knowledge.

[0018] Next, we conduct fine-grained knowledge extraction on scientific and technological literature. Starting from the established knowledge extraction model, we automatically identify and extract fine-grained knowledge from the target literature, such as research background, research objectives, research methods, and research results. Through this process, we determine the extracted knowledge.

[0019] Further, such as Figure 2 As shown, the method provided in the embodiment of the application uses the ModernBERT model as the underlying architecture for fine-tuning training to build a large knowledge extraction model, and also includes:

[0020] Acquire corpus identification samples, wherein the corpus identification samples are manually identified samples and include multiple corpus types; based on the corpus identification samples, train and fine-tune the ModernBERT model to determine a classification model, wherein the classification model is used to classify and identify corpus types; based on the classification model, semi-automatic classification annotation of scientific and technological literature is used to verify and iterate the annotation results to determine high-quality fine-tuned corpus; based on the high-quality fine-tuned corpus, train and fine-tune the classification model to determine a large knowledge extraction model.

[0021] In an embodiment of the present application, corpus identification samples are first obtained from a preset database. These identification samples are manually annotated and contain various corpus types, such as research background, research purpose, research method, research results, etc.

[0022] Next, the ModernBERT model is fine-tuned based on these labeled corpus samples. ModernBERT is a pre-trained language model based on the Transformer architecture with powerful language comprehension capabilities. During fine-tuning, the model adjusts its internal parameters so that it can learn how to classify and identify document content based on the input samples. Specifically, fine-tuning is performed using a supervised learning approach. By inputting labeled samples and adjusting parameters based on the existing labels, the model learns how to distinguish different types of text. During fine-tuning, the backpropagation algorithm is used to optimize the model parameters, enabling it to identify and classify different parts of the document, such as the research background and research methods. Based on the training data, the model can automatically identify hidden semantic information in the text and match it with corresponding categories (such as research objectives and research results). The goal of fine-tuning training is to enable the model to automatically and accurately classify and extract relevant knowledge points when processing new documents. Through this step, a fine-tuned classification model is ultimately obtained, which is used to classify and identify corpus types.

[0023] Next, the documents are automatically annotated through the classification model. In this process, the classification model automatically classifies unannotated scientific and technological documents based on the knowledge it has learned. Specifically, by passing the input document into the classification model, the model classifies and predicts each paragraph or sentence in the document, marking it as a category such as "research background" and "research methods". However, since automatic annotation may have errors, the annotation results are manually verified and corrected, which is the process of semi-automatic classification and annotation. Manual verifiers verify the results of the model's automatic annotation to ensure its accuracy and correct inaccurate annotations. This process is carried out through iterative optimization to continuously improve the annotation accuracy and ultimately obtain high-quality annotated data. Through this process, high-quality fine-tuned corpus is finally obtained. This data has undergone multiple rounds of manual verification and optimization to ensure the accuracy and consistency of the annotation.

[0024] After obtaining these verified and corrected high-quality fine-tuned corpora, the classification model was trained and fine-tuned again. Using this high-quality annotated data, the model was retrained, further improving its classification accuracy. Specifically, by inputting the corrected corpus into the model for fine-tuning, the model learned how to better extract information from the literature and determine the correct category for each section based on the context. After multiple rounds of training, the model's accuracy gradually improved, ultimately constructing a highly efficient large-scale knowledge extraction model. Through this process, a model was ultimately obtained that can accurately identify and extract various fine-grained knowledge from scientific and technological literature. This model can effectively extract information such as research background, research objectives, research methods, and research results.

[0025] Step S200: constructing a reverse reasoning instruction dataset based on the extracted knowledge, wherein the reverse reasoning instruction dataset is composed of triples.

[0026] In an embodiment of the present application, when constructing a reverse reasoning instruction dataset based on the extracted knowledge, a triple form is adopted for organization. In this process, the representation structure of the dataset is first set, and the dataset is constructed using triples (instruction-input-output). Specifically, the instruction part defines the requirements of the reasoning task and the key basis of the task original text, the input part contains fine-grained information extracted from the extracted knowledge (such as research background, research methods, etc.), and the output part is the original text extracted by reasoning. Then, according to the representation structure, the knowledge structure of the extracted knowledge is adjusted to ensure that the relationship between each knowledge point and the reverse reasoning task is clear. Finally, a reverse reasoning instruction dataset presented in the form of triples is constructed. The dataset corresponds to a variety of corpus types, and at least covers research methods, research purposes, innovation points, and research value.

[0027] Furthermore, in the method provided in the embodiment of the application, constructing a reverse reasoning instruction dataset based on the extracted knowledge also includes:

[0028] The representation structure of the instruction adjustment data set is set, wherein the instruction-input-output triple structure is used as the representation structure; according to the representation structure, the knowledge structure of the extracted knowledge is adjusted to construct the reverse reasoning instruction data set, wherein the reverse reasoning instruction data set corresponds to multiple corpus types, including at least research method type, research purpose type, innovation point type and research value type.

[0029] Furthermore, the method provided in the application embodiment also includes:

[0030] The instruction contains the reasoning task definition and the key basis of the task original text; the input contains fine-grained knowledge; and the output contains the original text extraction sentence.

[0031] In this embodiment of the present application, when constructing a reverse reasoning instruction dataset based on extracted knowledge, the representation structure of the instruction-adjusted dataset is first set. To clearly express the requirements, input, and output of the reasoning task, a triple structure (instruction-input-output) is used to organize the data. This structure effectively and orderly expresses all aspects of the reasoning task, so that each item in the dataset clearly reflects the reasoning task, input, and output. Specifically, the instruction section defines the requirements of the reasoning task and the key basis of the task text. For example, the instruction might be "Infer the purpose of this research based on the following research background information." In this section, the task definition clearly defines the direction of reasoning and the goal to be achieved. By setting the instruction, the model is ensured to understand the type of reasoning work it needs to perform and the core basis of the reasoning task. The input section contains fine-grained knowledge extracted from the extracted knowledge. This knowledge point may include research background, research purpose, research methods, research results, etc. Fine-grained knowledge provides specific context for the reasoning task, helping the model understand the information needed during the reasoning process. For example, the "research background" extracted from the literature provides the basis for inferring the "research purpose." The output section consists of the original text extracted from the input through the reasoning task. This section reflects the final result of the reasoning task and is typically a specific sentence extracted from the literature. For example, the output might be "The purpose of this research is to improve system efficiency." This step results in a clearly defined representation structure, a triple structure, comprising clear reasoning task instructions, input knowledge, and expected output, laying the foundation for dataset construction.

[0032] Next, based on the representation structure, the extracted knowledge is structured to ensure that there is a clear correspondence between each knowledge point and the reasoning task. Specifically, different types of knowledge points (such as research methods, research objectives, innovation points, research value, etc.) are matched with reasoning tasks using manual annotation. Through annotation, these fine-grained knowledge points are organized into a clear structure to ensure that each knowledge point can play a role in the reverse reasoning task. For example, the research background can correspond to the research purpose reasoning task, and the research method can be associated with the innovation point. Through this structural adjustment, it is ensured that the data can accurately serve the reasoning task and that each knowledge point can be effectively utilized. The purpose of the knowledge structure adjustment is to make the connection between each knowledge point and the reasoning task closer, and to ensure that the data set can provide accurate and relevant information when performing reverse reasoning.

[0033] After completing the knowledge structure adjustment, a reverse reasoning instruction dataset conforming to the triple structure was ultimately constructed. This dataset contains multiple triples, each of which consists of instruction, input, and output. These triples form the data foundation for the reverse reasoning task. The instruction includes the definition of the reasoning task and the key basis of the task original text, clarifying the task objective; the input contains fine-grained knowledge, which provides the necessary contextual information for the reasoning task; and the output is the result of the reasoning task, which is the original text extracted from the input. In this way, the constructed reverse reasoning instruction dataset not only has a clear structure but also can support a variety of reasoning tasks, ensuring the accuracy and comprehensiveness of the data. These triple data items cover a variety of corpus types, including research methods, research objectives, innovation points, and research value.

[0034] Step S300: introducing a masking mechanism and constructing a prompt word engineering based on reverse reasoning, wherein the masking mechanism is to mask the cause and retain the result, and the prompt word engineering is a mapping of the masked sentence before and after the masking to the original sentence.

[0035] In an embodiment of the present application, a masking mechanism is introduced to construct a prompt word project based on reverse reasoning. First, a special marker is introduced, which is used to mask the cause part of the sentence, while the result part remains unchanged. The masking mechanism realizes the masking of causes and retention of results in this way, that is, the cause part of the sentence (such as "in order to improve efficiency") is replaced with a special marker (such as "[MASK]"), while the result part (such as "adopting new technology") remains unchanged. Next, based on the special marker, the original sentence in the scientific and technological literature is masked to obtain a masked sentence, that is, the incomplete text after the cause part is masked. Then, a mapping between the masked sentence and the original sentence is established, that is, by comparing the masked sentence and the original sentence, the correspondence between the masked part and the original sentence is determined, and finally a prompt word project is formed.

[0036] Furthermore, the method provided in the embodiment of the application introduces a masking mechanism to construct a prompt word engineering based on reverse reasoning, and further includes:

[0037] A special marker is introduced, wherein the special marker is used to partially mask the cause of a sentence; based on the special marker, sentence masking is performed on the scientific and technological literature to determine the masked sentence, wherein the masked sentence is an incomplete text; a mapping between the masked sentence and the original sentence is established to determine the prompt word project.

[0038] In an embodiment of the present application, a special marker is first introduced, the purpose of which is to mask the cause part in the sentence to ensure that only the result part is focused on during the reasoning process. By using a marker (such as [MASK]), the cause part in the text (for example, "in order to improve efficiency") is replaced, while the result part (for example, "the study adopted new technologies") remains unchanged. In this way, the cause part is masked, and the reasoning task still revolves around the complete causal relationship. Then, after the special marker is introduced, the scientific and technological literature is sentence masked, that is, the cause part in the literature is replaced with [MASK] to generate a masked sentence. For example, the original text "In order to improve efficiency, the study adopted new technologies" becomes "In order to [MASK], the study adopted new technologies" after masking. Through this masking process, the model can focus on the causal reasoning task and concentrate it on reasoning about the causal relationship of the masked part.

[0039] Next, a mapping between the masked statement and the original statement is established to clarify the relationship between the two. Specifically, through alignment, the masked statement is compared with the original text to ensure that each [MASK] marker corresponds to the cause portion of the original statement. This mapping relationship ensures that the masked portion and the original text are correctly aligned, thereby helping the reasoning process accurately restore the obscured content. Through these steps, a prompt word project based on reverse reasoning is constructed. The mapping relationship between the masked statement and the original statement ensures the smooth progress of the reasoning task, allowing the reasoning of causal relationships to establish an effective connection between the masked portion and the known result, thereby promoting the completion of the reasoning task.

[0040] Step S400: Based on the reverse reasoning instruction dataset and the prompt word project, the knowledge extraction large model is enhanced and trained to obtain a knowledge extraction enhancement model, wherein the knowledge extraction enhancement model outputs reverse reasoning after text masking.

[0041] In an embodiment of the present application, when the knowledge extraction large model is enhanced and trained based on the reverse reasoning instruction data set and the prompt word engineering, the knowledge extraction large model is first enhanced and learned using the reasoning instruction data set to obtain a first-order training model. In this stage, the parameters of the model are adjusted so that it can understand and perform reasoning tasks and learn how to extract effective information from given inputs. Next, based on the prompt word engineering, the first-order training model is subjected to masked reasoning and comparison iterative training. In this process, the masking mechanism masks the cause part in the input so that the model can deduce the masked information from the result part. Through multiple rounds of iterative training, the reasoning ability of the model is enhanced, and a knowledge extraction enhanced model is finally generated. The enhanced model uses reverse reasoning after text masking as output, and can deduce accurate knowledge content based on the input masked sentences, especially in reverse reasoning tasks, ensuring that the masked causal relationship is restored from the masked text.

[0042] Furthermore, in the method provided in the embodiment of the application, based on the reverse reasoning instruction dataset and the prompt word project, the knowledge extraction large model is enhanced and trained to obtain the knowledge extraction enhanced model, which also includes:

[0043] According to the inference instruction data set, the knowledge extraction large model is subjected to enhanced learning to determine a first-order training model; according to the prompt word project, the first-order training model is subjected to masked reasoning and comparison iterative training to generate the knowledge extraction enhanced model.

[0044] In an embodiment of the present application, the knowledge extraction large model is first enhanced by using an inference instruction dataset. The inference instruction dataset contains multiple triple structures, where instruction defines the requirements of the reasoning task and the key basis of the task text, input is fine-grained knowledge extracted from the literature, and output is the reasoning result. To perform enhanced learning, the data is input into the knowledge extraction large model, which is based on the Transformer architecture and is suitable for natural language processing tasks. At this stage, the knowledge extraction large model uses supervised learning methods to learn how to generate corresponding reasoning results based on the input knowledge. This process uses the backpropagation algorithm to optimize the parameters of the model, so that the model can identify the relationship between input and output during training, and gradually improve the accuracy of reasoning, and finally generate a first-order training model.

[0045] After obtaining the first-order trained model, it is further trained based on the prompt word engineering process. This process uses a masking mechanism to mask the cause portion of the original text and replace it with the [MASK] symbol, while leaving the result portion unchanged. In this step, the masked sentence is used as input, and the model's reasoning process generates an inference result. Specifically, the first-order trained model first receives the masked sentence and performs reasoning based on the context to generate the missing causal relationship. This reasoning process relies on the parameters obtained in the reinforcement learning phase. The model uses a self-attention mechanism to capture key information in the input and infer the masked content.

[0046] Next, the masked sentence is compared with the original sentence and semantic similarity is calculated to ensure that the inference result is consistent with the original text. Semantic similarity is typically measured by calculating cosine similarity, which represents the similarity between the inference result and the original sentence. If the semantic similarity is low, multiple rounds of iterative training are performed, in which the model adjusts its parameters to reduce the gap between the inference result and the original sentence. During each round of training, the model uses the calculated similarity feedback to correct the inference result of the masked sentence, gradually improving the accuracy and precision of the inference until the semantic similarity reaches the set standard.

[0047] Through the above-mentioned iterative training of masked reasoning and comparison, an enhanced knowledge extraction model is ultimately generated. This enhanced model can perform reverse reasoning from masked text to recover hidden causal relationships. The enhanced knowledge extraction model can deduce the cause of the causal relationship based on the result and generate relevant knowledge extraction results. This enhanced training enables the model to more accurately extract fine-grained knowledge, such as research objectives, research methods, and innovative points, helping to extract and infer important knowledge from scientific literature.

[0048] Furthermore, in the method provided in the embodiment of the application, the mask inference and comparison iterative training is performed on the first-order training model, further comprising:

[0049] The masked sentence is used as input, and reasoning is performed based on the first-order training model to determine the reasoning result; the masked sentence and the original sentence are proofread to determine the semantic similarity, wherein the semantic similarity is used as the proofreading standard; based on the semantic similarity, multiple rounds of iterative training are performed until convergence to obtain the knowledge extraction enhancement model.

[0050] In an embodiment of the present application, a masked statement is first used as input and fed into a first-order training model. A masked statement refers to a statement in which the cause part of the original text is replaced with the [MASK] symbol, while the result part remains unchanged. By inputting the masked statement, the first-order training model performs reasoning based on the context, predicts and generates the masked part, and the reasoning process mainly depends on the parameters obtained by the model in the reinforcement learning stage. At this point, the first-order training model uses its internal self-attention mechanism to perform a deep analysis of the input, thereby inferring the masked causal part and generating an inference result, which is the content that fills the [MASK] part.

[0051] Next, the generated inference results are proofread. Specifically, the masked sentence is compared with the original sentence, and the semantic similarity between the inference result and the original sentence is calculated. Semantic similarity is a measure of the consistency between the inference result and the original content, and is calculated using cosine similarity. If the inference result has a high degree of semantic consistency with the original sentence, the semantic similarity is high; conversely, if the semantic similarity is low, it indicates that the difference between the inference result and the original sentence is significant, and the model requires further optimization. At this stage, semantic similarity serves as a proofreading criterion to help determine whether the model's inference results meet accuracy requirements.

[0052] Based on the calculated semantic similarity, the model undergoes multiple rounds of iterative training. If the semantic similarity between the inference result and the original sentence meets the preset convergence criteria, the inference task for that round is completed and the final inference result is output. If the semantic similarity does not meet the expected standard, the model will proceed to revise the inference result. This process involves comparing the original sentence with the inference result and adjusting the content of the inference result to bring it closer to the original semantics. The revised inference result serves as the new input to the model for the next round of training.

[0053] Through continuous iterative training, the model's inference results are gradually optimized until the semantic similarity meets the set standard. In each iteration, the first-order training model adjusts its parameters based on the revised inference results to optimize the inference process. Through multiple rounds of iterative training, the model gradually improves inference accuracy, ultimately achieving a convergent inference process.

[0054] Finally, after multiple rounds of iterative training, a knowledge extraction enhancement model was generated. This model can perform reverse reasoning based on masked sentences, recovering obscured causal relationships and extracting fine-grained knowledge from scientific literature, such as research objectives, research methods, and innovations. Through reinforcement learning and iterative training, the model's reasoning precision and accuracy have been significantly improved, enabling it to better support knowledge extraction tasks.

[0055] In summary, the masked sentence is input into the first-order training model for inference, and then the semantic similarity is calculated to compare it with the original sentence. Finally, multiple rounds of iterative training are used to correct the inference result until the semantic similarity converges, thus generating a knowledge extraction enhancement model.

[0056] Furthermore, in the method provided in the embodiment of the application, multiple rounds of iterative training are performed until convergence based on the semantic similarity, further comprising:

[0057] Identify the semantic similarity and determine whether it meets the preset convergence degree; if so, use the inference result as output; if not, correct the inference result based on the original sentence to generate a corrected inference result; and iteratively train the first-order training model according to the corrected inference result until the semantic similarity is met.

[0058] In the embodiment of the present application, semantic similarity is first identified, that is, the semantic similarity between the inference result and the original sentence is calculated. The cosine similarity is used to compare the similarity between the inference result and the original sentence in the vector space to determine their consistency. If the calculated semantic similarity is greater than the preset convergence threshold, the inference result is considered to be sufficiently accurate and meets the preset convergence, and the inference result is directly output. If the semantic similarity is lower than the preset convergence, that is, lower than the preset convergence threshold, it indicates that there is a large gap between the inference result and the original text, and the model needs to be further optimized.

[0059] This is the stage of correcting the inference results. First, the inference results are compared and analyzed with the original statement to identify any inconsistencies between the inference results and the original. Using difference analysis methods, semantic discrepancies are located and corrected based on the content of the original statement. The correction process modifies the inconsistent content in the inference results to bring them closer to the semantics and structure of the original text. For example, if the inference results lack causal relationships or are unclear in their logic, the inference results are supplemented or adjusted based on the original statement to ensure greater accuracy. During the correction process, grammatical correction techniques are used to adjust the inference results to ensure the correctness and fluency of the language structure.

[0060] The revised inference results are then used as new inputs and fed into the first-order training model for iterative training. After each training session, the model updates its internal parameters using backpropagation and gradient descent to optimize the inference results. In each iteration, semantic similarity is calculated based on the revised inference results and compared with the original sentence. If the semantic similarity reaches the preset convergence level, the inference process ends and the final result is output. If the similarity still does not meet the standard, the inference results are revised again and the next round of training begins. This process continues through multiple iterations, gradually improving the model's inference precision and accuracy until the semantic similarity meets the preset convergence level.

[0061] Finally, through multiple rounds of iterative training and correction, a knowledge extraction enhanced model is generated.

[0062] In the embodiments of the present application, in summary, the embodiments of the present application have at least the following technical effects:

[0063] This application uses the ModernBERT model as the underlying architecture for fine-tuning training, constructs a large model for knowledge extraction, performs fine-grained knowledge extraction on scientific and technological literature, and determines the extracted knowledge; based on the extracted knowledge, constructs a reverse reasoning instruction dataset, wherein the reverse reasoning instruction dataset is composed of triples; introduces a masking mechanism to construct a prompt word engineering based on reverse reasoning, wherein the masking mechanism is cause masking and result retention, and the prompt word engineering is a mapping of masked sentences and original sentences before and after masking; based on the reverse reasoning instruction dataset and the prompt word engineering, the large model for knowledge extraction is enhanced and trained to obtain a knowledge extraction enhanced model, wherein the knowledge extraction enhanced model outputs reverse reasoning after text masking. The present invention solves the technical problem that the existing technology lacks effective reverse reasoning verification capabilities in knowledge extraction and text reasoning tasks, resulting in low accuracy of reasoning results. By introducing a masking mechanism and the construction and enhanced training of a reverse reasoning instruction dataset, the reverse reasoning verification capability of the model in text reasoning is improved, and the accuracy of the reasoning results is improved.

[0064] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0065] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0066] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. A large model text reverse reasoning enhancement method based on a mask mechanism, characterized by: The method comprises: Using the ModernBERT model as the underlying architecture, we fine-tune and train it to build a large knowledge extraction model, extract fine-grained knowledge from scientific literature, and determine the extracted knowledge. Constructing a reverse reasoning instruction dataset based on the extracted knowledge, wherein the reverse reasoning instruction dataset is composed of triples; A masking mechanism is introduced to construct a prompt word engineering based on reverse reasoning. The masking mechanism masks the cause and retains the result, and the prompt word engineering is a mapping of the masked sentence before and after masking to the original sentence. Performing enhanced training on the knowledge extraction large model based on the reverse reasoning instruction dataset and the prompt word project to obtain a knowledge extraction enhanced model, wherein the knowledge extraction enhanced model outputs reverse reasoning after text masking; Based on the extracted knowledge, a reverse reasoning instruction dataset is constructed, including: Set the instruction to adjust the representation structure of the data set, where the triple structure of instruction-input-output is used as the representation structure; According to the representation structure, the extracted knowledge is adjusted to construct the reverse reasoning instruction dataset, wherein the reverse reasoning instruction dataset corresponds to multiple corpus types, including at least research method type, research purpose type, innovation point type, and research value type; The instruction contains the reasoning task definition and the key basis of the task original text; the input contains fine-grained knowledge; and the output contains the original text extraction sentence.

2. A large model text reverse reasoning enhancement method based on a mask mechanism as claimed in claim 1, characterized in that: Using the ModernBERT model as the underlying architecture, we fine-tune and train a large knowledge extraction model, including: Obtaining a corpus identification sample, wherein the corpus identification sample is a manually identified sample and includes multiple corpus types; Based on the corpus identification sample, the ModernBERT model is trained and fine-tuned to determine a classification model, wherein the classification model is used to classify and identify the corpus type; According to the classification model, semi-automatic classification annotation of scientific and technological literature is used to verify and iterate the annotation results to determine high-quality fine-tuning corpus; The classification model is trained and fine-tuned based on the high-quality fine-tuning corpus to determine a large knowledge extraction model.

3. The method for enhancing reverse reasoning of large model text based on a mask mechanism according to claim 1, characterized in that: Introducing a masking mechanism and building a prompt word engineering based on reverse reasoning, including: Introducing a special marker, wherein the special marker is used to partially mask the reason of the sentence; Perform sentence masking on the scientific document according to the special marker to determine a masked sentence, wherein the masked sentence is an incomplete text; A mapping between the masked sentence and the original sentence is established to determine the prompt word project.

4. The large model text reverse reasoning enhancement method based on the mask mechanism as claimed in claim 3, wherein the characteristic value is: According to the reverse reasoning instruction dataset and the prompt word project, the knowledge extraction large model is enhanced and trained to obtain a knowledge extraction enhanced model, including: Performing reinforcement learning on the knowledge extraction large model according to the inference instruction data set to determine a first-order training model; According to the prompt word project, mask reasoning and comparison iterative training are performed on the first-order training model to generate the knowledge extraction enhancement model.

5. The method for enhancing reverse reasoning of large model text based on a mask mechanism according to claim 4, characterized in that: The first-order training model is subjected to mask inference and comparison iterative training, including: Taking the masked sentence as input, performing reasoning based on the first-order training model, and determining a reasoning result; Proofreading the masked sentence and the original sentence to determine semantic similarity, wherein the semantic similarity is used as a proofreading criterion; Based on the semantic similarity, multiple rounds of iterative training are performed until convergence to obtain the knowledge extraction enhancement model.

6. A large model text reverse reasoning enhancement method based on a mask mechanism as claimed in claim 5, characterized in that: Based on the semantic similarity, multiple rounds of iterative training are performed until convergence, including: Identifying the semantic similarity and determining whether a preset convergence degree is satisfied; If satisfied, the reasoning result is used as output; If not, modify the inference result based on the original statement to generate a modified inference result; According to the modified inference result, the first-order training model is iteratively trained until the semantic similarity is satisfied.

Images