Generalized large model reasoning method and system for connotation quality control and storage medium

Abstract

The invention belongs to the technical field of medical data processing, and particularly relates to a generalization large model reasoning method and system for connotation quality control and a storage medium. The system of the present invention comprises: an input module configured to input text data; the reasoning module is configured to input the text data into the large model to obtain a first reasoning result; the causal relationship prediction module is configured to input the text data into the causal relationship graph model to obtain an implicit relationship and / or causal connection between entities in the data; and the integrated reasoning module is configured to introduce the implicit relationship and / or the causal connection into a large model as knowledge, and perform integrated reasoning of the model in combination with the first reasoning result to obtain a second reasoning result. The invention further provides a method for realizing large model reasoning based on the system. The large model reasoning method and system provided by the invention have better generalization ability, can be suitable for wider reasoning prediction tasks, and have good application prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical data processing, and specifically relates to a generalized large model reasoning method, system and storage medium for connotation quality control. Background Art

[0002] With the rapid development and widespread application of machine learning and deep learning technologies, remarkable results have been achieved in various fields such as natural language processing, image recognition, and speech recognition. In order to obtain better performance, researchers continue to increase the scale and complexity of the model. However, as the complexity of the model increases, how to ensure that the model has good generalization ability when facing data sets with different distributions becomes a key challenge.

[0003] Generalization refers to the ability of a model to make accurate predictions on unseen, novel data. A model with good generalization can better cope with various situations in real applications, rather than just performing well on the training set. However, since real-world data is often diversely distributed and noisy, traditional model training methods (such as standard supervised learning) may cause the model to overfit the training data, resulting in poor performance on new datasets.

[0004] In order to improve the generalization ability of the model, researchers have introduced a variety of regularization techniques, such as weight decay, dropout, early stopping, data augmentation, batch normalization, etc. In addition, researchers have also tried model integration, cross-domain learning, transfer learning and other methods to improve the generalization of the model on different data sets. In the large model stage, researchers often use knowledge distillation to let the small model learn the soft labels of the teacher model, and inherit the hidden information about classification in the teacher model to ensure the generalization ability of the model.

[0005] Currently, the commonly used large model generalization methods are usually zero-shot or few-shot learning and dynamic adjustment of learning methods. Zero-shot learning and few-shot learning techniques are designed to solve the problem of performance degradation of traditional machine learning models when faced with scenarios where there is a lack of a large amount of labeled training data. Zero-shot learning is achieved by leveraging knowledge transfer and shared attributes or semantic relationships between categories, that is, mapping the learning experience of seen categories to unknown categories. Few-shot learning enables the model to effectively learn quickly from limited samples. Dynamic adjustment of learning methods means allowing the model to adaptively adjust its learning strategy based on the needs of the current task and the characteristics of the available data.

[0006] In the prior art, for the generalization research of large models, there has been no research and practice on the quality control of connotation (i.e. the causal relationship, structural characteristics or patterns implied behind the data). Connotation quality control refers to monitoring and guiding the model to capture the most essential and critical information in the data during the model training process. For example: Chinese patent application CN114298314A - A multi-granularity causal relationship reasoning method based on electronic medical records, the technical solution of this patent application applies causal relationship reasoning.

[0007] How to combine the intrinsic quality control method in the training and reasoning process of large models, so as to monitor and guide the model to capture the most essential and critical information in the data during the model training process to improve the generalization ability of the model, is still an important issue that needs to be urgently solved in this field. Summary of the invention

[0008] In view of the problems of the prior art, the present invention provides a generalized large model reasoning method, system and storage medium for connotation quality control.

[0009] A generalized large model reasoning system for connotation quality control, including: An input module configured to input text data; An inference module, configured to input the text data into a large model to obtain a first inference result; A causal relationship prediction module is configured to input the text data into a causal relationship graph model to obtain implicit relationships and / or causal connections between entities in the data; The integrated reasoning module is configured to introduce the implicit relationship and / or causal connection as knowledge into the large model, combine the first reasoning result, perform integrated reasoning of the model, and obtain a second reasoning result.

[0010] Preferably, the causal relationship graph model is a graph neural network.

[0011] Preferably, the text data is medical record data, and in the causal relationship graph model, the nodes of the graph are medical entities, and the edges represent implicit relationships and / or causal connections between the nodes.

[0012] Preferably, the algorithm of the large model is selected from a generative large model that can be used for training based on the Transformer architecture.

[0013] Preferably, in the causal relationship prediction module, the text data is input into the causal relationship graph model after word embedding through the embedding layer; and / or the implicit relationship and / or causal connection is injected into the large model as reference information for the solution after passing through a gating mechanism.

[0014] Preferably, it also includes: An output module is configured to output the second reasoning result.

[0015] Preferably, it also includes: The training module is configured to train the large model and the causal graph model according to the following process: Input training data, wherein the training data includes connotation quality control supervision training data and causal relationship data annotated based on connotation quality control data; The training data is embedded into words through the embedding layer of the large model; The connotation quality control supervision training data is used for training a large model after word embedding; After word embedding, the causal relationship data is used to train a causal relationship graph model.

[0016] Preferably, the training loss of the large model and the causal graph model is obtained by weighting the CELoss of the large model and the BCELoss of the causal graph model in a multi-task manner.

[0017] The present invention also provides a generalized large model reasoning method for connotation quality control, and the generalized large model reasoning system for connotation quality control is used to perform the following steps: Input text data; Inputting the text data into a large model to obtain a first inference result; Inputting the text data into a causal graph model to obtain implicit relationships and / or causal connections between entities in the data; The implicit relationship and / or causal connection is introduced into the large model as knowledge, and combined with the first reasoning result, integrated reasoning of the model is performed to obtain the second reasoning result.

[0018] The present invention also provides a computer-readable storage medium, on which is stored: a computer program for implementing the generalized large model reasoning system for the above-mentioned connotation quality control, or a computer program for implementing the generalized large model reasoning method for the above-mentioned connotation quality control.

[0019] In order to improve the generalization ability of the large model, the present invention combines the large model with the causal graph model to provide a new model architecture. In the preferred embodiment, the present invention further provides optimization of the training data construction method and the training mechanism optimization. Based on the model architecture and its optimization, the following beneficial technical effects can be achieved: 1. The present invention strengthens the ability of large models to identify causal relationships in data. Compared with traditional association analysis, which only reveals the synchronous change trend between variables, the real causal inference in the present invention seeks to understand the potential dependencies and mechanisms of action between variables. By fully understanding and accurately modeling these causal effects, the model can distinguish which patterns are the factors that really affect the results and which are just accompanying phenomena when facing data. Causal relationships can help the model understand the dependencies between variables, which can better handle external changes when predicting.

[0020] 2. This invention learns the structured representation in the data, so that the model can capture the intrinsic structure of the data (such as syntax tree, graph structure, time series, etc.) to understand the true meaning of the data. By capturing the intrinsic structure of the data, the model can obtain the ability to capture complex data features, providing a more in-depth perspective for different types of data decisions.

[0021] 3. The present invention optimizes the conceptual abstraction and reasoning ability of the model, improves the model's ability in conceptual abstraction, and helps them develop a hierarchical knowledge system in which the underlying specific instances support the high-level abstract concepts. Based on the model architecture of the present invention, the algorithm can continuously improve its internal parameters to identify and integrate the essential connections between different instances, thereby generating applicable knowledge that can be applied across contexts, extracting general concepts from specific instances and applying these concepts in different contexts.

[0022] In the present invention, the numerical numbers "first" and "second" are used to distinguish terms, which do not indicate a certain order or priority, and have no limiting effect on the scope of the technical features represented by the terms.

[0023] Obviously, according to the above contents of the present invention, in accordance with common technical knowledge and customary means in the art, without departing from the above basic technical ideas of the present invention, other various forms of modification, replacement or change may be made.

[0024] The above contents of the present invention are further described in detail below through specific implementation methods in the form of embodiments. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following examples. All technologies realized based on the above contents of the present invention belong to the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a schematic diagram of the model training process in Example 1 of the present invention; Figure 2 Schematic diagram of the model reasoning process in Example 1 of the present invention. DETAILED DESCRIPTION

[0026] It should be noted that the algorithms of data collection, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structure, circuit connection, etc. not specifically described can all be implemented through the contents disclosed in the prior art.

[0027] Example 1 Generalized large model reasoning method and system for connotation quality control This embodiment provides a large model reasoning method and system with strong generalization capability, and the system includes: An input module configured to input text data; The inference module is configured to input the text data into the big model to obtain a first inference result; in this embodiment, the big model algorithm is selected from a generative big model that can be used for training based on the Transformer architecture, such as the qwen series; A causal relationship prediction module is configured to input the text data into a causal relationship graph model to obtain implicit relationships and / or causal connections between entities in the data; An integrated reasoning module is configured to introduce the implicit relationship and / or causal connection as knowledge into the large model, and perform integrated reasoning of the model in combination with the first reasoning result to obtain a second reasoning result; an output module, configured to output the second reasoning result; The training module is configured to train large models and causal graph models.

[0028] The method of continuing large model training and reasoning based on the above system is as follows Figure 1 and Figure 2 shown.

[0029] Figure 1 The training process of this embodiment is summarized. The training data of the model includes two types of data, connotation quality control supervision training data and causal relationship data based on connotation quality control data annotation. Both types of data are embedded in the embedding layer of the large model. Then, the connotation quality control supervision training data after word embedding goes through the large model SFT training process, and the causal relationship data after word embedding goes through the causal relationship probability graph training process. The loss of the overall model training is obtained by weighting the CELoss of the large model and the BCELoss of the causal relationship graph model in a multi-task manner. This method uses a multi-task learning method to simultaneously optimize the objective functions of structure recognition and causal reasoning, and uses integrated reasoning to improve the robustness of the model in identifying deep structures and causal reasoning.

[0030] Figure 2The reasoning process of the method is summarized. The connotation quality control test data (i.e. medical record data) is passed through the big model to obtain the reasoning results of the big model for medical records. At the same time, the connotation quality control test data is passed through the causal relationship graph model to obtain the implicit relationship and causal connection between entities in the data. The implicit relationship given by the causal relationship graph is introduced into the big model as medical knowledge for model integration reasoning to obtain the final output of the big model.

[0031] Specifically, the data set, model architecture, and training mechanism are constructed as follows: 1. Build a dataset When building an inpatient medical record dataset for connotation quality control, the goal is to annotate the cause-effect relationships in the text so that the model can better learn and infer the cause, treatment process, and efficacy. This will involve carefully analyzing medical texts, identifying factors such as diseases, symptoms, test results, drugs, and treatment measures, and annotating the cause-effect relationships between these factors. This step is completed by hospital experts (such as doctors) based on clinical experience. The following are detailed steps and a simplified example: 1) Medical records contain a large number of professional terms and abbreviations. Unstandardized terms should be mapped to standard terms in medical dictionaries or ontologies.

[0032] 2) Use named entity recognition technology to extract key entities in medical records, such as symptoms, drugs, disease names, treatment measures, etc.

[0033] 3) Identify the types of relationships in the text, such as the relationship between symptoms and diseases, the relationship between drugs and treatment effects, etc.

[0034] 4) Hospital experts manually annotate the relationships in the medical records, indicating the causal relationships, including the causal direction.

[0035] 5) Construct causal data negative samples based on pseudo data construction, data splicing, etc.

[0036] Suppose there is a hospital medical record text: "The patient was diagnosed with acute myocardial infarction due to persistent chest pain. After treatment with aspirin, the patient's symptoms improved." In this example, the following entities and causal relationships need to be identified and annotated: Entity: Persistent chest pain, acute myocardial infarction, aspirin, symptom relief.

[0037] Cause and effect: persistent chest pain -> acute myocardial infarction (symptom -> diagnosis); aspirin -> symptom relief (treatment -> outcome).

[0038] The labeled dataset should clearly indicate these entities and the causal relationships between them, and the direction should be correct. For example: Entity annotation: "Persistent chest pain" [symptom] was diagnosed as "acute myocardial infarction" [disease]. After treatment with "aspirin" [drug], the patient's "symptoms" [symptoms] were "relieved" [result].

[0039] Causal relationship annotation: (persistent chest pain, trigger, acute myocardial infarction), (aspirin, relief, symptoms).

[0040] 2. Model Architecture A graph neural network is used to construct a causal graph model. The input of the causal graph model is medical causal data with a graph structure. The nodes of the graph are medical entities (such as symptoms, treatment measures, drugs, etc.), and the edges represent the potential relationship between the nodes. Each node and edge is represented by a feature vector, such as text embedding, entity attributes, etc. The output of the model is a prediction of whether there is an implicit relationship and / or causal connection between each pair of entities, as well as the possible causal directionality. In the setting of multi-label classification (i.e., there are multiple possible implicit relationships and / or causal connections at the same time), the model may also need to output the relationship type. The loss function of the model uses a binary cross entropy loss, that is, to determine whether there is a causal relationship between the entity pair. For example, given an entity pair (A, B) and a label y∈{0,1}, where 1 indicates the existence of an implicit relationship or causal connection and 0 indicates the absence of an implicit relationship or causal connection, the cross entropy loss between the model output (predicted probability) and the true label is calculated.

[0041] 3. Training mechanism In order to jointly train the large model and the causal graph model, this embodiment designs a strategy that can simultaneously optimize the autoregressive language modeling and causal relationship identification tasks. The first is the data flow design. The data flow is created in the computational graph of the model. The text is diverted after passing through the embedding layer of the large model. The embedded text representation is trained for autoregressive language modeling in the large model part, and the probability learning training of causal classification is performed in the causal graph model part. The input of the causal graph model part is injected into the large model after a gating mechanism as reference information for the answer to improve the model's understanding of medical expertise, thereby improving the model's generalization ability. The two modules are trained using a multi-task learning strategy, and the training loss is weighted by the CELoss of the large model autoregressive training and the binary cross entropy loss of causality.

[0042] It can be seen from the above embodiments that the present invention provides a generalized large-model inference method and system for connotation quality control. Based on the optimization of the model architecture, as well as the optimization of the data set construction method and the training mechanism in the preferred method, the large-model inference method and system provided by the present invention have better generalization capabilities, can be applied to a wider range of reasoning and prediction tasks, and have good application prospects.

Claims

1. A generalized large model reasoning system for connotation quality control, characterized by: include: An input module configured to input text data; An inference module, configured to input the text data into a large model to obtain a first inference result; A causal relationship prediction module is configured to input the text data into a causal relationship graph model to obtain implicit relationships and / or causal connections between entities in the data; The integrated reasoning module is configured to introduce the implicit relationship and / or causal connection as knowledge into the large model, combine the first reasoning result, perform integrated reasoning of the model, and obtain a second reasoning result.

2. The generalized large model reasoning system for connotation quality control according to claim 1, characterized in that: The causal relationship graph model is a graph neural network.

3. The generalized large model reasoning system for connotation quality control according to claim 1 or 2, characterized in that: The text data is medical record data. In the causal relationship graph model, the nodes of the graph are medical entities, and the edges represent the implicit relationships and / or causal connections between the nodes.

4. The generalized large model reasoning system for connotation quality control according to claim 1, characterized in that: The algorithm of the large model is selected from a generative large model that can be used for training based on the Transformer architecture.

5. The generalized large model reasoning system for connotation quality control according to claim 1, characterized in that: In the causal relationship prediction module, the text data is input into the causal relationship graph model after word embedding through the embedding layer; and / or the implicit relationship and / or causal connection is injected into the large model as reference information for the solution after passing through a gating mechanism.

6. The generalized large model reasoning system for connotation quality control according to claim 1, characterized in that: Also includes: An output module is configured to output the second reasoning result.

7. The generalized large model reasoning system for connotation quality control according to claim 1, characterized in that: Also includes: The training module is configured to train the large model and the causal graph model according to the following process: Input training data, wherein the training data includes connotation quality control supervision training data and causal relationship data annotated based on connotation quality control data; The training data is embedded into words through the embedding layer of the large model; The connotation quality control supervision training data is used for training a large model after word embedding; After word embedding, the causal relationship data is used to train a causal relationship graph model.

8. The generalized large model reasoning system for connotation quality control according to claim 7, characterized in that: The training loss of the large model and the causal graph model is obtained by weighting the CELoss of the large model and the BCELoss of the causal graph model in a multi-task manner.

9. A generalized large model reasoning method for connotation quality control, characterized in that: The generalized large model reasoning system for connotation quality control according to any one of claims 1 to 8 is applied to perform the following steps: Input text data; Inputting the text data into a large model to obtain a first inference result; Inputting the text data into a causal graph model to obtain implicit relationships and / or causal connections between entities in the data; The implicit relationship and / or causal connection is introduced into the large model as knowledge, and combined with the first reasoning result, integrated reasoning of the model is performed to obtain the second reasoning result.

10. A computer-readable storage medium, characterized in that: Stored thereon are: a computer program for implementing a generalized large model reasoning system for connotation quality control as described in any one of claims 1-8, or a computer program for implementing a generalized large model reasoning method for connotation quality control as described in claim 9.

Images