Corpus construction method and system for large medical question and answer model

Abstract

The invention relates to the field of data processing, in particular to a corpus construction method and system for a large medical question and answer model, and the method comprises the steps: constructing a triple knowledge graph based on an existing corpus, and then carrying out the matching of historical corpora of a plurality of information publishers in a network with the triple knowledge graph, therefore, the reliability of the plurality of information publishers in the historical information publishing process is determined, and the credible information publishers are screened out. And then, carrying out credibility verification on the historical corpus published by the information publisher by utilizing the triple knowledge graph, and if the published historical corpus accords with a rule in the triple knowledge graph, directly recording the published historical corpus into a current corpus. And if not, putting the data into an expansion library for cross validation or manual verification. According to the method and the device, the corpus can be expanded by utilizing the network corpus, and during expansion, the triple knowledge graph of the existing corpus is utilized to perform reliability verification on the information publisher and the corpus, so that the reliability of the expanded corpus is ensured.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a corpus construction method and system for a large medical question-and-answer model. Background Art

[0002] In order to deal with some questions from patients or their families, existing hospitals often set up terminal devices in the hospital lobby. These terminal devices have built-in question and answer models to answer some common questions for patients or their families.

[0003] For the medical Q&A model used to respond to patients or their families, since the corpus is needed as the basis of the language model, the corpus used is crucial, and the corpus needs to be as authentic and reliable as possible. Therefore, the corpus of existing corpora generally comes from authoritative materials such as medical textbooks and authoritative journals, which also leads to the corpus of existing medical Q&A models not covering a wide range, and the Q&A model built on this basis cannot respond to many problems. If the network corpus is directly introduced for expansion, unreliable data samples may be introduced, resulting in some low-quality answers in the Q&A model, resulting in a decrease in the reliability of the Q&A model. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a corpus construction method and system for a large medical question and answer model to solve the problems in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A method for constructing a corpus of a medical question-answering large model of the present invention comprises the steps of: Acquire a current corpus of the medical question-answering model and historical corpora published by multiple information publishers, wherein the current corpus includes multiple pieces of corpus data; Constructing a triple knowledge graph based on the multiple corpus data; Extracting triple entities of the historical corpus, and matching the triple entities of the historical corpus with the triple knowledge graph to obtain matching results, wherein the matching results include consistency, contradiction, and non-matching; Evaluating the confidence of the information publisher based on the matching result to obtain the confidence of multiple information publishers; An information publisher whose confidence is greater than a preset threshold is regarded as a trusted publisher, and the trusted corpus and the unconfirmed corpus of the trusted publisher are determined based on the triple knowledge graph, the trusted corpus of the trusted publisher is included in the current corpus, and the unconfirmed corpus of the trusted publisher is included in the extended corpus; The corpus to be confirmed in the extended corpus is subjected to secondary verification, and the corpus to be confirmed that passes the secondary verification is included in the current corpus, wherein the secondary verification method includes cross-verification and manual verification.

[0006] In one embodiment of the present application, a triple knowledge graph is constructed based on the plurality of corpus data, including: Extracting triple entity templates from multiple pieces of corpus data respectively, wherein the triple entity includes a subject template, a relation entity template and an object template; Converting the triple entity template into an entity template vector, and clustering the entity template vectors of the subject template and the object template in multiple dimensions respectively, to obtain multiple subject clusters and multiple object clusters in each dimension, wherein the multiple dimensions include basic concepts, functions and principles; Labeling multiple subject clusters and multiple object clusters to obtain labels of each subject template and object template; Construct a triple knowledge graph based on labeled triple entity templates.

[0007] In one embodiment of the present application, triple entities of the historical corpus are extracted, and the triple entities of the historical corpus are matched with the triple knowledge graph to obtain matching results, including: Extracting the subject, the relation subject and the object in the historical corpus; and vectorizing the subject, the relation subject and the object to obtain a subject vector, a relation subject vector and an object vector; The subject vector is mixed into the entity template vector of the subject template with a label for multiple clustering, and the labels of other vectors in the cluster where the subject vector is located are used as the label of the subject vector; the object vector is mixed into the entity template vector of the object template with a label for multiple clustering, and the labels of other vectors in the cluster where the object vector is located are used as the label of the object vector; The triple entity is matched with the triple knowledge graph based on labels to obtain a matching result.

[0008] In one embodiment of the present application, the triple entity and the triple knowledge graph are matched based on labels to obtain a matching result, including: Filtering out a target triple entity template whose main body template is the same as the main body of the triple entity from the triple knowledge graph; Comparing the triple entity with the target triple entity template; when the triple entity and the target triple entity template meet the first target condition, determining the matching result is consistent; when the triple entity and the target triple entity template meet the second target condition, determining the matching result is inconsistent; and when the triple entity and the target triple entity template meet the third target condition, determining the matching result is not matched; Wherein, the first target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold; The second target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the relationship entity has the opposite meaning to the relationship entity template; or; the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, and any label of the object is different from any label of the object template; The third target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, the meaning of the relationship entity is not opposite to that of the relationship entity template, and the similarity between the relationship entity and the relationship entity template is less than or equal to the preset similarity threshold; or; the similarity between the relationship entity and the relationship entity template is greater than the preset similarity threshold, the object is inconsistent with the object template, and the label of the object has the same label as the object template; or; the similarity between the relationship entity and the relationship entity template is less than or equal to the preset similarity threshold, and the object is inconsistent with the object template.

[0009] In one embodiment of the present application, the confidence of the information publisher is evaluated based on the matching result to obtain the confidence of multiple information publishers, including: Get the initial confidence of multiple information publishers ; Matching results and initial confidence based on historical corpora of multiple information publishers Calculate the confidence of multiple information publishers , where the confidence of multiple information publishers The mathematical expression is:

[0010] In the formula, is the first weight, is the second weight, The serial number of the information publisher. For the The matching results of each information publisher are consistent with the number of corpora. No. The number of corpora with contradictory matching results for information publishers, For the The total number of corpora of information publishers.

[0011] In one embodiment of the present application, determining the trusted corpus and the corpus to be confirmed of the trusted publisher based on the triple knowledge graph includes: Comparing the triple entity of the historical corpus of the trusted publisher with the triple knowledge graph; The historical corpus that meets the fourth target condition is used as credible corpus, and the historical corpus that does not meet the fourth target condition is used as corpus to be confirmed, wherein the fourth target condition includes: the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, the subject and the subject template have consistent labels, and the object and the object template have consistent labels.

[0012] In one embodiment of the present application, performing secondary verification on the corpus in the extended corpus includes: Cross-validating the corpus in the extended corpus, and adding the corpus that passes the cross-validation to the current corpus; The corpus that failed the cross-validation was sent to the target object for manual verification.

[0013] In one embodiment of the present application, cross-validating the corpus in the extended corpus and adding the corpus that passes the cross-validation to the current corpus includes: Matching triplets of corpora from different trusted publishers in the extended corpus; When the triples of corpora from different trusted publishers are consistent in the extended corpus, the corpora from different trusted publishers are included in the current corpus.

[0014] In one embodiment of the present application, it also includes: Accumulating the storage time of all corpora in the extended corpus; Corpus whose storage time exceeds a preset time threshold is removed from the extended corpus.

[0015] The present application also provides a corpus construction system for a large medical question-answering model, including: An acquisition module, used to acquire the current corpus of the medical question-answering large model and historical corpora released by multiple information publishers, wherein the current corpus includes multiple pieces of corpus data; A knowledge graph construction module, used to construct a triple knowledge graph based on the multiple corpus data; A matching module, used for extracting triple entities of the historical corpus, and matching the triple entities of the historical corpus with the triple knowledge graph to obtain matching results, wherein the matching results include consistency, contradiction and non-matching; A confidence evaluation module, used to evaluate the confidence of the information publisher based on the matching result, and obtain the confidence of multiple information publishers; A screening and collection module, used to take information publishers whose confidence is greater than a preset threshold as trusted publishers, and determine the trusted corpus and the unconfirmed corpus of the trusted publisher based on the triple knowledge graph, collect the trusted corpus of the trusted publisher into the current corpus, and collect the unconfirmed corpus of the trusted publisher into the extended corpus; The verification and collection module is used to perform secondary verification on the unconfirmed corpus in the extended corpus, and collect the unconfirmed corpus that passes the secondary verification into the current corpus, wherein the secondary verification method includes cross-verification and manual verification.

[0016] The beneficial effects of the present invention are as follows: a corpus construction method and system of a large medical question-and-answer model of the present invention constructs a triple knowledge graph based on an existing corpus, and then matches the historical corpus of multiple information publishers in the network with the triple knowledge graph, thereby determining the reliability of multiple information publishers in the process of historical information release, and then screening out credible information publishers. The triple knowledge graph is then used to perform a credibility check on the historical corpus released by the information publisher. If the released historical corpus conforms to the rules in the triple knowledge graph, it is directly included in the current corpus. If it does not conform, it is placed in an extended library for cross-validation or manual verification. The present application can use network corpus to expand the corpus. During the expansion, the triple knowledge graph of the existing corpus is used to perform reliability verification on the information publisher and the corpus, thereby ensuring the reliability of the expanded corpus. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 It is a flowchart of a method for constructing a corpus of a medical question-and-answer large model shown in one embodiment of the present application; Figure 2 A schematic diagram of a triple knowledge graph in an embodiment of the present application; Figure 3 This is a schematic diagram of a triplet matching process in an embodiment of the present application; Figure 4 It is a structural diagram of a corpus construction system of a medical question and answer large model shown in one embodiment of the present application. DETAILED DESCRIPTION

[0019] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show the layers related to the present invention rather than being drawn according to the number, shape and size of the layers in actual implementation. In actual implementation, the type, quantity and proportion of each layer may be changed arbitrarily, and the layer layout may also be more complicated.

[0021] In the following description, numerous details are discussed to provide a more thorough explanation of embodiments of the present invention; however, it is apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details.

[0022] Figure 1 is a flowchart of a method for constructing a corpus of a medical question-answering large model shown in an embodiment of the present application. Figure 1 As shown, a corpus construction method of a medical question and answer large model in this embodiment may include the steps of: S110, obtaining a current corpus of the medical question-and-answer big model and historical corpora released by multiple information publishers, wherein the current corpus includes multiple pieces of corpus data; Among them, the current corpus is the foundation of the medical question-answering model. The corpus provides a large number of question and answer samples, which can help the model learn how to generate or select the correct answer based on the input question. By being exposed to diverse language expressions and sentence structures, the question-answering model can better understand the complexity and nuances of natural language. This includes but is not limited to synonym recognition, context understanding, and intent parsing.

[0023] The corpus in the corpus is usually stored in a structured form. Take the following question-answer pair as an example: Q: “What is chitosan?” A: “Chitosan is a naturally occurring linear polysaccharide, mainly composed of N-acetylglucosamine units connected by β-(1→4) glycosidic bonds. It is a product obtained by deacetylation of chitin and is widely found in the shells of crustaceans. Chitosan has a wide range of applications in medicine, food, agriculture and other fields due to its unique biological activity and physicochemical properties.” Store part of the question and answer text in the following form: {Qv, Av, question: "What is chitosan?", answers: "Chitosan is a naturally occurring linear polysaccharide, mainly composed of N-acetylglucosamine units connected by β-(1→4) glycosidic bonds. It is a product obtained by deacetylation of chitin and is widely found in the shells of crustaceans. Chitosan is widely used in medicine, food, agriculture and other fields due to its unique biological activity and physicochemical properties."} Among them, Qv is the vector of question text, and Av is the vector of answer text.

[0024] In addition, the information publisher is the object of information publication in the network environment, which can be a public object such as a public account, a video account, or an information publisher in a medical forum, etc.

[0025] The information published by the above-mentioned objects is highly timely and can therefore make up for the shortcomings of data sources such as journals and books that are less timely.

[0026] S120, constructing a triple knowledge graph based on the multiple corpus data; In this application, if there is a triple knowledge graph in the current corpus, a label is assigned to the existing triple knowledge graph, and the classification label is used to perform the subsequent matching process.

[0027] In this application, the process of directly constructing the triple knowledge graph is as follows: S121, extracting triple entity templates from the plurality of corpus data respectively, wherein the triple entity includes a subject template, a relation entity template and an object template; In this application, the extraction of triples can be based on the following process: (1) First, the input sentence is segmented and each word is assigned a corresponding part-of-speech tag. This is to distinguish different types of words such as nouns and verbs, in preparation for further analysis.

[0028] (2) The structure of the sentence is parsed by a syntactic analyzer to generate a syntactic tree or a dependency tree. The dependency tree is particularly useful because it directly shows the dependency relationship between words and helps to quickly locate the subject, predicate, and object.

[0029] (3) Identify named entities (such as names of people, places, organizations, etc.) or other important entities in a sentence. These are usually the subject or object in a triple.

[0030] (4) Based on the results of syntactic analysis and information from entity recognition, determine the predicate and the subject and object it connects. This step requires the use of predefined relationship templates or machine learning models to improve accuracy.

[0031] S122, converting the triple entity template into an entity template vector, and clustering the entity template vectors of the subject template and the object template in multiple dimensions to obtain multiple subject clusters and multiple object clusters; The vector conversion can adopt the Word2Vec model and use the trained Word2Vec model to perform vector conversion.

[0032] In the present application, clustering is used to divide the subject and object in all triple entity templates. Since entity template vectors are similar if their meanings are close, their vectors are also similar. Clustering can be used to effectively classify entity template vectors.

[0033] For example, the present application divides entity template vectors into multiple categories such as "anti-inflammatory drugs", "treatment methods", "disease types", "surgical equipment", etc., thereby obtaining multiple corresponding clusters.

[0034] As an optimized embodiment in the present application, clustering in the present application can be multiple clustering, that is, clustering from multiple dimensions, for example, clustering from multiple dimensions such as the function, principle, and basic concept of the entity, thereby obtaining multiple labels. When performing multi-dimensional clustering, it is necessary to capture the relationship between entities from different dimensions by training the Word2Vec model, thereby converting them into word vectors in multiple dimensions, and then clustering. Entities with similarities in different dimensions (such as similar functions, similar basic concepts, etc.) can be clustered together to obtain a common label.

[0035] S123, labeling the multiple subject clusters and the multiple object clusters to obtain a label of each subject template and object template; Since the entities in the cluster are similar entities, the object cluster can be directly labeled and the label can be transferred to all entities in the cluster. The above process can achieve the purpose of rapid labeling.

[0036] S124, constructing a triple knowledge graph based on the labeled triple entity template.

[0037] After the annotation is completed, the labeled entities can be used to construct multiple triples, which can be further combined into a triple knowledge graph.

[0038] Figure 2 is a schematic diagram of a triple knowledge graph in an embodiment of the present application. The triple knowledge graph constructed in the present application is as follows Figure 2 shown.

[0039] S130, extracting triple entities of the historical corpus, and matching the triple entities of the historical corpus with the triple knowledge graph to obtain matching results, wherein the matching results include consistency, contradiction, and non-matching; The process of extracting triple entities from historical corpus can be referred to in the previous article and will not be repeated here.

[0040] Figure 3 FIG. 1 is a schematic diagram of a triple matching process in an embodiment of the present application. Figure 3 As shown, the triple entity of the historical corpus is extracted, and the triple entity of the historical corpus is matched with the triple knowledge graph to obtain a matching result, including: S131, extracting the subject, relation subject and object in the historical corpus; and vectorizing the subject, relation subject and object to obtain a subject vector, a relation subject vector and an object vector; S132, mixing the subject vector into the entity template vector of the subject template with a label for multiple clustering, and using the labels of other vectors in the cluster where the subject vector is located as the label of the subject vector; mixing the object vector into the entity template vector of the object template with a label for multiple clustering, and using the labels of other vectors in the cluster where the object vector is located as the label of the object vector; Please refer to the previous article for the clustering and labeling process. Multi-dimensional clustering is also used to obtain multi-dimensional labels.

[0041] S133, performing label-based rule matching on the triple entity and the triple knowledge graph to obtain a matching result.

[0042] The process of using the extracted triple entity to match the triple knowledge graph with rules is as follows: S1331, filtering out a target triple entity template whose main body template is the same as the main body of the triple entity from the triple knowledge graph; In this application, the subject in the triple is used as a benchmark for matching, and multiple target triple entity templates in the triple knowledge graph are obtained. Then, the comparison between the relation entity and the object is performed.

[0043] S1332, comparing the triple entity with the target triple entity template; when the triple entity and the target triple entity template meet the first target condition, determining that the matching result is consistent; when the triple entity and the target triple entity template meet the second target condition, determining that the matching result is inconsistent; and when the triple entity and the target triple entity template meet the third target condition, determining that the matching result is not matched; a. The first target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold; When the similarity between the object and the object template is greater than a preset similarity threshold, and the relationship entity and the relationship entity template are similar, it can be determined that the two are consistent.

[0044] For example, the triple entity is {chitosan-interaction-adsorption of negatively charged substances} The triple entity template is {chitosan-function-adsorption of negatively charged substances} The subject and object of the above triple entity and triple entity template are consistent, and the relationship entities are similar, so it can be determined that the matching results are consistent.

[0045] In addition, the similarity in this application uses the cosine similarity calculation method to calculate the similarity between vectors.

[0046] b. The second target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the relationship entity has the opposite meaning to the relationship entity template; or; the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, and any label of the object is different from any label of the object template; b.1 The current object is similar or consistent with the object template, but the meanings expressed by the relationship entity are opposite. In this case, the meanings expressed by the two triples are contradictory, for example: The triple entity is {chitosan-can be used-before-surgery} The triple entity template is {chitosan-contraindications-taken before surgery} The relationship entities "can be used for" and "taboo" have opposite meanings. Therefore, the two are considered contradictory.

[0047] In this application, existing vocabulary resources such as WordNet are used, which provide the relationship between words (synonyms, antonyms, etc.). By querying such databases, word pairs with opposite meanings can be directly found.

[0048] b.2 When the subjects of the two are consistent and the relationship entities are consistent or similar, then if the objects are mutually exclusive, then the two can be judged to be contradictory.

[0049] In a triple, the same relationship of the subject may correspond to multiple objects, but multiple objects corresponding to multiple entities with the same relationship will be clustered together and get the same label when clustered in the previous text. Therefore, labels can be used to determine whether different objects have mutually exclusive relationships. For example: The triple entity is {Chitosan - is - a natural polysaccharide (label: polysaccharide / basic concept)} The triple entity template is {chitosan-is-drug (label: drug)} Since chitosan was not classified into the cluster where drugs were located in the previous clustering, its label belongs to "long-chain polymer-biological macromolecules". Entities with similar clusters include "starch", "cellulose", "heparin", etc. Therefore, by comparing the labels, it can be concluded that "chitosan-is-drug" is false, and there is a contradiction between the two triples.

[0050] This application uses the label of the object to judge whether the triples are contradictory, which can effectively solve the problem that the contradictory relationships between different triples cannot be determined because there may be multiple objects corresponding to the same relationship of the subject.

[0051] In the above process, if all the labels are different, it means that the concepts corresponding to the two entities are completely mutually exclusive, which can more accurately judge the contradictory relationship between the triples.

[0052] c. The third target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, the meaning of the relationship entity is not opposite to that of the relationship entity template, and the similarity between the relationship entity and the relationship entity template is less than or equal to the preset similarity threshold; or; the similarity between the relationship entity and the relationship entity template is greater than the preset similarity threshold, the object is inconsistent with the object template, and the object and the object template have the same label; or the similarity between the relationship entity and the relationship entity template is less than or equal to the preset similarity threshold, and the object is inconsistent with the object template.

[0053] c.1 Since there may be multiple relationships between the same subject and object, if the similarity between the object and the object template is high, but there is no clear opposite or similar relationship between the relationship entities, it is difficult to determine whether there is a problem with the triple, for example: {Chitosan-similar-to-starch (label: polysaccharide)} {Chitosan-with-starch (label: polysaccharide)} Therefore, the matching result is judged as no match.

[0054] c.2 If the subject and the subject template are identical or similar and the relationship entity is determined, the object and the object template should have the same label. However, if there are different entities with the same label, it is impossible to directly determine whether the two match. For example: Triad entity {Chitosan - Function is - Adsorbs negatively charged substances (label: function)} Triad entity template {Chitosan - Function - Making drug carrier (label: function)} Since the above-mentioned function of adsorbing negatively charged substances is not recorded in the triple knowledge graph, it is impossible to verify whether the function of chitosan includes adsorbing negatively charged substances through the triple knowledge graph. Therefore, further verification is required, and the matching result is judged as no match at this time.

[0055] c.3 The object is not similar to the object model, and the relationship between the relationship entity and the relationship entity template is unclear. Neither can the opposite relationship be inferred, and the similarity is low. In this case, it is difficult to directly infer the matching relationship, for example: {Chitosan - is - a natural polysaccharide (tag: polysaccharide / basic concept)} {Chitosan-Function-Adsorbs negatively charged substances (label: function)} In this case, it is highly likely that the two groups of triples express concepts of different dimensions, so the matching relationship is judged as unmatched.

[0056] S140, evaluating the confidence of the information publisher based on the matching result to obtain the confidence of multiple information publishers; In the previous article, a triple knowledge graph with labels of multiple dimensions is used to match historical corpus, and the confidence of the information publisher can be evaluated based on the matching results.

[0057] Since the cost of error correction for online corpus is high, this application will deduct the confidence of publishers who publish erroneous information. For publishers who frequently publish information on knowledge graph matching, their confidence will also be increased accordingly. Therefore, the confidence evaluation process is as follows: S141, obtaining initial confidences of multiple information publishers ; S142, matching results and initial confidence based on historical corpora of multiple information publishers Calculate the confidence of multiple information publishers , where the confidence of multiple information publishers The mathematical expression is:

[0058] In the formula, is the first weight, is the second weight, The serial number of the information publisher. For the The matching results of each information publisher are consistent with the number of corpora. No. The number of corpora with contradictory matching results for information publishers, For the The total number of corpora of information publishers.

[0059] Specifically, in this application, the error rate is used To weight the number of corpora with contradictory matching results, so as to strengthen the impact of wrong results on the confidence score.

[0060] S150, taking the information publisher whose confidence is greater than a preset threshold as a trusted publisher, and determining the trusted corpus and the to-be-confirmed corpus of the trusted publisher based on the triple knowledge graph, adding the trusted corpus of the trusted publisher to the current corpus, and adding the to-be-confirmed corpus of the trusted publisher to the extended corpus; This application authenticates the trusted publisher and expands the corpus published by the trusted publisher. During the expansion process, the triple knowledge graph needs to be used again to verify the corpus. The specific process is as follows: S151, comparing the triple entity of the historical corpus of the trusted publisher with the triple knowledge graph; S152, taking the historical corpus that meets the fourth target condition as the credible corpus, and taking the historical corpus that does not meet the fourth target condition as the corpus to be confirmed, wherein the fourth target condition includes: the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, the subject and the subject template have consistent labels, and the object and the object template have consistent labels.

[0061] Since the credibility of each information publisher has been evaluated in the previous article, it can be judged here that most of the information published by credible publishers is true and reliable. It is only necessary to simply verify whether the historical corpus conforms to the basic rules of the triple knowledge graph.

[0062] This application uses subject label-relation entity-object label as the basic rule. As long as it meets this range, it is inferred that the corpus is basically credible and thus included in the current corpus. If it does not meet the basic rule, it will be included in the extended corpus and wait for secondary verification.

[0063] S160, performing secondary verification on the corpus to be confirmed in the extended corpus, and adding the corpus to be confirmed that passes the secondary verification into the current corpus, wherein the secondary verification includes cross-verification and manual verification.

[0064] In one embodiment of the present application, performing secondary verification on the corpus in the extended corpus includes: S161, cross-validating the corpus in the extended corpus, and adding the corpus that passes the cross-validation to the current corpus; Cross-validation refers to using different information sources to verify the corpus to be confirmed. If the same triple entity comes from different information sources, it is likely to be authentic and can be included in the current corpus. Specifically, it includes: S1611, matching triples of corpora from different trusted publishers in the extended corpus; S1612: When the triples of the corpora from different trusted publishers are consistent in the extended corpus, the corpora from different trusted publishers are included in the current corpus.

[0065] S162, sending the corpus that has not passed the cross-validation to the target object for manual verification.

[0066] In addition, in the present application, the storage time of all corpora in the extended corpus is accumulated, and corpora whose storage time exceeds a preset time threshold are removed from the extended corpus, thereby avoiding the long-term accumulation of invalid corpora in the extended corpus, occupying space, and causing waste of resources.

[0067] The present invention provides a corpus construction method for a large medical question-and-answer model, which constructs a triple knowledge graph based on an existing corpus, and then matches the historical corpus of multiple information publishers in the network with the triple knowledge graph, thereby determining the reliability of multiple information publishers in the process of historical information release, and then screening out credible information publishers. The triple knowledge graph is then used to perform a credibility check on the historical corpus released by the information publisher. If the released historical corpus conforms to the rules in the triple knowledge graph, it is directly included in the current corpus. If it does not conform, it is placed in an extended library for cross-validation or manual verification. The present application can use network corpus to expand the corpus. During the expansion, the triple knowledge graph of the existing corpus is used to perform reliability verification on the information publisher and the corpus, thereby ensuring the reliability of the expanded corpus.

[0068] like Figure 4 As shown, the present application also provides a corpus construction system for a large medical question-answering model, including: An acquisition module, used to acquire the current corpus of the medical question-answering large model and historical corpora released by multiple information publishers, wherein the current corpus includes multiple pieces of corpus data; A knowledge graph construction module, used to construct a triple knowledge graph based on the multiple corpus data; A matching module, used for extracting triple entities of the historical corpus, and matching the triple entities of the historical corpus with the triple knowledge graph to obtain matching results, wherein the matching results include consistency, contradiction and non-matching; A confidence evaluation module, used to evaluate the confidence of the information publisher based on the matching result, and obtain the confidence of multiple information publishers; A screening and collection module, used to take information publishers whose confidence is greater than a preset threshold as trusted publishers, and determine the trusted corpus and the unconfirmed corpus of the trusted publisher based on the triple knowledge graph, collect the trusted corpus of the trusted publisher into the current corpus, and collect the unconfirmed corpus of the trusted publisher into the extended corpus; The verification and collection module is used to perform secondary verification on the unconfirmed corpus in the extended corpus, and collect the unconfirmed corpus that passes the secondary verification into the current corpus, wherein the secondary verification method includes cross-verification and manual verification.

[0069] A corpus construction system of a large medical question-and-answer model of the present invention constructs a triple knowledge graph based on an existing corpus, and then matches the historical corpus of multiple information publishers in the network with the triple knowledge graph, so as to determine the reliability of multiple information publishers in the process of historical information release, and then screen out credible information publishers. Then, the triple knowledge graph is used to perform a credibility check on the historical corpus released by the information publisher. If the released historical corpus conforms to the rules in the triple knowledge graph, it is directly included in the current corpus. If it does not conform, it is placed in an extended library for cross-validation or manual verification. The present application can use network corpus to expand the corpus. During the expansion, the triple knowledge graph of the existing corpus is used to perform reliability verification on the information publisher and the corpus, thereby ensuring the reliability of the expanded corpus.

[0070] This embodiment also provides an electronic terminal, including: a processor and a memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0071] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0072] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0073] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0074] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0075] In the above-mentioned embodiments, although the present invention has been described in conjunction with the specific embodiments of the present invention, many replacements, modifications and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. The embodiments of the present invention are intended to cover all such replacements, modifications and variations falling within the broad scope of the appended claims.

[0076] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for constructing a corpus of a large medical question-answering model, characterized in that: Includes steps: Acquire a current corpus of the medical question-answering model and historical corpora published by multiple information publishers, wherein the current corpus includes multiple pieces of corpus data; Constructing a triple knowledge graph based on the multiple corpus data; Extracting triple entities of the historical corpus, and matching the triple entities of the historical corpus with the triple knowledge graph to obtain matching results, wherein the matching results include consistency, contradiction, and non-matching; Evaluating the confidence of the information publisher based on the matching result to obtain the confidence of multiple information publishers; An information publisher whose confidence is greater than a preset threshold is regarded as a trusted publisher, and the trusted corpus and the unconfirmed corpus of the trusted publisher are determined based on the triple knowledge graph, the trusted corpus of the trusted publisher is included in the current corpus, and the unconfirmed corpus of the trusted publisher is included in the extended corpus; The corpus to be confirmed in the extended corpus is subjected to secondary verification, and the corpus to be confirmed that passes the secondary verification is included in the current corpus, wherein the secondary verification method includes cross-verification and manual verification.

2. The method for constructing a corpus of a medical question-answering large model according to claim 1, characterized in that: Constructing a triple knowledge graph based on the multiple corpus data includes: Extracting triple entity templates from multiple pieces of corpus data respectively, wherein the triple entity includes a subject template, a relation entity template and an object template; Converting the triple entity template into an entity template vector, and clustering the entity template vectors of the subject template and the object template in multiple dimensions respectively, to obtain multiple subject clusters and multiple object clusters in each dimension, wherein the multiple dimensions include basic concepts, functions and principles; Labeling multiple subject clusters and multiple object clusters to obtain labels of each subject template and object template; Construct a triple knowledge graph based on labeled triple entity templates.

3. The method for constructing a corpus of a medical question-answering large model according to claim 2, characterized in that: Extracting triple entities of the historical corpus, and matching the triple entities of the historical corpus with the triple knowledge graph to obtain matching results, including: Extracting the subject, the relation subject and the object in the historical corpus; and vectorizing the subject, the relation subject and the object to obtain a subject vector, a relation subject vector and an object vector; The subject vector is mixed into the entity template vector of the subject template with a label for multiple clustering, and the labels of other vectors in the cluster where the subject vector is located are used as the label of the subject vector; the object vector is mixed into the entity template vector of the object template with a label for multiple clustering, and the labels of other vectors in the cluster where the object vector is located are used as the label of the object vector; The triple entity is matched with the triple knowledge graph based on labels to obtain a matching result.

4. The method for constructing a corpus of a medical question-answering model according to claim 3, characterized in that: Performing label-based rule matching on the triple entity and the triple knowledge graph to obtain a matching result includes: Filtering out a target triple entity template whose main body template is the same as the main body of the triple entity from the triple knowledge graph; Comparing the triple entity with the target triple entity template; when the triple entity and the target triple entity template meet the first target condition, determining the matching result is consistent; when the triple entity and the target triple entity template meet the second target condition, determining the matching result is inconsistent; and when the triple entity and the target triple entity template meet the third target condition, determining the matching result is not matched; Wherein, the first target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold; The second target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the relationship entity has the opposite meaning to the relationship entity template; or; the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, and any label of the object is different from any label of the object template; The third target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, the meaning of the relationship entity is not opposite to that of the relationship entity template, and the similarity between the relationship entity and the relationship entity template is less than or equal to the preset similarity threshold; or; the similarity between the relationship entity and the relationship entity template is greater than the preset similarity threshold, the object is inconsistent with the object template, and the label of the object has the same label as the object template; or; the similarity between the relationship entity and the relationship entity template is less than or equal to the preset similarity threshold, and the object is inconsistent with the object template.

5. The method for constructing a corpus of a medical question-answering large model according to claim 1, characterized in that: The confidence of the information publisher is evaluated based on the matching result to obtain the confidence of multiple information publishers, including: Get the initial confidence of multiple information publishers ; Matching results and initial confidence based on historical corpora of multiple information publishers Calculate the confidence of multiple information publishers , where the confidence of multiple information publishers The mathematical expression is: In the formula, is the first weight, is the second weight, The serial number of the information publisher. For the The matching results of each information publisher are consistent with the number of corpora. No. The number of corpora with contradictory matching results for information publishers, For the The total number of corpora of information publishers.

6. The method for constructing a corpus of a medical question-answering large model according to claim 3, characterized in that: Determining the trusted corpus and the corpus to be confirmed of the trusted publisher based on the triple knowledge graph includes: Comparing the triple entity of the historical corpus of the trusted publisher with the triple knowledge graph; The historical corpus that meets the fourth target condition is used as credible corpus, and the historical corpus that does not meet the fourth target condition is used as corpus to be confirmed, wherein the fourth target condition includes: the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, the subject and the subject template have consistent labels, and the object and the object template have consistent labels.

7. The method for constructing a corpus of a medical question-answering large model according to claim 1, characterized in that: The corpus in the extended corpus is subjected to secondary verification, including: Cross-validating the corpus in the extended corpus, and adding the corpus that passes the cross-validation to the current corpus; The corpus that failed the cross-validation was sent to the target object for manual verification.

8. The method for constructing a corpus of a medical question-answering large model according to claim 7, characterized in that: Cross-validating the corpus in the extended corpus and adding the cross-validated corpus to the current corpus includes: Matching triplets of corpora from different trusted publishers in the extended corpus; When the triples of corpora from different trusted publishers are consistent in the extended corpus, the corpora from different trusted publishers are included in the current corpus.

9. The method for constructing a corpus of a medical question-answering large model according to claim 1, characterized in that: Also includes: Accumulating the storage time of all corpora in the extended corpus; Corpus whose storage time exceeds a preset time threshold is removed from the extended corpus.

10. A corpus construction system for a large medical question-answering model, characterized in that: include: An acquisition module, used to acquire the current corpus of the medical question-answering model and historical corpora released by multiple information publishers, wherein the current corpus includes multiple pieces of corpus data; A knowledge graph construction module, used to construct a triple knowledge graph based on the multiple corpus data; A matching module, used for extracting triple entities of the historical corpus, and matching the triple entities of the historical corpus with the triple knowledge graph to obtain matching results, wherein the matching results include consistency, contradiction and non-matching; A confidence evaluation module, used to evaluate the confidence of the information publisher based on the matching result, and obtain the confidence of multiple information publishers; A screening and collection module, used to take information publishers whose confidence is greater than a preset threshold as trusted publishers, and determine the trusted corpus and the unconfirmed corpus of the trusted publisher based on the triple knowledge graph, collect the trusted corpus of the trusted publisher into the current corpus, and collect the unconfirmed corpus of the trusted publisher into the extended corpus; The verification and collection module is used to perform secondary verification on the unconfirmed corpus in the extended corpus, and collect the unconfirmed corpus that passes the secondary verification into the current corpus, wherein the secondary verification method includes cross-verification and manual verification.

Images