A corpus construction method and system of a medical question and answer large model

Abstract

The present application relates to the field of data processing, and in particular to a medical question and answer large model corpus construction method and system, a triple knowledge graph is constructed based on an existing corpus, then historical corpus of multiple information publishers in the network is matched with the triple knowledge graph, so that the reliability of the multiple information publishers in the historical information publishing process is determined, and then the reliable information publishers are screened out. Then the triple knowledge graph is used to perform trust verification on the historical corpus published by the information publisher, if the published historical corpus conforms to the rules in the triple knowledge graph, it is directly included in the current corpus. If it does not conform, it is put into an extension library for cross verification or manual verification. The present application can expand the corpus by using network corpus, and when expanding, the triple knowledge graph of the existing corpus is used to perform reliability verification on the information publisher and the corpus, so that the reliability of the expanded corpus is ensured.

Description

Technical Field

[0001] This invention relates to the field of data processing, specifically to a method and system for constructing a corpus for a large-scale medical question-and-answer model. Background Technology

[0002] To address questions from patients or their families, existing hospitals often set up terminal devices in the hospital lobby. These terminals have embedded question-and-answer models to answer common questions for patients or their families.

[0003] For large-scale medical question-and-answer models used to respond to patients or their families, the corpus is crucial as it forms the foundation of the language model. The corpus itself needs to be as authentic and reliable as possible. Therefore, existing corpora generally come from authoritative materials such as medical textbooks and authoritative journals. This results in the current medical question-and-answer models having insufficient corpus coverage, making them unable to handle many questions. However, directly introducing online corpora for expansion may introduce unreliable data samples, potentially leading to low-quality answers and reducing the reliability of the question-and-answer model. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method and system for constructing a corpus of a large medical question-answering model, so as to solve the problems in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] The present invention provides a method for constructing a corpus for a large-scale medical question-answering model, comprising the following steps:

[0007] Obtain the current corpus of the medical question-and-answer model and the historical corpus published by multiple information publishers, wherein the current corpus includes multiple corpus data;

[0008] A triplet knowledge graph is constructed based on the aforementioned multiple corpus data;

[0009] Extract the triple entities from the historical corpus, and match the triple entities from the historical corpus with the triple knowledge graph to obtain matching results, wherein the matching results include consistent, contradictory, and unmatched;

[0010] The confidence level of the information publisher is evaluated based on the matching results, resulting in the confidence levels of multiple information publishers;

[0011] Information publishers with a confidence level greater than a preset threshold are identified as trusted publishers. Based on the triplet knowledge graph, the trusted corpus and the corpus to be confirmed of the trusted publishers are determined. The trusted corpus of the trusted publishers is included in the current corpus, and the corpus to be confirmed of the trusted publishers is included in the extended corpus.

[0012] The unconfirmed corpus in the extended corpus is subjected to secondary verification, and the unconfirmed corpus that passes the secondary verification is included in the current corpus. The secondary verification methods include cross-verification and manual verification.

[0013] In one embodiment of this application, a triplet knowledge graph is constructed based on the multiple corpus data, including:

[0014] Triple entity templates are extracted from multiple corpus data, wherein the triple entity includes a subject template, a relation entity template, and an object template;

[0015] The triplet entity template is converted into an entity template vector, and the entity template vectors of the subject template and the object template are clustered in multiple dimensions to obtain multiple subject clusters and multiple object clusters in each dimension. The multiple dimensions include basic concepts, functions and principles.

[0016] Label multiple subject clusters and multiple object clusters to obtain the labels for each subject template and object template;

[0017] Construct a triplet knowledge graph based on labeled triplet entity templates.

[0018] In one embodiment of this application, triple entities of the historical corpus are extracted, and the triple entities of the historical corpus are matched with the triple knowledge graph to obtain matching results, including:

[0019] Extract the subject, relation subject, and object from the historical corpus; and vectorize the subject, relation subject, and object to obtain subject vector, relation subject vector, and object vector;

[0020] The subject vector is mixed into the entity template vector of the labeled subject template for multiple clustering, and the labels of other vectors in the cluster where the subject vector is located are used as the labels of the subject vector; the object vector is mixed into the entity template vector of the labeled object template for multiple clustering, and the labels of other vectors in the cluster where the object vector is located are used as the labels of the object vector.

[0021] The triplet entity is matched with the triplet knowledge graph based on labels to obtain the matching result.

[0022] In one embodiment of this application, tag-based rule matching is performed between the triple entity and the triple knowledge graph to obtain a matching result, including:

[0023] Filter out target triple entity templates from the triple knowledge graph whose subject template is the same as the subject of the triple entity;

[0024] The triplet entity is compared with the target triplet entity template; if the triplet entity and the target triplet entity template meet the first target condition, the matching result is determined to be consistent; if the triplet entity and the target triplet entity template meet the second target condition, the matching result is determined to be contradictory; and if the triplet entity and the target triplet entity template meet the third target condition, the matching result is determined to be unmatched.

[0025] The first target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the similarity between the relation entity and the relation entity template is greater than a preset similarity threshold.

[0026] The second target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the meaning of the relation entity is opposite to that of the relation entity template; or: the similarity between the relation entity and the relation entity template is greater than a preset similarity threshold, and any label of the object is different from any label of the object template;

[0027] The third objective condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, the meaning of the relation entity and the relation entity template is not opposite, and the similarity between the relation entity and the relation entity template is less than or equal to the preset similarity threshold; or; the similarity between the relation entity and the relation entity template is greater than the preset similarity threshold, the object and the object template are inconsistent, and the object's label has the same label as the object template; or; the similarity between the relation entity and the relation entity template is less than or equal to the preset similarity threshold, and the object and the object template are inconsistent.

[0028] In one embodiment of this application, the confidence level of the information publisher is evaluated based on the matching results to obtain the confidence levels of multiple information publishers, including:

[0029] Obtain the initial confidence level of multiple information publishers ;

[0030] Matching results and initial confidence levels based on historical corpora from multiple information publishers Calculate the confidence level of multiple information publishers Among them, the confidence levels of multiple information publishers The mathematical expression is:

[0031]

[0032] In the formula, As the first weight, As the second weight, For information publisher serial number, For the first The matching results for each information publisher show a consistent number of corpora. No. The matching results for each information publisher represent the number of contradictory corpora. For the first The total number of corpora from each information publisher.

[0033] In one embodiment of this application, determining the trusted corpus and the corpus to be verified of the trusted publisher based on the triplet knowledge graph includes:

[0034] Compare the triple entities in the historical corpus of the trusted publisher with the triple knowledge graph;

[0035] Historical corpora that meet the fourth objective condition are considered reliable corpora, while historical corpora that do not meet the fourth objective condition are considered corpora to be confirmed. The fourth objective condition includes: the similarity between the relation entity and the relation entity template is greater than a preset similarity threshold, the subject and the subject template have the same label, and the object and the object template have the same label.

[0036] In one embodiment of this application, secondary verification of the corpus in the extended corpus includes:

[0037] Cross-validation is performed on the corpus in the extended corpus, and the corpus that passes cross-validation is added to the current corpus;

[0038] The corpus that fails cross-validation is sent to the target object for manual verification.

[0039] In one embodiment of this application, cross-validation is performed on the corpus in the extended corpus, and the corpus that passes the cross-validation is added to the current corpus, including:

[0040] Match the triples from the corpus of different trusted publishers in the extended corpus;

[0041] When the triples of corpora from different trusted publishers are consistent in the extended corpus, the corpora from different trusted publishers are included in the current corpus.

[0042] In one embodiment of this application, it further includes:

[0043] The storage time of all corpora in the extended corpus is accumulated;

[0044] Corpora that have been stored for a duration exceeding a preset duration threshold will be removed from the extended corpus.

[0045] This application also provides a corpus construction system for a large-scale medical question-answering model, including:

[0046] The acquisition module is used to acquire the current corpus of the medical question-and-answer model and the historical corpus published by multiple information publishers, wherein the current corpus includes multiple corpus data;

[0047] The knowledge graph construction module is used to construct a triplet knowledge graph based on the multiple corpus data.

[0048] The matching module is used to extract the triple entities of the historical corpus and match the triple entities of the historical corpus with the triple knowledge graph to obtain the matching results, wherein the matching results include consistent, contradictory and unmatched;

[0049] The confidence evaluation module is used to evaluate the confidence of the information publisher based on the matching results, and obtain the confidence of multiple information publishers;

[0050] The filtering and inclusion module is used to identify information publishers with a confidence level greater than a preset threshold as trusted publishers, and to determine the trusted corpus and unconfirmed corpus of the trusted publishers based on the triplet knowledge graph. The trusted corpus of the trusted publishers is included in the current corpus, and the unconfirmed corpus of the trusted publishers is included in the extended corpus.

[0051] The verification and inclusion module is used to perform secondary verification on the unconfirmed corpus in the extended corpus, and to include the unconfirmed corpus that has passed the secondary verification into the current corpus. The secondary verification methods include cross-verification and manual verification.

[0052] The beneficial effects of this invention are as follows: This invention provides a corpus construction method and system for a large-scale medical question-and-answer model. It constructs a triplet knowledge graph based on an existing corpus, then matches the historical corpora of multiple information publishers on the network with the triplet knowledge graph to determine the reliability of these publishers during their historical information publishing process, thereby filtering out trustworthy publishers. Next, the triplet knowledge graph is used to verify the trustworthiness of the historical corpora published by the information publishers. If the published historical corpora conform to the patterns in the triplet knowledge graph, they are directly included in the current corpus. If they do not conform, they are placed in an extended corpus for cross-validation or manual verification. This application can expand the corpus using network corpora. During expansion, the triplet knowledge graph of the existing corpus is used to verify the reliability of the information publishers and the corpora, ensuring the reliability of the expanded corpus. Attached Figure Description

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0054] Figure 1 This is a flowchart illustrating a corpus construction method for a large medical question-answering model in one embodiment of this application;

[0055] Figure 2 This is a schematic diagram of a triplet knowledge graph in one embodiment of this application;

[0056] Figure 3 This is a schematic diagram of the triple matching process in one embodiment of this application;

[0057] Figure 4 This is a structural diagram of a corpus construction system for a large medical question-and-answer model, as shown in one embodiment of this application. Detailed Implementation

[0058] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0059] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0060] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of the invention; however, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details.

[0061] Figure 1 This is a flowchart illustrating a corpus construction method for a large-scale medical question-answering model, as shown in one embodiment of this application. Figure 1 As shown in this embodiment, a method for constructing a corpus for a large-scale medical question-answering model may include the following steps:

[0062] S110, Obtain the current corpus of the medical question-and-answer model and the historical corpus published by multiple information publishers, wherein the current corpus includes multiple corpus data;

[0063] The current corpus forms the foundation of the large-scale medical question-answering model. The corpus provides a vast number of question and answer samples, which help the model learn how to generate or select the correct answer based on the input question. By engaging with diverse linguistic expressions and sentence structures, the question-answering model can better understand the complexity and nuances of natural language. This includes, but is not limited to, synonym recognition, contextual understanding, and intent parsing.

[0064] Corpus data is typically stored in a structured format. For example, consider the following question-answer pairs:

[0065] Q: What is chitosan?

[0066] A: "Chitosan is a naturally occurring linear polysaccharide, mainly composed of N-acetylglucosamine units linked by β-(1→4) glycosidic bonds. It is a product obtained from the deacetylation of chitin and is widely found in the shells of crustaceans. Due to its unique biological activity and physicochemical properties, chitosan has a wide range of applications in medicine, food, agriculture and other fields."

[0067] Store a portion of the question-and-answer text in the following format:

[0068] {Qv, Av, question: "What is chitosan?", answers: "Chitosan is a naturally occurring linear polysaccharide, mainly composed of N-acetylglucosamine units linked by β-(1→4) glycosidic bonds. It is a product obtained from the deacetylation of chitin and is widely found in the shells of crustaceans. Due to its unique biological activity and physicochemical properties, chitosan has wide applications in medicine, food, agriculture, and other fields."}

[0069] Where Qv is the vector of the question text and Av is the vector of the answer text.

[0070] In addition, the information publisher is the object of information publication in the online environment, which can be a public object such as a public account or video account, or an object of information publication in a medical forum, etc.

[0071] The information published by the aforementioned entities is highly timely, thus effectively compensating for the poor timeliness of data sources such as journals and books.

[0072] S120, Construct a triplet knowledge graph based on the multiple corpus data;

[0073] In this application, if a triplet knowledge graph exists in the current corpus, labels are assigned to the existing triplet knowledge graph, and the classification labels are used for the subsequent matching process.

[0074] In this application, the process of directly constructing the triplet knowledge graph is as follows:

[0075] S121, extract triple entity templates from multiple corpus data respectively, wherein the triple entity includes a subject template, a relation entity template and an object template;

[0076] In this application, the extraction of triples can be based on the following process:

[0077] (1) First, the input sentence is segmented into words, and each word is assigned a corresponding part-of-speech tag. This is to distinguish different types of words such as nouns and verbs, in preparation for further analysis.

[0078] (2) The structure of the sentence is parsed by a syntactic analyzer to generate a syntactic tree or a dependency tree. The dependency tree is particularly useful because it directly shows the dependencies between words, which helps to quickly locate the subject, predicate, and object.

[0079] (3) Identify named entities (such as personal names, place names, organization names, etc.) or other important entities in the sentence. These are usually the subject or object in the triple.

[0080] (4) Based on the results of syntactic analysis and entity recognition information, determine the predicate and the subject and object it connects. This step requires the use of predefined relation templates or machine learning models to improve accuracy.

[0081] S122, the triplet entity template is converted into an entity template vector, and the entity template vectors of the subject template and the object template are clustered in multiple dimensions to obtain multiple subject clusters and multiple object clusters;

[0082] Vector transformation can be performed using the Word2Vec model, which utilizes a pre-trained Word2Vec model.

[0083] In this application, clustering is used to divide the subject and object in all triplet entity templates. Since entity template vectors with similar meanings are also similar, clustering can effectively classify entity template vectors.

[0084] For example, this application divides entity template vectors into multiple categories such as "anti-inflammatory drugs", "treatment methods", "disease types", and "surgical instruments", thereby obtaining multiple corresponding clusters.

[0085] As an optimized embodiment of this application, clustering in this application can be multi-stage clustering, that is, clustering from multiple dimensions, such as the function, principle, and basic concept of an entity, to obtain multiple labels. When performing multi-dimensional clustering, it is necessary to train a Word2Vec model to capture the relationships between entities from different dimensions, thereby converting them into word vectors in multiple dimensions for clustering. Entities with similarities in different dimensions (e.g., similar functions, similar basic concepts, etc.) can be grouped together to obtain a common label.

[0086] S123, label multiple subject clusters and multiple object clusters to obtain the labels for each subject template and object template;

[0087] Since the entities within a cluster are similar entities, directly labeling the object cluster allows the labels to be applied to all entities within the cluster, achieving the goal of rapid labeling.

[0088] S124, Construct a triplet knowledge graph based on labeled triplet entity templates.

[0089] Once the annotations are complete, the tagged entities can be used to construct multiple triples, which can then be further combined to form a triple knowledge graph.

[0090] Figure 2 This is a schematic diagram of a triplet knowledge graph in one embodiment of this application. The triplet knowledge graph constructed in this application is as follows: Figure 2 As shown.

[0091] S130, extract the triple entities from the historical corpus, and match the triple entities from the historical corpus with the triple knowledge graph to obtain matching results, wherein the matching results include consistent, contradictory, and unmatched;

[0092] The process of extracting triple entities from historical corpora can be referred to in the previous text, and will not be repeated here.

[0093] Figure 3 This is a schematic diagram of the triple matching process in one embodiment of this application, as shown below. Figure 3As shown, the triple entities of the historical corpus are extracted, and the triple entities of the historical corpus are matched with the triple knowledge graph to obtain the matching results, including:

[0094] S131, extract the subject, relation subject, and object from the historical corpus; and vectorize the subject, relation subject, and object to obtain the subject vector, relation subject vector, and object vector;

[0095] S132, the subject vector is mixed into the entity template vector of the labeled subject template for multiple clustering, and the labels of other vectors in the cluster where the subject vector is located are used as the labels of the subject vector; the object vector is mixed into the entity template vector of the labeled object template for multiple clustering, and the labels of other vectors in the cluster where the object vector is located are used as the labels of the object vector.

[0096] Please refer to the previous text for the clustering and labeling process. Similarly, multi-dimensional clustering is used to obtain multi-dimensional labels.

[0097] S133, perform label-based rule matching on the triple entity and the triple knowledge graph to obtain the matching result.

[0098] The process of using extracted triple entities to perform rule matching with the triple knowledge graph is as follows:

[0099] S1331, Select target triplet entity templates from the triplet knowledge graph whose subject template is the same as the subject of the triplet entity;

[0100] In this application, matching is performed based on the subject in the triplet to obtain multiple target triplet entity templates in the triplet knowledge graph. Then, the comparison between relation entities and objects is performed.

[0101] S1332, compare the triplet entity with the target triplet entity template; when the triplet entity and the target triplet entity template meet the first target condition, determine the matching result as consistent; when the triplet entity and the target triplet entity template meet the second target condition, determine the matching result as contradictory; and when the triplet entity and the target triplet entity template meet the third target condition, determine the matching result as unmatched.

[0102] a. The first target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the similarity between the relation entity and the relation entity template is greater than a preset similarity threshold;

[0103] When the similarity between an object and an object template is greater than a preset similarity threshold, and the relation entity and the relation entity template are similar, it can be determined that the two are consistent.

[0104] For example, the ternary entity is {chitosan - function - adsorbs negatively charged substances}.

[0105] The ternary template is {chitosan - function - adsorption of negatively charged substances}

[0106] The subjects and objects of the triplet entities and the triplet entity templates mentioned above are the same, and the relational entities are similar. Therefore, it can be determined that the matching results are consistent.

[0107] Furthermore, the similarity in this application uses the cosine similarity method to calculate the similarity between vectors.

[0108] b. The second target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the meaning of the relation entity is opposite to that of the relation entity template; or; the similarity between the relation entity and the relation entity template is greater than a preset similarity threshold, and any label of the object is different from any label of the object template;

[0109] b.1 If the current object is similar to or identical to the object template, but the relation entity expresses a different meaning, then the meanings expressed by the two triples are contradictory. For example:

[0110] The ternary entity is {chitosan - can be taken preoperatively}.

[0111] The ternary entity template is {Chitosan - Contraindications - Preoperative use}

[0112] The relational entities "can be used" and "forbidden" have opposite meanings. Therefore, they are determined to be contradictory.

[0113] This application utilizes existing vocabulary resources, such as WordNet, which provide relationships between words (synonyms, antonyms, etc.). By querying such databases, word pairs with opposite meanings can be found directly.

[0114] b.2 When the subjects of the two are the same and the relational entities are also the same or similar, then if the objects have a mutually exclusive relationship, then it can be determined that the two are contradictory.

[0115] Since a triple may contain multiple objects corresponding to the same relation of an entity, these objects will be clustered together and receive the same label during the clustering process described earlier. Therefore, the label can be used to determine whether different objects have mutually exclusive relations. For example:

[0116] The ternary entity is {chitosan - is - a natural polysaccharide (tag: polysaccharide / basic concept)}

[0117] The ternary entity template is {Chitosan-is-pharmaceutical (tag: medicine)}

[0118] Since chitosan was not classified into the coarse cluster containing pharmaceuticals in the previous clustering, its label belongs to "long-chain polymers - biological macromolecules." Entities in similar clusters include "starch," "cellulose," and "heparin." Therefore, by comparing the labels, we can conclude that "chitosan - is - pharmaceutical" is false, and there is a contradiction between the two triples.

[0119] This application uses the label of the object to determine whether the triples are contradictory, which can effectively solve the problem that the contradictory relationship between different triples cannot be determined because there may be multiple objects corresponding to the same relationship of the subject.

[0120] In the above process, if all the labels are different, it means that the concepts corresponding to the two entities are completely mutually exclusive, which can more accurately determine the contradictory relationship between the triples.

[0121] c. The third target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, the meaning of the relation entity and the relation entity template is not opposite, and the similarity between the relation entity and the relation entity template is less than or equal to the preset similarity threshold; or; the similarity between the relation entity and the relation entity template is greater than the preset similarity threshold, the object and the object template are inconsistent, and the object and the object template have the same label; or the similarity between the relation entity and the relation entity template is less than or equal to the preset similarity threshold, and the object and the object template are inconsistent.

[0122] c.1 Since there may be multiple relationships between the same subject and object, if the object has a high similarity to the object template, but there is no explicit opposite or similar relationship between the relational entities, it is difficult to determine whether the triple is problematic. For example:

[0123] {Chitosan-like-starch (label: polysaccharide)}

[0124] {Chitosan - Combined with starch (Label: Polysaccharide)}

[0125] Therefore, the matching result is determined to be a no match.

[0126] c.2 If the subject and subject template are the same or similar, and the relational entity is determined, the object and object template should have the same label. However, if different entities have the same label, it is impossible to directly determine whether they match. For example:

[0127] The ternary entity {chitosan - its function is to adsorb negatively charged substances (label: function)}

[0128] Ternary entity template {Chitosan - Function: To create a drug carrier (Label: Function)}

[0129] Since the ternary knowledge graph does not record the above-mentioned function of adsorbing negatively charged substances, it is impossible to verify whether the role of chitosan includes adsorbing negatively charged substances through the ternary knowledge graph. Therefore, further verification is required, and the matching result is determined to be unmatched.

[0130] c.3 The objects and object models are dissimilar, and the relationship between the relational entity and the relational entity template is ambiguous; neither an inverse relationship can be inferred, and the similarity is low. In this case, it is difficult to directly infer their matching relationship, for example:

[0131] Chitosan is a natural polysaccharide (Label: Polysaccharide / Basic Concepts)

[0132] {Chitosan - Function - Adsorbs negatively charged substances (Tag: Function)}

[0133] In this case, it is highly likely that the two sets of triples represent concepts of different dimensions, so the matching relationship is determined to be unmatched.

[0134] S140, Based on the matching results, evaluate the confidence level of the information publisher to obtain the confidence levels of multiple information publishers;

[0135] In the previous section, a triplet knowledge graph with multi-dimensional labels was used to match historical corpora, and the confidence of information publishers could be evaluated based on the matching results.

[0136] Because correcting errors in online corpora is costly, this application deducts the confidence score of publishers who post erroneous information. For publishers who frequently post information on knowledge graph matching platforms, their confidence score is increased accordingly. Therefore, the confidence score evaluation process is as follows:

[0137] S141, Obtain the initial confidence levels of multiple information publishers. ;

[0138] S142, Matching results and initial confidence levels based on historical corpora from multiple information publishers. Calculate the confidence level of multiple information publishers Among them, the confidence levels of multiple information publishers The mathematical expression is:

[0139]

[0140] In the formula, As the first weight, As the second weight, For information publisher serial number, For the first The matching results for each information publisher show a consistent number of corpora. No. The matching results for each information publisher represent the number of contradictory corpora. For the first The total number of corpora from each information publisher.

[0141] Specifically, this application uses an error rate. We can weight the number of corpora with contradictory matching results to amplify the impact of erroneous results on the confidence score.

[0142] S150, Information publishers with confidence scores greater than a preset threshold are identified as trusted publishers, and the trusted corpus and unconfirmed corpus of the trusted publishers are determined based on the triplet knowledge graph. The trusted corpus of the trusted publishers is added to the current corpus, and the unconfirmed corpus of the trusted publishers is added to the extended corpus.

[0143] This application expands upon the corpus published by trusted publishers through authentication. During the expansion process, the corpus needs to be validated again using a triplet knowledge graph. The specific process is as follows:

[0144] S151, compare the triple entities of the historical corpus of the trusted publisher with the triple knowledge graph;

[0145] S152, historical corpora that meet the fourth objective condition are used as credible corpora, and historical corpora that do not meet the fourth objective condition are used as corpora to be confirmed. The fourth objective condition includes: the similarity between the relation entity and the relation entity template is greater than a preset similarity threshold, the subject and the subject template have the same label, and the object and the object template have the same label.

[0146] Since the credibility of each information publisher has already been evaluated in the preceding text, it can be concluded that most of the information published by credible publishers is true and reliable. A simple verification of whether the historical corpus conforms to the basic rules of triplet knowledge graphs is sufficient.

[0147] This application uses the subject label-relational entity-object label as a basic rule. If the data conforms to this range, the corpus is considered basically reliable and thus included in the current corpus. If the data does not conform to the basic rule, it is included in an expanded corpus for secondary verification.

[0148] S160, perform secondary verification on the unconfirmed corpus in the extended corpus, and include the unconfirmed corpus that has passed the secondary verification into the current corpus. The secondary verification method includes cross-verification and manual verification.

[0149] In one embodiment of this application, secondary verification of the corpus in the extended corpus includes:

[0150] S161, perform cross-validation on the corpus in the extended corpus, and include the cross-validated corpus in the current corpus;

[0151] Cross-validation refers to using different information sources to verify the corpus to be verified. If the same triple entity comes from different information sources, it is likely to be true and reliable, and can then be included in the current corpus. Specifically, this includes:

[0152] S1611, Match the triples from the corpora of different trusted publishers in the extended corpus;

[0153] S1612, when the triples of corpora from different trusted publishers are consistent in the extended corpus, the corpora from different trusted publishers are included in the current corpus.

[0154] S162, send the corpus that failed cross-validation to the target object for manual verification.

[0155] Furthermore, this application also accumulates the storage time of all corpora in the extended corpus; corpora whose storage time exceeds a preset time threshold are removed from the extended corpus. This avoids the accumulation of invalid corpora in the extended corpus for extended periods, thus preventing space occupation and resource waste.

[0156] This invention discloses a corpus construction method for a large-scale medical question-answering model. Based on an existing corpus, a triplet knowledge graph is constructed. Then, the historical corpora of multiple information publishers in the network are matched with the triplet knowledge graph to determine the reliability of the publishers during their historical information publishing process, thereby filtering out trustworthy publishers. Next, the triplet knowledge graph is used to verify the trustworthiness of the publishers' historical corpora. If the published historical corpora conform to the patterns in the triplet knowledge graph, they are directly added to the current corpus. If they do not conform, they are placed in an extended corpus for cross-validation or manual verification. This application can expand the corpus using network corpora. During expansion, the reliability of the publishers and corpora is verified using the triplet knowledge graph of the existing corpus, ensuring the reliability of the expanded corpus.

[0157] like Figure 4As shown, this application also provides a corpus construction system for a large-scale medical question-answering model, including:

[0158] The acquisition module is used to acquire the current corpus of the medical question-and-answer model and the historical corpus published by multiple information publishers, wherein the current corpus includes multiple corpus data;

[0159] The knowledge graph construction module is used to construct a triplet knowledge graph based on the multiple corpus data.

[0160] The matching module is used to extract the triple entities of the historical corpus and match the triple entities of the historical corpus with the triple knowledge graph to obtain the matching results, wherein the matching results include consistent, contradictory and unmatched;

[0161] The confidence evaluation module is used to evaluate the confidence of the information publisher based on the matching results, and obtain the confidence of multiple information publishers;

[0162] The filtering and inclusion module is used to identify information publishers with a confidence level greater than a preset threshold as trusted publishers, and to determine the trusted corpus and unconfirmed corpus of the trusted publishers based on the triplet knowledge graph. The trusted corpus of the trusted publishers is included in the current corpus, and the unconfirmed corpus of the trusted publishers is included in the extended corpus.

[0163] The verification and inclusion module is used to perform secondary verification on the unconfirmed corpus in the extended corpus, and to include the unconfirmed corpus that has passed the secondary verification into the current corpus. The secondary verification methods include cross-verification and manual verification.

[0164] This invention discloses a corpus construction system for a large-scale medical question-and-answer model. It constructs a triplet knowledge graph based on an existing corpus, then matches the historical data of multiple information publishers on the network with the triplet knowledge graph to determine the reliability of these publishers during their historical information dissemination, thereby filtering out trustworthy publishers. Next, the triplet knowledge graph is used to verify the trustworthiness of the publishers' historical data. If the historical data conforms to the patterns in the triplet knowledge graph, it is directly added to the current corpus. If not, it is placed in an extended corpus for cross-validation or manual verification. This application can expand the corpus using network data. During expansion, the existing triplet knowledge graph is used to verify the reliability of both the publishers and the data, ensuring the reliability of the expanded corpus.

[0165] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0166] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory so that the terminal performs any of the methods in this embodiment.

[0167] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0168] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0169] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0170] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0171] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0172] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A corpus construction method of a medical question and answer large model, characterized in that, The method comprises the steps of: obtaining a current corpus of a medical question and answer large model and historical corpus published by a plurality of information publishers, wherein the current corpus comprises a plurality of corpus data; constructing a triple knowledge graph based on the plurality of corpus data; constructing a triple knowledge graph based on the plurality of corpus data comprises: extracting triple entity templates from the plurality of corpus data respectively, wherein the triple entity comprises a subject template, a relationship entity template and an object template; converting the triple entity template into an entity template vector, and clustering the entity template vectors of the subject template and the object template in multiple dimensions respectively to obtain a plurality of subject clusters and a plurality of object clusters in each dimension, wherein the multiple dimensions comprise basic concepts, functions and principles; labeling the plurality of subject clusters and the plurality of object clusters to obtain labels of each subject template and object template; and constructing a triple knowledge graph based on the labeled triple entity template; extracting the triple entity of the historical corpus, and matching the triple entity of the historical corpus with the triple knowledge graph to obtain a matching result, wherein the matching result includes consistency, contradiction and non-matching; extracting the triple entity of the historical corpus, and matching the triple entity of the historical corpus with the triple knowledge graph to obtain a matching result, including: extracting the subject, the relationship subject and the object in the historical corpus; and vectorizing the subject, the relationship subject and the object to obtain a subject vector, a relationship subject vector and an object vector; mixing the subject vector into the entity template vector of the subject template with a label for multiple clustering, and taking the label of other vectors in the cluster where the subject vector is located as the label of the subject vector; mixing the object vector into the entity template vector of the object template with a label for multiple clustering, and taking the label of other vectors in the cluster where the object vector is located as the label of the object vector; performing label-based rule matching on the triple entity and the triple knowledge graph to obtain a matching result; performing label-based rule matching on the triple entity and the triple knowledge graph to obtain a matching result, including: screening a target triple entity template with the same subject as the triple entity from the triple knowledge graph; comparing the triple entity with the target triple entity template; when the triple entity and the target triple entity template meet a first target condition, determining that the matching result is consistent; when the triple entity and the target triple entity template meet a second target condition, determining that the matching result is contradictory; and when the triple entity and the target triple entity template meet a third target condition, determining that the matching result is non-matching; wherein the first target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold; the second target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the meaning of the relationship entity and the relationship entity template is opposite; or the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, and any label of the object is different from any label of the object template; the third target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, the meaning of the relationship entity and the relationship entity template is not opposite, and the similarity between the relationship entity and the relationship entity template is less than or equal to a preset similarity threshold; or the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, the object and the object template are inconsistent, and the label of the object and the label of the object template exist the same label; or the similarity between the relationship entity and the relationship entity template is less than or equal to a preset similarity threshold, and the object and the object template are inconsistent; based on the matching result, evaluating the confidence of the information publisher to obtain the confidence of multiple information publishers; publishers with the confidence greater than the preset threshold as trusted publishers, and determine trusted corpus and to-be-confirmed corpus of the trusted publishers based on the triple knowledge graph, and collect the trusted corpus of the trusted publishers to the current corpus library and collect the to-be-confirmed corpus of the trusted publishers to an extended corpus library; perform secondary verification on the to-be-confirmed corpus in the extended corpus library, and collect the to-be-confirmed corpus passing the secondary verification to the current corpus library, wherein the secondary verification includes cross verification and manual checking.

2. The corpus construction method of a medical question and answer large model according to claim 1, characterized in that, evaluate the confidence of the information publisher based on the matching result, and obtain the confidence of multiple information publishers, including: Obtaining initial confidence for a plurality of information publishers ; Matching results and initial confidence based on historical corpus of multiple information publishers Computing confidence of multiple information publishers wherein the confidence of multiple information publishers is mathematically expressed as: wherein, is a first weight, is a second weight, is an information publisher sequence number, is a number of corpus in which the matching result of the first information publisher is consistent, is a number of corpus in which the matching result of the second information publisher is inconsistent, is a total number of corpus of the first information publisher.

3. The corpus construction method of a medical question and answer large model according to claim 1, characterized in that, determine the trusted corpus and the to-be-confirmed corpus of the trusted publisher based on the triple knowledge graph, including: compare the triple entities of the historical corpus of the trusted publisher with the triple knowledge graph; take the historical corpus satisfying the fourth target condition as the trusted corpus, and take the historical corpus not satisfying the fourth target condition as the to-be-confirmed corpus, wherein the fourth target condition includes that the similarity of the relationship entity and the relationship entity template is greater than a preset similarity threshold, the subject and the subject template have consistent labels, and the object and the object template have consistent labels.

4. The corpus construction method of a medical question and answer large model according to claim 1, characterized in that, perform secondary verification on the corpus in the extended corpus library, including: perform cross verification on the corpus in the extended corpus library, and collect the corpus passing the cross verification to the current corpus library; send the corpus not passing the cross verification to a target object to perform manual checking.

5. The corpus construction method of a medical question and answer large model according to claim 4, characterized in that, perform cross verification on the corpus in the extended corpus library, and collect the corpus passing the cross verification to the current corpus library, including: match the triples of the corpus from different trusted publishers in the extended corpus library; when the triples of the corpus from different trusted publishers in the extended corpus library are consistent, collect the corpus from different trusted publishers to the current corpus library.

6. The corpus construction method of a medical question and answer large model according to claim 1, characterized in that, further including: accumulate the storage duration of all corpus in the extended corpus library; remove the corpus with the storage duration exceeding a preset duration threshold from the extended corpus library. 7.A corpus construction system of a medical question and answer large model, characterized in that, including: an acquisition module, configured to acquire a current corpus library of a medical question and answer large model and historical corpus published by multiple information publishers, wherein the current corpus library includes multiple corpus data; a knowledge graph construction module, configured to construct a triple knowledge graph based on the multiple corpus data; the construction of the triple knowledge graph based on the multiple corpus data includes: extracting triple entity templates from the multiple corpus data respectively, wherein the triple entity includes a subject template, a relationship entity template and an object template; converting the triple entity templates into entity template vectors, and clustering the entity template vectors of the subject template and the object template in multiple dimensions respectively to obtain multiple subject clusters and multiple object clusters in each dimension, wherein the multiple dimensions include basic concepts, functions and principles; labeling the multiple subject clusters and the multiple object clusters to obtain labels of each subject template and object template; and constructing the triple knowledge graph based on the triple entity templates with labels; The matching module is configured to extract the triple entity of the historical corpus and match the triple entity of the historical corpus with the triple knowledge graph to obtain a matching result, wherein the matching result includes consistency, contradiction and non-matching; the matching result is obtained by extracting the triple entity of the historical corpus and matching the triple entity of the historical corpus with the triple knowledge graph, including: extracting the subject, the relationship subject and the object in the historical corpus; and performing vectorization processing on the subject, the relationship subject and the object to obtain a subject vector, a relationship subject vector and an object vector; mixing the subject vector into the entity template vector of the subject template with a label for multiple times of clustering, and taking the label of other vectors in the cluster where the subject vector is located as the label of the subject vector; mixing the object vector into the entity template vector of the object template with a label for multiple times of clustering, and taking the label of other vectors in the cluster where the object vector is located as the label of the object vector; performing label-based rule matching on the triple entity and the triple knowledge graph to obtain a matching result; the matching result is obtained by performing label-based rule matching on the triple entity and the triple knowledge graph, including: screening a target triple entity template with the same subject as the triple entity from the triple knowledge graph; comparing the triple entity with the target triple entity template; when the triple entity and the target triple entity template meet a first target condition, determining that the matching result is consistent; when the triple entity and the target triple entity template meet a second target condition, determining that the matching result is contradictory; and when the triple entity and the target triple entity template meet a third target condition, determining that the matching result is non-matching; wherein the first target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold; the second target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, and the meaning of the relationship entity and the relationship entity template is opposite; or the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, and any label of the object is different from any label of the object template; the third target condition includes: the similarity between the object and the object template is greater than a preset similarity threshold, the meaning of the relationship entity and the relationship entity template is not opposite, and the similarity between the relationship entity and the relationship entity template is less than or equal to a preset similarity threshold; or the similarity between the relationship entity and the relationship entity template is greater than a preset similarity threshold, the object and the object template are inconsistent, and the label of the object and the label of the object template exist the same label; or the similarity between the relationship entity and the relationship entity template is less than or equal to a preset similarity threshold, and the object and the object template are inconsistent; The confidence evaluation module is configured to evaluate the confidence of the information publisher based on the matching result to obtain the confidence of the plurality of information publishers. The screening and inclusion module is configured to include information publishers with a confidence level greater than a preset threshold as trusted publishers, determine trusted corpus and to-be-confirmed corpus of the trusted publishers based on the triple knowledge graph, include the trusted corpus of the trusted publishers into the current corpus, and include the to-be-confirmed corpus of the trusted publishers into an extended corpus. The verification and inclusion module is configured to perform secondary verification on the to-be-confirmed corpus in the extended corpus, and include the to-be-confirmed corpus that passes the secondary verification into the current corpus, where the secondary verification includes cross verification and manual verification.

Images