Medical knowledge extraction method and device, storage medium and computer device

Abstract

Embodiments of the present application disclose a medical knowledge extraction method and device, a storage medium and a computer device. The method comprises: obtaining an input medical text and performing encoding processing to obtain the encoding features of each character in the medical text; for each subtext of any length in the medical text, determining the self feature representation of each subtext through an attention mechanism and the encoding features of each subtext, so as to improve the accuracy of the self feature representation of each subtext; determining the target entity to which each subtext belongs according to the self feature representation of each subtext, obtaining an entity set composed of the target entities to which the subtexts in the medical text belong, and then for any two subtexts in the entity set, determining the target relationship between the target entities to which the two subtexts belong through a multi-view context attention mechanism according to the self feature representations of the two subtexts, so as to improve the accuracy of the relationship determination between the entities and the accuracy of medical knowledge acquisition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and particularly to a method, device, computer-readable storage medium and computer equipment for extracting medical knowledge. Background Art

[0002] In various application scenarios of intelligent healthcare, such as auxiliary diagnosis, medication recommendation, and department recommendation, medical professional knowledge, as the cornerstone of each application scenario, plays an indispensable supporting role. The medical knowledge involved in these application scenarios is very extensive and professional, such as diseases, symptoms, medications, treatments, etc. Therefore, it is crucial to accurately and efficiently extract medical knowledge from various authoritative sources (such as textbooks, guidelines, etc.) for the requirements of downstream application scenarios.

[0003] Existing medical knowledge extraction technologies can generally be abstracted as entity-relationship joint extraction technologies, which mainly extract entity features and relationship features in the text input, and classify the entity features and relationship features to obtain medical knowledge such as entities and relationships in the text input. Although entity-relationship joint extraction technologies can learn medical knowledge to a certain extent, when constructing entity features and relationship features, entity-relationship joint extraction technologies mainly focus on the entity and relationship self-features in the text input, and have weak context modeling ability, resulting in inaccurate medical knowledge learned. Summary of the Invention

[0004] Embodiments of this application provide a method, device, computer-readable storage medium and computer equipment for extracting medical knowledge, which can improve the accuracy of medical knowledge extraction.

[0005] Embodiments of this application provide a method for extracting medical knowledge, including:

[0006] Obtain a medical text, and perform encoding processing on the medical text to obtain the encoding features of each character in the medical text;

[0007] For each sub-text of any length in the medical text, according to the encoding features of each character included in each sub-text, determine the self-feature representation of each sub-text through an attention mechanism;

[0008] According to the self-feature representation of each sub-text, determine the target entity to which each sub-text belongs to obtain the entity set of the medical text;

[0009] For any two sub-texts in the entity set, according to the self-feature representations of the two sub-texts, determine the target relationship between the target entities to which the two sub-texts belong through a multi-perspective context attention mechanism;

[0010] Determine the target entities to which each sub - text in the medical text belongs and the target relationships between the target entities to which any two sub - texts belong as the medical knowledge of the medical text.

[0011] An embodiment of the present application provides a medical knowledge extraction method, including:

[0012] Obtain a plurality of training samples and an initial medical knowledge extraction model, where each training sample in the plurality of training samples includes a training medical text, target entity labels in the training medical text, and target relationship labels between training target entities to which any two training medical texts belong;

[0013] Use the initial medical knowledge extraction model to perform encoding processing on the training medical text to obtain the training encoding features of each word in the training medical text;

[0014] For each training sub - text of any length in the training medical text, determine the training self - feature representation of each training sub - text through an attention mechanism according to the training encoding features of each word included in each training sub - text;

[0015] Determine the training target entity to which each training sub - text belongs according to the training self - feature representation of each training sub - text to obtain the training entity set of the training medical text;

[0016] Determine a first loss value according to the training target entities in the training entity set of the training medical text and the target entity labels of the training medical text;

[0017] For any two training sub - texts in the training entity set, determine the training target relationship between the training target entities to which the two training sub - texts belong through a multi - perspective context attention mechanism according to the training self - feature representations of the two training sub - texts;

[0018] Determine a second loss value according to the training target relationship and the target relationship label;

[0019] Determine the overall loss value according to the first loss value and the second loss value, and update the initial medical knowledge extraction model according to the overall loss value to obtain a medical knowledge extraction model.

[0020] An embodiment of the present application further provides a medical knowledge extraction device, including:

[0021] An acquisition module for acquiring a medical text;

[0022] An encoding module for performing encoding processing on the medical text to obtain the encoding features of each word in the medical text;

[0023] A feature determination module, configured to, for each sub-text of any length in the medical text, determine the self-feature representation of each sub-text through an attention mechanism according to the encoding features of each character included in each sub-text;

[0024] An entity extraction module, configured to determine the target entity to which each sub-text belongs according to the self-feature representation of each sub-text, so as to obtain the entity set of the medical text;

[0025] A relationship extraction module, configured to, for any two sub-texts in the entity set, determine the target relationship between the target entities to which the two sub-texts belong through a multi-perspective context attention mechanism according to the self-feature representations of the two sub-texts;

[0026] A knowledge determination module, configured to determine the target entity to which each sub-text in the medical text belongs and the target relationship between the target entities to which any two sub-texts belong as the medical knowledge of the medical text.

[0027] An embodiment of the present application further provides a medical knowledge extraction device, including:

[0028] A training and acquisition module, configured to acquire a plurality of training samples and an initial medical knowledge extraction model, wherein each training sample in the plurality of training samples includes a training medical text, the target entity label in the training medical text, and the target relationship label between the training target entities to which any two training medical texts belong;

[0029] A training encoding module, configured to perform encoding processing on the training medical text by using the initial medical knowledge extraction model to obtain the training encoding features of each character in the training medical text;

[0030] A training feature determination module, configured to, for each training sub-text of any length in the training medical text, determine the training self-feature representation of each training sub-text through an attention mechanism according to the training encoding features of each character included in each training sub-text;

[0031] A training entity extraction module, configured to determine the training target entity to which each training sub-text belongs according to the training self-feature representation of each training sub-text, so as to obtain the training entity set of the training medical text;

[0032] A first loss determination module, configured to determine a first loss value according to the training target entities in the training entity set of the training medical text and the target entity label of the training medical text;

[0033] A training relation extraction module, configured to, for any two training sub-texts in the training entity set, determine a training target relation between the training target entities to which the two training sub-texts belong according to the training self-feature representations of the two training sub-texts through a multi-perspective context attention mechanism;

[0034] A second loss determination module, configured to determine a second loss value according to the training target relation and the target relation label;

[0035] An update module, configured to determine an overall loss value according to the first loss value and the second loss value, and update the initial medical knowledge extraction model according to the overall loss value to obtain a medical knowledge extraction model.

[0036] An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the steps in the medical knowledge extraction method described in any one of the above embodiments.

[0037] An embodiment of the present application further provides a computer device, where the computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the steps in the medical knowledge extraction method described in any one of the above embodiments by calling the computer program stored in the memory.

[0038] The medical knowledge extraction method, device, computer-readable storage medium and computer device provided by the embodiments of the present application obtain an input medical text, perform encoding processing on the medical text to obtain the encoding features of each word in the medical text, and for each sub-text with any length in the medical text, determine the self-feature representation of each sub-text through an attention mechanism to improve the accuracy of the self-feature representation of each sub-text. According to the self-feature representation of each sub-text, determine the target entity to which each sub-text belongs. After obtaining the entity set composed of the target entities to which the sub-texts in the medical text belong, for any two sub-texts in the entity set, according to the self-feature representations of the two sub-texts, determine the relationship between the target entities to which the two sub-texts belong through a multi-perspective context attention mechanism. Since the context information from different perspectives is considered through the multi-perspective context attention mechanism when determining the relationship between the target entities to which the two sub-texts belong, the accuracy of determining the relationship between entities is improved. In this way, the accuracy of the medical knowledge in the medical text is improved. Description of the Drawings

[0039] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those skilled in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0040] Figure 1 It is a schematic flowchart of the medical knowledge extraction method provided by the embodiment of the present application.

[0041] Figure 2 It is a schematic sub - flowchart of the medical knowledge extraction method provided by the embodiment of the present application.

[0042] Figure 3 It is a schematic sub - flowchart of the medical knowledge extraction method provided by the embodiment of the present application.

[0043] Figure 4 It is a schematic diagram of the network topology structure of the entity extraction part provided by the embodiment of the present application.

[0044] Figure 5 It is another schematic sub - flowchart of the medical knowledge extraction method provided by the embodiment of the present application.

[0045] Figure 6 It is a schematic diagram of the network topology structure of the relationship extraction part provided by the embodiment of the present application.

[0046] Figure 7 It is another schematic flowchart of the medical knowledge extraction method provided by the embodiment of the present application.

[0047] Figure 8 It is a schematic diagram of the structure of the medical knowledge extraction device provided by the embodiment of the present application.

[0048] Figure 9 It is another schematic diagram of the structure of the medical knowledge extraction device provided by the embodiment of the present application.

[0049] Figure 10 It is a schematic diagram of the structure of the computer device provided by the embodiment of the present application. Detailed implementation manners

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope protected by the present application.

[0051] The embodiments of the present application provide a medical knowledge extraction method, apparatus, computer-readable storage medium, and computer device. Specifically, the medical knowledge extraction method of the embodiments of the present application can be executed by a computer device, where the computer device can be a terminal or a server, etc. The terminal can be a smart phone, a tablet computer, a notebook computer, a touch screen, a robot, a personal computer (PC), a vehicle-mounted terminal, or other terminal devices. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services and cloud databases.

[0052] Before introducing the solutions in the embodiments of the present application in detail, a further analysis and understanding of the current technology will be carried out first.

[0053] Traditional medical knowledge extraction methods rely on professionals to manually extract medical knowledge. Such methods are inefficient and costly. Therefore, a medical knowledge self-learning solution has emerged. Medical knowledge self-learning aims to design a system that can start from a small amount of labeled data, predict unlabeled data through a model, submit it for manual review and then re-learn, thereby forming a self-learning ability, and finally achieve efficient and accurate extraction of medical knowledge from new data sources. Therefore, how to design the model structure of medical knowledge extraction so that it can accurately extract medical knowledge from text becomes particularly important.

[0054] Current medical knowledge extraction technologies can generally be abstracted as entity-relationship joint extraction technologies. Such methods first use a pre-trained model based on BERT (Bidirectional Encoder Representations from Transformer) to encode the input text, and then splice the corresponding features. For entity extraction, mainly the encoded splices of the start and end characters of the entity are used. For relationship extraction, mainly the features of the head and tail entities are spliced, and after splicing the corresponding features, a multi-layer perceptron is used for classification.

[0055] Entity-relation joint extraction techniques can effectively model the intrinsic features of entities and relations in text. However, their ability to model context is weak, while medical knowledge extraction in the medical field often heavily relies on contextual information. This is reflected in: 1) Current entity-relation joint extraction techniques struggle to address the problem of polysemy, where the same word may represent different types of entities in different contexts, such as "disease" and "symptom"; 2) Current entity-relation joint extraction techniques have difficulty handling unfamiliar technical terms, while the application scenario of knowledge self-learning involves extracting knowledge from unseen data. Given the complexity of medical knowledge, this means that many entity and relation models have never been seen in the training set before, making it difficult for current methods to effectively address this issue.

[0056] Therefore, this application provides a medical knowledge self-learning technology based on a contextual attention mechanism, which introduces a multi-perspective attention mechanism to model contextual information in medical texts, thereby extracting rich contextual information and accurately realizing medical knowledge self-learning.

[0057] The following will provide a detailed description of a medical knowledge extraction method, apparatus, computer-readable storage medium, and computer device provided in the embodiments of this application. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0058] like Figure 1 The diagram shown is a flowchart illustrating the medical knowledge extraction method provided in an embodiment of this application. This method is applied in a computer device and includes the following steps.

[0059] 101. Obtain the medical text and encode it to obtain the encoding features of each character in the medical text.

[0060] Obtain medical text. Medical text can take the form of medical information from textbooks, guidelines, drug instructions, medical records, diagnoses, medical examination reports, etc. This medical information is preprocessed, such as through standardized format conversion, sentence segmentation, and word segmentation, to obtain the preprocessed medical text. This preprocessed medical text includes the text subject, text subheadings, and the text content itself. The format of the obtained medical text can be as follows: [CLS]Text Subject [SEP]Text Subheading [SEP]Text Content [SEP].

[0061] The process involves acquiring medical text and encoding it. For example, a BERT pre-trained language model can be used to perform semantic encoding on the acquired medical text to obtain the encoded features of each character. Specifically, if the medical text is in Chinese, each character includes not only the Chinese character itself but also each punctuation mark; if the medical text is in English, each character includes every letter of every English word and every punctuation mark. If the medical text is in a different language, the meaning of each character will vary.

[0062] In this embodiment, medical text in Chinese is used as an example for illustration. After semantic encoding processing of the medical text, the encoded features w∈R of each character w are obtained. d The encoding features of each character can exist in the form of an encoding vector, which is illustrated using a 1*512 dimension example.

[0063] In one embodiment, each step of the method described in this application can be automatically implemented using a medical knowledge extraction model. Correspondingly, this medical knowledge extraction model may include an input layer, an encoding layer (encoding module), a context attention layer (context attention module), and an output layer. The input to this medical knowledge extraction model is medical text, which is input from the input layer to the encoding layer. The encoding layer may be composed of a BERT pre-trained language model, which performs encoding processing to obtain the encoding features of each character, or it may be an encoding layer with other structures. The context attention layer will be described in detail below. The context attention layer includes all the contents of steps 102 to 104. The context attention layer mainly includes a feature determination part (feature determination module), an entity extraction part (entity extraction module), and a relation extraction part (relation extraction module), which will be described in detail below.

[0064] 102. For each subtext of arbitrary length in a medical text, the feature representation of each subtext is determined by an attention mechanism based on the encoding features of each character included in each subtext.

[0065] Consider a continuous text of arbitrary length in a medical text as a subtext. Assuming the length of the medical text is L, each subtext can be represented as span[i,j], where i<=j, i>=1, j<=L, where i and j are the subtext indices in the medical text. The index of the first character in the medical text starts from 1. In some other embodiments, the index of the first character in the medical text can also start from 0, but this does not affect the solution of the present invention.

[0066] For example, a medical text might include "Capillary malformations commonly occur on the face, manifesting as red macules." Corresponding subtexts could include "capillary malformation," "malformation," "capillary," "blood vessel," "face," "facial region," "commonly occurs in," "occurs in," etc. It should be noted that these are just some examples of subtexts listed here and do not constitute a limitation on the subtexts.

[0067] For each subtext of arbitrary length in a medical text, an attention mechanism is used to determine the feature representation of each subtext.

[0068] Specifically, such as Figure 2 As shown, step 102 includes the following steps.

[0069] 201. For each subtext of arbitrary length in a medical text, obtain the encoding features of each character included in each subtext.

[0070] For example, if a subtext is "capillary malformation", then the subtext includes the encoding features of six characters: hair, thin, blood, tube, malformation, and shape.

[0071] 202. Through an attention mechanism, the encoded features of each character included in each subtext are processed to determine the overall feature representation of each subtext.

[0072] For the encoded feature w of each character in each subtext, determine the weight of each character in the corresponding subtext span[i,j]. For example, use a multilayer perceptron to process the encoded features of each character in each subtext to obtain the weight of each character in each subtext. Specifically, input the encoded features of each character in the subtext into the multilayer perceptron, process them using the multilayer perceptron, and use the result obtained by the multilayer perceptron as the weight of the corresponding character in the subtext. The weight of the corresponding character can be represented by the following formula (1).

[0073] a k =MLP(w k (1)

[0074] Where k is the index variable of the corresponding word in the span, and its value ranges from [i,j]. k That is, the feature encoding of the word with index k, MLP stands for Multilayer Perceptron processing, a k This is the weight of the corresponding character with index k. The weight of each character in each subtext can be obtained through formula (1).

[0075] After obtaining the weight of each character in each sub-text, the feature codes of each character in each sub-text are weighted and summed according to their weights to obtain the overall feature representation of each sub-text. An attention mechanism is used in the overall feature representation to better reflect the features of the sub-text as a whole, thereby improving the accuracy of the overall feature representation of the sub-text.

[0076] The overall feature representation of the subtext can be expressed using the following formula (2). Wherein, This represents the overall feature representation of the subtext span[i,j].

[0077]

[0078] 203. The feature representation of each subtext is determined based on the encoding features of the starting character, the overall feature representation, and the encoding features of the ending character.

[0079] Since the determination of each subtext is random, boundary information is equally important. Therefore, after obtaining the overall feature representation of the subtext, it is also necessary to determine its boundary information. In one embodiment, the boundary information can be represented using the encoded features of the start and end words of each subtext.

[0080] The self-feature representation of each subtext is determined by using the encoding features of the starting character, the overall feature representation, and the encoding features of the ending character. For example, the encoding features of the starting character, the overall feature representation, and the encoding features of the ending character of each subtext are concatenated to obtain the concatenated features of each subtext. If the encoding features of each character are 1*512 dimensions, the self-feature representation of the concatenated subtext is also 1*512 dimensions. Then, a multilayer perceptron is used to process the concatenated features of each subtext to obtain the self-feature representation of each subtext. That is, the concatenated features of the subtext are input into the multilayer perceptron for processing to obtain the self-feature representation of the subtext.

[0081] The self-feature representation of a subtext includes not only the overall feature representation of each subtext, but also the boundary feature representation of each subtext. The self-feature representation of a subtext can represent the features of the subtext from multiple aspects, thereby improving the accuracy of the self-feature representation of the subtext.

[0082] The process of obtaining the overall feature representation of the subtext can be represented by the following formula (3). Where, h [i:j] This is the feature representation of the subtext span[i,j].

[0083]

[0084] The above section utilizes the feature determination module within the medical knowledge extraction model to determine the unique feature representation of each sub-text. It's important to note that the inputs to both the entity extraction and relation extraction parts include the unique feature representation of each sub-text and the encoded features of each character within the sub-text.

[0085] 103. Based on the characteristic representation of each sub-text, determine the target entity to which each sub-text belongs, so as to obtain the entity set of the medical text.

[0086] In one embodiment, the target entity to which each subtext belongs can be determined based on its own feature representation and the contextual feature representation of different entity categories, thus obtaining the entity set of the medical text. Specifically, the entity extraction module in the medical knowledge extraction model is used to determine the target entity to which each subtext belongs based on its own feature representation and the contextual feature representation of different entity categories.

[0087] The different entity categories in this application embodiment are entity categories used for entity classification. These different entity categories may include M entity categories, where each of the M entity categories includes M-1 different entities and one category that is not an entity. For example, the different entity categories may include entities such as disease name, drug name, clinical manifestation, and symptom name, as well as five categories such as "not an entity". It should be noted that these five categories are merely illustrative; other embodiments may include more or fewer entity categories.

[0088] Since different entity categories focus on different contextual information, different contextual feature representations are extracted for different entity categories based on the subtext's own feature representation. Then, based on the contextual feature representations of different entity categories, the target entity to which each subtext belongs is determined.

[0089] In one embodiment, such as Figure 3 As shown, the steps for determining the target entity to which each subtext belongs based on its own feature representation and the contextual feature representation of different entity categories include the following steps.

[0090] 301. Based on the feature representation of each subtext and the encoding features of each character in the corresponding subtext, determine the context feature representation of different entity categories extracted from the corresponding subtext.

[0091] In one embodiment, multilayer perceptrons for entity classification of different entity categories are used to process the feature representation of each subtext and the encoded features of each character in the subtext to determine the entity attention weights corresponding to different entity categories for each character in the subtext. Based on the entity attention weights corresponding to different entity categories for each character in the subtext and the encoded features of each character in the subtext, the context feature representations of different entity categories extracted from the subtext body are determined. Where there are M different entity categories, there are M corresponding multilayer perceptrons for entity classification.

[0092] For the feature representation h of the subtext span[i,j] [i:j] Different contextual feature representations are extracted for different entity categories. Assuming there are M different entity categories, and the value of m ranges from [1,M], the following explanation will be given using entity category m as an example to illustrate how to determine the contextual feature representation extracted based on span[i,j].

[0093] Using the multilayer perceptron for entity classification corresponding to entity category m, the feature representation of the subtext span[i,j] and the encoded features of each character in the subtext are processed to determine the entity attention weight of each character in the subtext span[i,j] corresponding to entity category m. The process of determining the entity attention weight of the subtext span[i,j] corresponding to entity category m can be represented by formula (4).

[0094]

[0095] Among them, MLP (m) This represents the entity category classification perceptron corresponding to entity category m. The subtext span[i,j] represents the entity attention weight of the character with index k corresponding to entity category m.

[0096] Based on the entity attention weights of each character in the subtext span[i,j] corresponding to entity category m and the encoding features of each character in the subtext span[i,j], the contextual feature representation extracted from the subtext span[i,j] for entity category m is determined. For example, the entity attention weights of each character in the subtext span[i,j] corresponding to entity category m and the encoding features of each character in the subtext span[i,j] are weighted and summed to determine the contextual feature representation extracted from span[i,j] for category m.

[0097] Formula (5) can be used to represent the contextual feature representation extracted based on span[i,j] for classification category m. The process.

[0098]

[0099] Thus, for each category m, different contextual feature representations can be extracted based on each subtext. Finally, it is possible to extract the contextual feature representation of each subtext for different categories, or to extract the contextual feature representation of different entity categories based on each subtext.

[0100] Since different entity categories focus on different contextual information, we extract contextual feature representations for each subtext for different categories to extract contextual information. For example, for a disease name, we look for contextual features related to the disease name in the context; for a drug name, we look for contextual features related to the drug name in the context. This improves the accuracy of contextual feature representations for each different category, which in turn improves the accuracy of entity extraction.

[0101] 302. Using a multilayer perceptron for entity classification, perform a first classification process on the self-feature representation of the corresponding subtext and the contextual feature representation of different entity categories extracted from the corresponding subtext to determine the target entity category of the corresponding subtext.

[0102] After obtaining the contextual feature representations of different entity categories extracted from each subtext, for each category, features are constructed for each subtext under each entity category based on the subtext's own feature representation and the extracted contextual feature representation. Using the multilayer perceptron for entity classification corresponding to each entity category, the features constructed under each entity category are subjected to first classification processing to determine the entity classification score of the subtext for different entity categories. The entity category corresponding to the highest entity classification score among the entity classification scores of different entity categories is determined as the target entity category of the subtext.

[0103] Continuing with the example above, for entity category m, h is represented by the features of the subtext span[i,j]. [i:j] and extracted contextual feature representation Construct features for the subtext span[i,j] under entity category m. For example, represent the features of the subtext span[i,j] itself as h. [i:j] and extracted contextual feature representation The concatenated features obtained after concatenation are used as the constructed features. Formula (6) can be used to represent the process of constructing features for the classification category m, where span[i,j] is the concatenation feature. For the features constructed.

[0104]

[0105] Using the entity classification multilayer perceptron corresponding to entity category m, the constructed features are first classified to determine the entity classification score of the subtext belonging to category m. For example, the constructed features are input into the entity classification multilayer perceptron (MLP) corresponding to entity category m. (m) Perform the first classification process to obtain the entity classification score of the subtext span[i,j] belonging to entity category m. The corresponding process can be represented by formula (7).

[0106]

[0107] Thus, we can obtain the entity classification scores of the subtext span[i,j] for different entity categories. The entity category corresponding to the highest entity classification score is determined as the target entity category of the subtext. If entity category m is a symptom name, and the subtext span[i,j] has the highest entity classification score for entity category m, then the target entity category of the subtext is entity category m.

[0108] The content in step 103 is what needs to be completed in the entity extraction part, such as... Figure 4 The diagram shown is a network topology diagram of the entity extraction part provided in the embodiments of this application. It can be combined with... Figure 4 This should be understood in conjunction with the content mentioned above. Figure 4 The information input from the middle to the top of the MLP includes w i , w j and

[0109] In the medical entity extraction section of this application embodiment, a multi-view contextual attention mechanism is used to capture contextual information, adaptively capturing appropriate contextual information for different entity categories, thereby further improving the accuracy of entity extraction.

[0110] 303. The target entity corresponding to the target entity category is determined as the target entity to which the corresponding subtext belongs, so as to determine the target entity to which each subtext belongs.

[0111] The entity corresponding to the target entity category is the target entity to which the subtext belongs. Continuing with the example above, the symptom name is taken as the target entity to which the subtext span[i,j] belongs.

[0112] In this way, the target entities to which different subtexts belong can be determined. For example, for the medical text "Capillary malformation commonly occurs on the face and manifests as red macules.", the target entity to which the subtext "capillary malformation" belongs is the disease name, the target entity to which the subtext "face" belongs is the location, the target entity to which the subtext "red macules" belongs is the symptom name, and the subtexts "occurs on", "commonly occurs on", and "manifests as" do not belong to entities, etc.

[0113] After identifying the target entity to which each subtext in the medical text belongs, subtexts that are not entities are removed to obtain the entity set of the medical text. The entity set includes the subtexts and the target entities to which the subtexts belong. For example, the entity set for the medical text "Capillary malformation commonly occurs on the face and manifests as red macules." includes: "capillary malformation" - disease name, "face" - location, and "red macules" - symptom name.

[0114] 104. For any two sub-texts in the entity set, the target relationship between the target entities to which the two sub-texts belong is determined by a multi-view contextual attention mechanism based on the feature representations of the two sub-texts themselves.

[0115] Since the entity set includes both subtexts and the target entity to which the subtexts belong, for any two subtexts in the entity set, we can know the target entity to which those two subtexts belong.

[0116] For any two sub-texts in the entity set, based on the self-feature representations of the two sub-texts and a multi-view contextual attention mechanism, the contextual attention feature representation of the intermediate text between the two sub-texts is determined. Then, based on the self-feature representations of the two sub-texts and the contextual attention feature representation of the intermediate text, the target relationship between the target entities to which the two sub-texts belong is determined. Specifically, the relationship extraction module in a medical knowledge extraction model can be used to determine the target relationship between the target entities to which the two sub-texts belong. This embodiment uses a multi-view contextual attention mechanism, considering the most relevant contextual information from multiple perspectives to improve the accuracy of relationship extraction.

[0117] In this context, intermediate text refers to the text between two sub-texts. For example, in the medical text "Capillary malformation commonly occurs on the face and manifests as red macules.", if the two sub-texts with identified target entities are "capillary malformation" and "red macules", then the intermediate text between the two sub-texts is "commonly occurs on the face and manifests as."

[0118] If the target entity of the first subtext in any two subtexts is a head entity and the target entity of the second subtext is a tail entity, for example, the target entity of "capillary malformation" is a head entity and the target entity of "red rash" is a tail entity, then the multi-perspective contextual attention mechanism includes at least two of the following: head entity perspective, tail entity perspective, and global perspective. The contextual attention feature representation of the intermediate text between the two subtexts includes at least two of the following: attention feature representation under the head entity perspective of the intermediate text, attention feature representation under the tail entity perspective of the intermediate text, and attention feature representation under the global perspective of the intermediate text.

[0119] In this embodiment, a multi-view contextual attention mechanism is used to determine the contextual attention feature representation of the text (intermediate text) between any two sub-texts. Based on the self-feature representations of the two sub-texts and the contextual attention feature representation, the target relationship between the target entities to which the two sub-texts belong is determined.

[0120] This application embodiment uses a multi-view approach, including a head entity view, a tail entity view, and a global view, as an example for illustration. Any two sub-texts can be span[i1,j1] and span[i2,j2], respectively, and the self-feature representations between these two sub-texts are as follows: Let span[j1,i2] be the intermediate text between two subtexts, where i1≤j1≤i2≤j2.

[0121] In one embodiment, such as Figure 5 As shown, step 104 includes the following steps.

[0122] 401. Based on the attention mechanism, the first attention feature representation of the intermediate text is determined from the perspective of the head entity, according to the feature representation of the first sub-text and the encoding features of the intermediate text.

[0123] Specifically, a multilayer perceptron corresponding to the head entity viewpoint is used to process the feature representation of the first sub-text and the encoded features of each character in the intermediate text to determine the first attention weight of each character in the intermediate text from the head entity viewpoint. The input to the multilayer perceptron corresponding to the head entity viewpoint is the concatenated feature representation of the first sub-text and the encoded features of each character in the intermediate text.

[0124] For example, the feature representation of the first sub-text is concatenated with the encoded features of each character in the intermediate text. The resulting feature is still 512-dimensional. For instance, the feature obtained by concatenating the feature representation of the first sub-text with the encoded feature of the first character in the intermediate text, the feature obtained by concatenating the feature representation of the first sub-text with the encoded feature of the second character in the intermediate text, and so on. The concatenated features are then processed using a multilayer perceptron corresponding to the head entity view to determine the first attention weight of each character in the intermediate text from the head entity view. For example, the concatenated features are input into the multilayer perceptron corresponding to the head entity view for processing to obtain the first attention weight of each character in the intermediate text from the head entity view.

[0125] The process of obtaining the first attention weight can be represented by formula (8).

[0126]

[0127] In formula (8), k∈(j1,i2) represents the word subscript in the intermediate text. This represents the first attention weight of the word with index k in the middle text from the perspective of the head entity.

[0128] After obtaining the first attention weight of each character in the intermediate text from the perspective of the head entity, the encoded features of each character in the intermediate text are weighted and summed according to the first attention weight of each character in the intermediate text to obtain the first attention feature representation of the intermediate text from the perspective of the head entity. The first attention feature representation of the intermediate text span[j1,i2] from the perspective of the head entity can be obtained using formula (9). The process.

[0129]

[0130] The process of obtaining the first attention feature representation of the intermediate text from the perspective of the head entity is the process of selecting the context information most relevant to the head entity. The multilayer perceptron corresponding to the head entity perspective focuses on the context information most relevant to the head entity from the perspective of the head entity, thereby improving the accuracy of relation extraction.

[0131] 402. Based on the attention mechanism, the second attention feature representation of the intermediate text is determined from the perspective of the tail entity, according to the feature representation of the second sub-text and the encoding features of the intermediate text.

[0132] Specifically, a multilayer perceptron corresponding to the tail entity's perspective is used to process the tail entity's own feature representation and the encoded features of each character in the intermediate text to determine the second attention weight of each character in the intermediate text from the tail entity's perspective. The input to the multilayer perceptron corresponding to the tail entity's perspective is the concatenated feature representation of the second sub-text and the encoded features of each character in the intermediate text.

[0133] For example, the feature representation of the tail entity is concatenated with the encoded features of each character in the middle text. The concatenated features are then processed using a multilayer perceptron corresponding to the tail entity's perspective to determine the second attention weight of each character in the middle text from the tail entity's perspective. For instance, the features obtained by concatenating the feature representation of the tail entity with the encoded features of the first character in the middle text, or by concatenating the feature representation of the tail entity with the encoded features of the second character in the middle text, are then input into the multilayer perceptron corresponding to the tail entity's perspective for processing to obtain the second attention weight of each character in the middle text from the tail entity's perspective.

[0134] The process of obtaining the second attention weight can be represented by formula (10).

[0135]

[0136] In formula (10), k∈(j1,j2) represents the word subscript in the intermediate text. This represents the second attention weight of the character with index k in the middle text from the perspective of the tail entity.

[0137] After obtaining the second attention weight of each character in the intermediate text from the tail entity perspective, the encoded features of each character in the intermediate text are weighted and summed according to the second attention weight of each character in the intermediate text to obtain the second attention feature representation of the intermediate text from the tail entity perspective. Formula (11) can be used to represent the second attention feature representation of the intermediate text span[j1,i2] from the tail entity perspective. The process.

[0138]

[0139] The process of obtaining the second attention feature representation of the intermediate text from the perspective of the tail entity is the process of selecting the context information most relevant to the tail entity. The multilayer perceptron corresponding to the tail entity perspective focuses on the context information most relevant to the tail entity from the perspective of the tail entity, thereby improving the accuracy of relation extraction.

[0140] 403. Based on the attention mechanism, the third attention feature representation of the intermediate text is determined from the global perspective according to the encoding features of each character in the intermediate text.

[0141] In this process, the encoded features of each character in the intermediate text are processed using a multilayer perceptron corresponding to the global perspective to determine the third attention weight of each character in the intermediate text from the global perspective. For example, the encoded features of each character in the intermediate text are input into the multilayer perceptron corresponding to the global perspective for processing to obtain the third attention weight of each character in the intermediate text from the global perspective. The third attention weight can be expressed using formula (12). The process.

[0142]

[0143] Then, based on the third attention weight of each character in the intermediate text, the encoded features of each character in the intermediate text are weighted and summed to obtain the third attention feature representation of the intermediate text from a global perspective. Formula (13) can be used to represent the third attention feature representation of the intermediate text span[j1,i2] from a global perspective. The process.

[0144]

[0145] The process of obtaining the third attention feature representation of the intermediate text from a global perspective is the process of selecting the most important contextual information from a global perspective. The multilayer perceptron corresponding to the global perspective only focuses on the importance of each word itself, without starting from the head entity perspective or the tail entity perspective.

[0146] 404. Based on the first attention feature representation, the second attention feature representation, and the third attention feature representation, determine the contextual attention feature representation of the intermediate text between the two target entities.

[0147] The first, second, and third attention feature representations can be fused, such as by concatenation, to obtain an attention fusion feature. The dimension of this attention fusion feature is the same as the dimensions of the first, second, and third attention feature representations. This attention fusion feature is then defined as the contextual attention feature representation of the intermediate text between the two target entities. Specifically, formula (14) can be used to represent the contextual attention feature representation of the intermediate text.

[0148]

[0149] This contextual attention feature representation includes not only the context most relevant to the head entity and the context most relevant to the tail entity, but also the features related to each word in the middle text itself. It integrates contextual information from multiple perspectives and improves the accuracy of relation extraction.

[0150] 405. Based on the self-feature representations of the two target entities and the contextual attention feature representations of the intermediate text, determine the target relationship between the target entities to which the two sub-texts belong.

[0151] The intrinsic feature representations of the two sub-texts and the contextual attention feature representation of the intermediate text are fused (e.g., concatenated) to obtain relationship fusion features. A multilayer perceptron, corresponding to different relationship categories, is then used to perform a second classification process on the relationship fusion features to determine the target relationship between the two target entities. In this embodiment, the fusion features consider both the intrinsic feature representations of the target entity's sub-texts and the contextual attention feature representations of the intermediate text, improving the accuracy of the relationship fusion features and thus enhancing the accuracy of target relationship determination.

[0152] In this embodiment, since the subtext and the target entity to which the subtext belongs have already been extracted in the entity extraction section, if a relationship is determined between two target entities in this step, the specific target relationship can be determined directly through the target entity corresponding to the subtext. For example, "capillary malformation" is the name of a disease, and "face" is the location. If there is a relationship between the subtext "capillary malformation" and the subtext "face", then the relationship is the location, indicating that the location where "capillary malformation" occurs is the "face". Therefore, the different relationship categories here include two categories: "yes" and "no". "Yes" indicates that there is a relationship between the two target entities, and this relationship is the target relationship. "No" indicates that there is no relationship between the two target entities. The relationship classification multilayer perceptron corresponding to different relationship categories includes the relationship classification multilayer perceptron corresponding to "yes" and the relationship classification multilayer perceptron corresponding to "no".

[0153] In one embodiment, different relation categories include not only whether a relationship exists between the target entities to which two sub-texts belong, but also the specific relation category. Assume there are N relation categories, where N-1 represents the specific relationship between the target entities to which the sub-texts belong, such as location or symptom, and also includes relation categories where no relationship exists. Correspondingly, there are also N relation classification multilayer perceptrons for different relation categories, with one relation category corresponding to one relation classification multilayer perceptron.

[0154] Correspondingly, the above-mentioned step of using a relation classification multilayer perceptron corresponding to different relation categories to perform a second classification process on the relation fusion features in order to determine the target relationship between the target entities to which the two sub-texts belong includes: using a relation classification multilayer perceptron corresponding to each relation category to perform a second classification process on the relation fusion features to obtain the relation classification scores of the target entities to which the two sub-texts belong as the corresponding relation categories, and determining the relation corresponding to the highest relation classification score among the relation classification scores as the target relationship between the target entities to which the two sub-texts belong.

[0155] For example, taking the nth relation category as an example, where n ranges from [1, N], the relationship between two subtexts span[i1, j1] and span[i2, j2] is the relation classification score of the nth relation category. As shown in formula (15).

[0156]

[0157] Among them, MLP (n) This is a multilayer perceptron for classifying relations, corresponding to the nth relation category.

[0158] For each category, the process is performed as shown in formula (15) to obtain the relationship between the target entities to which the two sub-texts belong, which is the relationship classification score of the corresponding relationship category. The relationship corresponding to the highest relationship classification score is determined as the target relationship between the target entities to which the two sub-texts belong.

[0159] The content in step 104 is what needs to be completed in the relation extraction part, such as... Figure 6 The diagram shown is a network topology diagram of the relationship extraction part provided in an embodiment of this application. It can be combined with... Figure 6 This should be understood together with the content on relation extraction mentioned above. Figure 6 The information input to the top of the MLP includes and

[0160] In the relation extraction part of this application embodiment, a multi-view contextual attention mechanism is used to capture contextual information of different granularities from local and global perspectives, thereby further improving the accuracy of relation extraction.

[0161] It is important to note that the medical knowledge extraction model includes both entity extraction and relation extraction. Therefore, entity extraction and relation extraction in this application are performed within a single model. This approach reduces error propagation caused by multi-module coupling compared to existing cascaded solutions. Existing cascaded solutions extract entities first and then relations separately, with non-differentiable connections between modules, thus errors from upstream can propagate to downstream. In contrast, this application uses joint entity and relation extraction, integrating both parts into a single model, thus avoiding the error propagation problem caused by multi-module coupling.

[0162] 105. The target entity to which each subtext belongs in the medical text and the target relationship between any two target entities to which any two subtexts belong are defined as the medical knowledge of the medical text.

[0163] After determining the target entity to which each subtext belongs in the medical text and the target relationship between any two target entities, the medical knowledge of the medical text is output. This medical knowledge includes the target entity to which each subtext belongs and the target relationship between any two target entities.

[0164] In the above method embodiments, an attention mechanism is used to determine the self-feature representation of each sub-text of arbitrary length in the medical text, thereby improving the accuracy of the self-feature representation of each sub-text. Moreover, since different entities have different contextual information, their contextual feature representations are also different. Therefore, based on the self-feature representation of each sub-text and the contextual feature representations of different entities, the target entity to which each sub-text belongs can be accurately determined, improving the accuracy of the target entity to which the sub-text belongs. Furthermore, since a multi-perspective contextual attention mechanism is used to determine the target relationship between the target entities to which two sub-texts belong, the contextual information from different perspectives is considered, improving the accuracy of the target relationship determination between target entities. Thus, the accuracy of medical knowledge in the medical text is improved.

[0165] Figure 7 This is another flowchart illustrating the medical knowledge extraction method provided in this application embodiment. This medical knowledge extraction method can also be called a medical knowledge extraction model training method, which is used to train a medical knowledge extraction model so as to use the trained medical knowledge extraction model to extract medical knowledge. In this embodiment, the word "training" is added before the name to distinguish it from the name in the above text. Specifically, the steps for training the medical knowledge extraction model include the following.

[0166] 501. Obtain multiple training samples and an initial medical knowledge extraction model. Each training sample includes training medical text, target entity labels in the training medical text, and target relationship labels between any two training target entities.

[0167] Obtain the training dataset, which includes multiple training samples. Each training sample includes training medical text, target entity labels in the training medical text, and target relationship labels between any two training target entities.

[0168] The training medical text in the training samples can be medical information obtained from a vast amount of textbooks, guidelines, drug instructions, medical records, disease diagnoses, medical examination reports, etc. This medical information undergoes preprocessing, such as standardized format conversion, sentence segmentation, and word segmentation, to obtain the training medical text. The training medical text can include the text topic, text subheadings, and the text content itself. The format of the training medical text can be as follows: [CLS]Text Topic [SEP]Text Subheading [SEP]Text Content [SEP].

[0169] Among them, the target entity label is the real target entity that has been determined in advance in the training medical text, and the target relationship label is the real target relationship between the target time extracted from the training medical text.

[0170] The initial medical knowledge extraction model is a medical knowledge extraction model with network parameters to be updated. This initial medical knowledge extraction model includes an input layer, an encoding layer (encoding module), a context attention layer (context attention module), and an output layer. The context attention layer includes a feature determination part (feature determination module, used to determine the feature representation of each sub-text), an entity extraction part (entity extraction module), and a relation extraction part (relation extraction module). The entity extraction part sets M entity categories and the corresponding multilayer perceptron for entity classification of the M entity categories. The relation extraction part sets N relation categories and the corresponding multilayer perceptron for relation classification of the N relation categories. M and N are both integers greater than 1.

[0171] 502. The initial medical knowledge extraction model is used to encode the training medical text to obtain the training encoding features of each character in the training medical text.

[0172] The encoding layer of the initial medical knowledge extraction model is used to encode the training medical text to obtain the training encoding features of each character in the training medical text.

[0173] 503. For each training subtext of arbitrary length in the training medical text, the training feature representation of each training subtext is determined by an attention mechanism based on the training encoding features of each character included in each training subtext.

[0174] Specifically, the feature determination part of the context attention layer in the initial medical knowledge extraction model is used to determine the training feature representation of each training subtext.

[0175] Specifically, for each training sub-text of arbitrary length in the training medical text, the training encoding features of each character included in each training sub-text are obtained; the training encoding features of each character included in each training sub-text are processed through an attention mechanism to determine the overall training feature representation of each training sub-text; and the training feature representation of each training sub-text is determined based on the training encoding features of the starting character, the overall training feature representation, and the training encoding features of the ending character.

[0176] For more detailed information, please refer to the corresponding description in the above section on "For each subtext of arbitrary length in a medical text, determine the self-feature representation of each subtext based on the encoding features of each character included in each subtext through an attention mechanism," which will not be repeated here.

[0177] 504. Based on the training feature representation of each training sub-text, determine the training target entity to which each training sub-text belongs, so as to obtain the training entity set of the training medical text.

[0178] Specifically, the training target entity to which each training sub-text belongs can be determined based on the training feature representation of each training sub-text and the training context feature representation of different entity categories, so as to obtain the training entity set of the training medical text.

[0179] Specifically, the entity extraction part of the context attention layer in the initial medical knowledge extraction model is used to determine the training target entity to which each training sub-text belongs, so as to obtain the training entity set of the training medical text.

[0180] Specifically, the training context feature representations of different entity categories extracted from the training sub-texts can be determined based on the training self-feature representation of each training sub-text and the training encoding features of each character in the training sub-text; using an entity classification multilayer perceptron, the training self-feature representation of the training sub-texts and the training context feature representations of different entity categories extracted from the training sub-texts are subjected to a first classification process to determine the training target entity category of the training sub-texts; and the training entity corresponding to the training target entity category is determined as the training target entity to which each training sub-text belongs.

[0181] The step of determining the training context feature representations of different entity categories extracted from the training subtext based on the training self-feature representation of each training subtext and the training encoding features of each character in the training subtext includes: using a multilayer perceptron for entity classification corresponding to different entity categories to process the training self-feature representation of each training subtext and the training encoding features of each character in the training subtext to determine the training entity attention weights of the training subtext corresponding to different entity categories; and determining the training context feature representations of different entity categories extracted from the training subtext based on the training entity attention weights and the training encoding features of each character in the training subtext.

[0182] The step described above, which utilizes a multilayer perceptron for entity classification to perform a first classification process on the training subtext's own feature representation and the training context feature representations extracted from the training subtext for different entity categories, to determine the training target entity category of the training subtext, includes: for each entity category, constructing training features for each training subtext under each entity category based on the training subtext's own feature representation and the extracted training context feature representation; using the multilayer perceptron for entity classification corresponding to each entity category to perform a first classification process on the training features constructed under each entity category, to determine the training entity classification score of the training subtext for different entity categories; and determining the training entity category corresponding to the highest training entity classification score as the training target entity category of the training subtext. After obtaining the training target entity category, the training entity corresponding to the training target entity category is determined as the training target entity to which each training subtext belongs.

[0183] After obtaining the training target entity to which each training subtext belongs, the training subtexts that are not entities are removed to obtain the training entity set of the training medical text. The training entity set includes the training subtexts and the training target entities to which the training subtexts belong.

[0184] For more detailed information, please refer to the corresponding description in the step above, "Based on the self-feature representation of each sub-text and the contextual feature representation of different entity categories, determine the target entity to which each sub-text belongs, so as to obtain the entity set of the medical text," which will not be repeated here.

[0185] 505. The first loss value is determined based on the training target entity in the training entity set of the training medical text and the target entity label of the training medical text.

[0186] The training target entity and the target entity label are compared to calculate the error, and the first loss value is obtained based on the error.

[0187] 506. For any two training sub-texts in the training entity set, the training target relationship between the training target entities to which the two training sub-texts belong is determined based on the training feature representations of the two training sub-texts and through a multi-view contextual attention mechanism.

[0188] Specifically, the relation extraction part in the context attention layer of the initial medical knowledge extraction model is used to determine the training target relationship between the training target entities to which the two training sub-texts belong.

[0189] Specifically, for any two training sub-texts in the training entity set, based on the training self-feature representations of the two training sub-texts and a multi-view context attention mechanism, the training context attention feature representation of the intermediate training text between the two training sub-texts is determined; based on the training self-feature representations of the two training sub-texts and the training context attention feature representation of the intermediate training text, the training target relationship between the training target entities to which the two training sub-texts belong is determined.

[0190] In this context, the training target entity to which the first training sub-text belongs is the training head entity, and the training target entity to which the second training sub-text belongs is the training tail entity. The step of determining the training context attention feature representation of the intermediate training text between the two training sub-texts based on the training self-feature representations of the two training sub-texts and a multi-view context attention mechanism includes: determining the first training attention feature representation of the intermediate training text from the head entity perspective based on the training self-feature representation of the first training sub-text and the training encoding features of the intermediate training text; determining the second training attention feature representation of the intermediate training text from the tail entity perspective based on the training self-feature representation of the second training sub-text and the training encoding features of the intermediate training text; determining the third training attention feature representation of the intermediate training text from a global perspective based on the training encoding features of each character in the intermediate training text; and determining the training context attention feature representation of the intermediate training text between the two training sub-texts based on the first, second, and third training attention feature representations.

[0191] The step of determining the training target relationship between the training target entities to which the two training sub-texts belong, based on the training self-feature representations of the two training sub-texts and the training context attention feature representation of the training intermediate text, includes: fusing the training self-feature representations of the two training sub-texts and the training context attention feature representation of the training intermediate text to obtain training relationship fusion features; and using a relation classification multilayer perceptron corresponding to different relation categories to perform a second classification process on the training relationship fusion features to determine the training target relationship between the training target entities to which the two training sub-texts belong.

[0192] For more detailed information, please refer to the corresponding description in the above text on the step "For any two sub-texts in the entity set, determine the target relationship between the target entities to which the two sub-texts belong based on the feature representations of the two sub-texts and through a multi-view contextual attention mechanism", which will not be repeated here.

[0193] 507. The second loss value is determined based on the training target relation and the target relation label.

[0194] The training target relation and the target relation label are compared to calculate the error, and the second loss value is obtained based on the error.

[0195] 508. Determine the total loss value based on the first loss value and the second loss value, and update the initial medical knowledge extraction model based on the total loss value to obtain the medical knowledge extraction model.

[0196] Add the first loss value and the second loss value together, or perform a weighted summation, to obtain the total loss value.

[0197] It should be noted that the overall loss value in this embodiment is determined based on the first loss value of the entity extraction part and the second loss value of the relation extraction part. Therefore, the entity extraction part and the relation extraction part in this embodiment are jointly trained, that is, entity extraction and relation extraction are in the same model. Compared with the existing cascaded scheme, this reduces the error propagation caused by multi-module coupling. The existing cascaded scheme first extracts entities and then extracts relations. The connection between modules is not differentiable, so the error in the upstream will be propagated to the downstream. However, in this application, entity extraction and relation extraction are in the same model, and this kind of problem will not occur.

[0198] The network parameters in the initial medical knowledge extraction model are updated based on the overall loss value until the training stopping condition is met, such as the number of training rounds reaching a preset number or the overall loss value converging. The network obtained after stopping training is the medical knowledge extraction model.

[0199] After obtaining the medical knowledge extraction model, the acquired medical text is processed to extract the medical knowledge within it. For details on the knowledge extraction process, please refer to the above text. [[ID=!21]]Figure 1 The parts shown will not be repeated here.

[0200] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0201] To facilitate better implementation of the medical knowledge extraction method of this application, this application also provides a medical knowledge extraction device. Please refer to... Figure 8 , Figure 8 This is a schematic diagram of the structure of a medical knowledge extraction device provided in an embodiment of this application. The medical knowledge extraction device 600 may include an acquisition module 601, an encoding module 602, a feature determination module 603, an entity extraction module 604, a relationship extraction module 605, and a knowledge determination module 606.

[0202] Module 601 is used to acquire medical text.

[0203] The encoding module 602 is used to encode the medical text to obtain the encoding features of each character in the medical text.

[0204] The feature determination module 603 is used to determine the feature representation of each subtext of arbitrary length in the medical text by means of an attention mechanism, based on the encoding features of each character included in each subtext.

[0205] Specifically, the feature determination module 603 is used to obtain the encoding features of each character included in each subtext of arbitrary length in the medical text; process the encoding features of each character included in each subtext through an attention mechanism to determine the overall feature representation of each subtext; and determine the self-feature representation of each subtext based on the encoding features of the starting character, the overall feature representation, and the encoding features of the ending character of each subtext.

[0206] The entity extraction module 604 is used to determine the target entity to which each sub-text belongs based on its own feature representation, so as to obtain the entity set of the medical text.

[0207] The entity extraction module 604 is specifically used to determine the target entity to which each subtext belongs based on its own feature representation and the context feature representation of different entity categories, so as to obtain the entity set of the medical text.

[0208] Specifically, the entity extraction module 604 is used to determine the contextual feature representations of different entity categories extracted from the corresponding subtext based on the self-feature representation of each subtext and the encoding features of each character in the corresponding subtext; to perform a first classification process on the self-feature representation of the corresponding subtext and the contextual feature representations of different entity categories extracted from the corresponding subtext using an entity classification multilayer perceptron, so as to determine the target entity category of the corresponding subtext; and to determine the entity corresponding to the target entity category as the target entity to which the corresponding subtext belongs, so as to obtain the target entity to which each subtext belongs.

[0209] The relationship extraction module 605 is used to determine the target relationship between the target entities to which the two sub-texts belong, based on the feature representations of the two sub-texts themselves and through a multi-view contextual attention mechanism, for any two sub-texts in the entity set.

[0210] Specifically, the relationship extraction module 605 is used to determine the context attention feature representation of the intermediate text between the two sub-texts based on the self-feature representations of the two sub-texts and a multi-view context attention mechanism; and to determine the target relationship between the target entities to which the two sub-texts belong based on the self-feature representations of the two sub-texts and the context attention feature representation of the intermediate text.

[0211] The knowledge determination module 606 is used to determine the target entity to which each subtext belongs in the medical text and the target relationship between any two target entities to which any two subtexts belong as the medical knowledge of the medical text.

[0212] For details on the specific content and beneficial effects of each module mentioned above, please refer to the corresponding descriptions in the methods section above; they will not be repeated here.

[0213] In one embodiment, this application also provides a medical knowledge extraction device. Please refer to... Figure 9 , Figure 9 This is another schematic diagram of the medical knowledge extraction device provided in an embodiment of this application. The medical knowledge extraction device 700 may include a training acquisition module 701, a training encoding module 702, a training feature determination module 703, a training entity extraction module 704, a first loss determination module 705, a training relationship extraction module 706, a second loss determination module 707, and an update module 708.

[0214] The training acquisition module 701 is used to acquire multiple training samples and an initial medical knowledge extraction model. Each training sample includes training medical text, target entity labels in the training medical text, and target relationship labels between any two training target entities to which the training medical texts belong.

[0215] The training encoding module 702 is used to encode the training medical text using the initial medical knowledge extraction model to obtain the training encoding features of each character in the training medical text.

[0216] The training feature determination module 703 is used to determine the training feature representation of each training sub-text of arbitrary length in the training medical text by means of an attention mechanism, based on the training encoding features of each character included in each training sub-text.

[0217] The training entity extraction module 704 is used to determine the training target entity to which each training sub-text belongs based on the training self-feature representation of each training sub-text, so as to obtain the training entity set of the training medical text.

[0218] The first loss determination module 705 is used to determine a first loss value based on the training target entity in the training entity set of the training medical text and the target entity label of the training medical text.

[0219] The training relationship extraction module 706 is used to determine the training target relationship between the training target entities to which the two training sub-texts belong, based on the training feature representations of the two training sub-texts and through a multi-view contextual attention mechanism, for any two training sub-texts in the training entity set.

[0220] The second loss determination module 707 determines a second loss value based on the training target relationship and the target relationship label.

[0221] The update module 708 is used to determine the total loss value based on the first loss value and the second loss value, and update the initial medical knowledge extraction model based on the total loss value to obtain the medical knowledge extraction model.

[0222] For details on the specific content and beneficial effects of each module mentioned above, please refer to the corresponding descriptions in the methods section above; they will not be repeated here.

[0223] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0224] Accordingly, embodiments of this application also provide a computer device, which can be a terminal or a server. For example... Figure 10 As shown, Figure 10This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device 800 includes a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, and a computer program stored on the memory 802 and executable on the processor. The processor 801 and the memory 802 are electrically connected. Those skilled in the art will understand that the computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0225] The processor 801 is the control center of the computer device 800. It connects various parts of the computer device 800 through various interfaces and lines. By running or loading software programs (computer programs) and / or modules stored in the memory 802, and calling data stored in the memory 802, it performs various functions of the computer device 800 and processes data, thereby monitoring the computer device 800 as a whole.

[0226] In this embodiment, the processor 801 in the computer device 800 loads the instructions corresponding to the processes of one or more applications into the memory 802 according to the following steps, and the processor 801 runs the applications stored in the memory 802 to achieve various functions:

[0227] A medical text is acquired and encoded to obtain the encoded features of each character in the medical text. For each subtext of arbitrary length in the medical text, based on the encoded features of each character included in each subtext, an attention mechanism is used to determine the feature representation of each subtext. Based on the feature representation of each subtext, the target entity to which each subtext belongs is determined to obtain the entity set of the medical text. For any two subtexts in the entity set, based on the feature representations of the two subtexts, a multi-view contextual attention mechanism is used to determine the target relationship between the target entities to which the two subtexts belong. The target entities to which each subtext belongs in the medical text and the target relationship between the target entities to which any two subtexts belong are determined as the medical knowledge of the medical text.

[0228] And / or,

[0229] Multiple training samples and an initial medical knowledge extraction model are obtained. Each training sample includes training medical text, target entity labels in the training medical text, and target relationship labels between any two training target entities to which the training medical texts belong. The initial medical knowledge extraction model is used to encode the training medical text to obtain the training encoding features of each character in the training medical text. For each training sub-text of arbitrary length in the training medical text, based on the training encoding features of each character included in each training sub-text, an attention mechanism is used to determine the training self-feature representation of each training sub-text. Based on the training self-feature representation of each training sub-text, the training target to which each training sub-text belongs is determined. The system obtains a training entity set for the training medical text; determines a first loss value based on the training target entities in the training entity set and the target entity labels of the training medical text; for any two training sub-texts in the training entity set, determines the training target relationship between the training target entities to which the two training sub-texts belong through a multi-view contextual attention mechanism based on the training self-feature representations of the two training sub-texts; determines a second loss value based on the training target relationship and the target relationship labels; determines a total loss value based on the first loss value and the second loss value, and updates the initial medical knowledge extraction model based on the total loss value to obtain a medical knowledge extraction model.

[0230] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0231] Optional, such as Figure 10 As shown, the computer device 800 also includes: a touch screen display 803, a radio frequency circuit 804, an audio circuit 805, an input unit 806, and a power supply 807. The processor 801 is electrically connected to the touch screen display 803, the radio frequency circuit 804, the audio circuit 805, the input unit 806, and the power supply 807.

[0232] The touch display screen 803 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 803 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the computer device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar devices. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program. The touch panel may cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the data to the processor 801 to determine the type of touch event. Subsequently, the processor 801 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and display panel can be integrated into the touch display screen 803 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 803 can be implemented as two independent components to achieve input and output functions. That is, the touch display screen 803 can also be used as part of the input unit 806 to achieve input functions.

[0233] In this embodiment, the touch display screen 803 is used to present a graphical user interface and receive operation commands generated by the user interacting with the graphical user interface.

[0234] The radio frequency circuit 804 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other computer devices, and to transmit and receive signals with network devices or other computer devices.

[0235] Audio circuitry 805 can be used to provide an audio interface between a user and a computer device via a speaker and a microphone. Audio circuitry 805 converts received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 805, converted back into audio data, and output to processor 801 for processing. The audio data is then transmitted via radio frequency circuitry 804 to, for example, another computer device, or output to memory 802 for further processing. Audio circuitry 805 may also include an earphone jack to facilitate communication between peripheral headphones and computer devices.

[0236] The input unit 806 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0237] Power supply 807 is used to supply power to various components of computer device 800. Optionally, power supply 807 can be logically connected to processor 801 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 807 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0238] although Figure 10 It seems there is a typo in the original text where [[ID=!21]] should probably be . I've translated it as it is but please check if that's correct. As not shown in the diagram, the computer device 800 may also include a camera, sensors, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0239] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0240] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0241] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of computer programs that can be loaded by a processor to execute steps in any of the medical knowledge extraction methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0242] A medical text is acquired and encoded to obtain the encoded features of each character in the medical text. For each subtext of arbitrary length in the medical text, based on the encoded features of each character included in each subtext, an attention mechanism is used to determine the feature representation of each subtext. Based on the feature representation of each subtext, the target entity to which each subtext belongs is determined to obtain the entity set of the medical text. For any two subtexts in the entity set, based on the feature representations of the two subtexts, a multi-view contextual attention mechanism is used to determine the target relationship between the target entities to which the two subtexts belong. The target entities to which each subtext belongs in the medical text and the target relationship between the target entities to which any two subtexts belong are determined as the medical knowledge of the medical text.

[0243] And / or,

[0244] Multiple training samples and an initial medical knowledge extraction model are obtained. Each training sample includes training medical text, target entity labels in the training medical text, and target relationship labels between any two training target entities to which the training medical texts belong. The initial medical knowledge extraction model is used to encode the training medical text to obtain the training encoding features of each character in the training medical text. For each training sub-text of arbitrary length in the training medical text, based on the training encoding features of each character included in each training sub-text, an attention mechanism is used to determine the training self-feature representation of each training sub-text. Based on the training self-feature representation of each training sub-text, the training target to which each training sub-text belongs is determined. The system obtains a training entity set for the training medical text; determines a first loss value based on the training target entities in the training entity set and the target entity labels of the training medical text; for any two training sub-texts in the training entity set, determines the training target relationship between the training target entities to which the two training sub-texts belong through a multi-view contextual attention mechanism based on the training self-feature representations of the two training sub-texts; determines a second loss value based on the training target relationship and the target relationship labels; determines a total loss value based on the first loss value and the second loss value, and updates the initial medical knowledge extraction model based on the total loss value to obtain a medical knowledge extraction model.

[0245] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0246] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0247] Since the computer program stored in the storage medium can execute the steps of any of the medical knowledge extraction methods provided in the embodiments of this application, the beneficial effects that any of the medical knowledge extraction methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0248] The above provides a detailed description of a medical knowledge extraction method, apparatus, storage medium, and computer device provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for extracting medical knowledge, characterized in that, include: Acquire medical text and encode the medical text to obtain the encoding features of each character in the medical text; For each subtext of arbitrary length in the medical text, obtain the encoding features of each character included in each subtext; wherein, each subtext is a continuous text of arbitrary length in the medical text; The attention mechanism is used to process the encoded features of each character in each subtext to determine the overall feature representation of each subtext. The feature representation of each subtext is determined based on the encoding features of the starting character, the overall feature representation, and the encoding features of the ending character. Based on the characteristic representation of each sub-text, the target entity to which each sub-text belongs is determined to obtain the entity set of the medical text; For any two sub-texts in the entity set, the target relationship between the target entities to which the two sub-texts belong is determined based on the feature representations of the two sub-texts themselves, through a multi-view contextual attention mechanism; wherein, the multi-view includes at least one of the following: head entity view, tail entity view, and global view; The target entity to which each subtext belongs in the medical text and the target relationship between any two target entities to which any two subtexts belong are determined as the medical knowledge of the medical text.

2. The method according to claim 1, characterized in that, The step of determining the target relationship between the target entities to which the two sub-texts belong, based on their own feature representations and through a multi-view contextual attention mechanism, includes: Based on the feature representations of the two sub-texts themselves, and using a multi-view contextual attention mechanism, the contextual attention feature representation of the intermediate text between the two sub-texts is determined. Based on the self-feature representations of the two sub-texts and the contextual attention feature representation of the intermediate text, the target relationship between the target entities to which the two sub-texts belong is determined.

3. The method according to claim 2, characterized in that, The target entity to which the first sub-text belongs is the head entity, and the target entity to which the second sub-text belongs is the tail entity. The step of determining the contextual attention feature representation of the intermediate text between the two sub-texts based on their own feature representations and a multi-view contextual attention mechanism includes: Based on the attention mechanism, the first attention feature representation of the intermediate text is determined from the perspective of the head entity, according to the feature representation of the first sub-text itself and the encoding features of the intermediate text between the two sub-texts. Based on the attention mechanism, the second attention feature representation of the intermediate text is determined from the perspective of the tail entity, according to the feature representation of the second sub-text and the encoding features of the intermediate text. Based on the attention mechanism, the third attention feature representation of the intermediate text in the global perspective is determined according to the encoding features of each character in the intermediate text; Based on the first attention feature representation, the second attention feature representation, and the third attention feature representation, the contextual attention feature representation of the intermediate text between the two sub-texts is determined.

4. The method according to claim 3, characterized in that, The attention-based mechanism, which determines the first attention feature representation of the intermediate text from the head entity's perspective based on the feature representation of the first sub-text and the encoded features of the intermediate text between the two sub-texts, includes: The first sub-text's own feature representation and the encoded features of each character in the intermediate text are processed using a multilayer perceptron to determine the first attention weight of each character in the intermediate text from the perspective of the head entity. Based on the first attention weight of each character in the intermediate text, the encoded features of each character in the intermediate text are weighted and summed to obtain the first attention feature representation of the intermediate text from the perspective of the head entity.

5. The method according to claim 2, characterized in that, Determining the target relationship between the target entities to which the two sub-texts belong, based on the self-feature representations of the two sub-texts and the contextual attention feature representation of the intermediate text, includes: The self-feature representations of the two sub-texts and the contextual attention feature representation of the intermediate text are fused to obtain the relational fusion feature; The relation fusion features are subjected to a second classification process using a relation classification multilayer perceptron corresponding to different relation categories to determine the target relationship between the target entities to which the two sub-texts belong.

6. The method according to claim 1, characterized in that, The step of determining the target entity to which each sub-text belongs based on its own feature representation includes: Based on the feature representation of each subtext and the encoding features of each character in the corresponding subtext, determine the context feature representation of different entity categories extracted from the corresponding subtext; Using a multilayer perceptron for entity classification, a first classification process is performed on the self-feature representation of the corresponding subtext and the contextual feature representation of different entity categories extracted from the corresponding subtext to determine the target entity category of the corresponding subtext. The entity corresponding to the target entity category is determined as the target entity to which the corresponding subtext belongs, such as obtaining the target entity to which each subtext belongs.

7. The method according to claim 6, characterized in that, The step of determining the contextual feature representations of different entity categories extracted from the corresponding sub-text based on the self-feature representation of each sub-text and the encoded features of each character in the corresponding sub-text includes: By using multilayer perceptrons for entity classification corresponding to different entity categories, the feature representation of each subtext and the encoding features of each character in the corresponding subtext are processed to determine the entity attention weights of the corresponding subtext for different entity categories; Based on the entity attention weights and the encoding features of each character in the corresponding subtext, the contextual feature representations of different entity categories extracted from the corresponding subtext body are determined.

8. The method according to claim 6, characterized in that, The first classification process, which utilizes a multilayer perceptron for entity classification, performs a first classification on the self-feature representation of the corresponding sub-text and the contextual feature representations of different entity categories extracted from the corresponding sub-text to determine the target entity category of the corresponding sub-text, includes: For each entity category, features are constructed for the corresponding subtext under each entity category based on the subtext's own feature representation and the extracted context feature representation; Using the multilayer perceptron for entity classification corresponding to each entity category, the features constructed under each entity category are subjected to first classification processing to determine the entity classification score of the corresponding subtext for different entity categories; The entity category corresponding to the highest entity classification score among the entity classification scores is determined as the target entity category of the corresponding subtext.

9. A method for extracting medical knowledge, characterized in that, include: Multiple training samples and an initial medical knowledge extraction model are obtained. Each training sample includes training medical text, target entity labels in the training medical text, and target relationship labels between any two training target entities. The initial medical knowledge extraction model is used to encode the training medical text to obtain the training encoding features of each character in the training medical text; For each training sub-text of arbitrary length in the training medical text, the training encoding features of each character included in each training sub-text are obtained; wherein, each training sub-text is a continuous text of arbitrary length in the training medical text; the training encoding features of each character included in each training sub-text are processed through an attention mechanism to determine the overall training feature representation of each training sub-text; the training feature representation of each training sub-text is determined based on the training encoding features of the starting character, the overall training feature representation, and the training encoding features of the ending character of each training sub-text; Based on the training feature representation of each training sub-text, determine the training target entity to which each training sub-text belongs, so as to obtain the training entity set of the training medical text; A first loss value is determined based on the training target entity in the training entity set of the training medical text and the target entity label of the training medical text; For any two training sub-texts in the training entity set, based on the training feature representations of the two training sub-texts, the training target relationship between the training target entities to which the two training sub-texts belong is determined through a multi-view contextual attention mechanism; wherein, the multi-view includes at least one of the following: head entity view, tail entity view, and global view. The second loss value is determined based on the training target relationship and the target relationship label; The first loss value and the second loss value are added together or weighted summed to determine the total loss value, and the initial medical knowledge extraction model is updated according to the total loss value to obtain the medical knowledge extraction model.

10. A medical knowledge extraction device, characterized in that, include: The acquisition module is used to acquire medical text. An encoding module is used to encode the medical text to obtain the encoding features of each character in the medical text; The feature determination module is used to obtain the encoding features of each character included in each sub-text of arbitrary length in the medical text; wherein each sub-text is a continuous text of arbitrary length in the medical text; the encoding features of each character included in each sub-text are processed through an attention mechanism to determine the overall feature representation of each sub-text; and the feature representation of each sub-text is determined based on the encoding features of the starting character, the overall feature representation, and the encoding features of the ending character. The entity extraction module is used to determine the target entity to which each sub-text belongs based on its own feature representation, so as to obtain the entity set of the medical text; The relationship extraction module is used to determine the target relationship between the target entities to which the two sub-texts belong, based on the feature representations of the two sub-texts themselves, through a multi-view contextual attention mechanism, for any two sub-texts in the entity set; wherein, the multi-view includes at least one of the following: head entity view, tail entity view, and global view; The knowledge determination module is used to determine the target entity to which each subtext belongs in the medical text and the target relationship between any two target entities to which any two subtexts belong as the medical knowledge of the medical text.

11. A medical knowledge extraction device, characterized in that, include: The training acquisition module is used to acquire multiple training samples and an initial medical knowledge extraction model. Each training sample includes training medical text, target entity labels in the training medical text, and target relationship labels between any two training target entities. The training encoding module is used to encode the training medical text using the initial medical knowledge extraction model to obtain the training encoding features of each character in the training medical text. The training feature determination module is used to obtain the training encoding features of each character in each training sub-text of arbitrary length in the training medical text; wherein each training sub-text is a continuous text of arbitrary length in the training medical text; the training encoding features of each character in each training sub-text are processed through an attention mechanism to determine the overall training feature representation of each training sub-text; and the training feature representation of each training sub-text is determined based on the training encoding features of the starting character, the overall training feature representation, and the training encoding features of the ending character. The training entity extraction module is used to determine the training target entity to which each training sub-text belongs based on the training self-feature representation of each training sub-text, so as to obtain the training entity set of the training medical text. The first loss determination module is used to determine a first loss value based on the training target entity in the training entity set of the training medical text and the target entity label of the training medical text. The training relationship extraction module is used to determine the training target relationship between the training target entities to which the two training sub-texts belong, based on the training feature representations of the two training sub-texts and through a multi-view contextual attention mechanism, for any two training sub-texts in the training entity set; wherein, the multi-view includes at least one of the following: head entity view, tail entity view, and global view. The second loss determination module is used to determine a second loss value based on the training target relationship and the target relationship label; An update module is used to add or weight the first loss value and the second loss value to determine the total loss value, and update the initial medical knowledge extraction model according to the total loss value to obtain the medical knowledge extraction model.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the steps of the medical knowledge extraction method as described in any one of claims 1-9.

13. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the steps of the medical knowledge extraction method as described in any one of claims 1-9 by calling the computer program stored in the memory.

Images