Large language model training method and training device for medical term normalization
The similar representation of medical terms is obtained through the training method of large language model and the term database and model parameters are updated, which solves the accuracy and efficiency of medical terms normalization, reduces costs and improves output accuracy.
Patent Information
- Application Number
- CN202411053125.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing normalization methods for medical terminology have problems such as high cost of formulation and maintenance, poor flexibility in adapting to new terms, and low accuracy in dealing with complex literature, making it difficult to effectively solve the problem of unified terms in the medical field.
Through the training method of large language model, similar term representations of standard medical terms are obtained, and the standard term database and model parameters are updated based on accuracy identification to improve the accuracy and efficiency of normalization of medical terms.
It realizes efficient and accurate normalization of medical terms, reduces costs, improves the output accuracy of large language models, and provides medical workers with correct medical logical reasoning and information.
Smart Images

Figure CN118982017A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing, and more specifically, to a large language model (LLM) training method and a training device for normalizing medical terms. Background Art
[0002] With the deepening of medical research and the advancement of technology, a large number of professional terms have been generated in the medical field (especially in the field of tumors). In different studies, documents, hospitals and regions, the same medical terms or medical concepts may be expressed in multiple ways - for example, lung cancer can be called "lung cancer", "lung malignancy", etc., and "cystic lesions" are often written as "cystic lesions". This brings difficulties to data interaction, information retrieval and knowledge acquisition. At the same time, due to the lack of unified terminology, the efficiency and effectiveness of analysis and modeling based on medical text libraries are seriously affected. Therefore, it is particularly important to effectively normalize the terms in the medical field.
[0003] Currently, term normalization mainly relies on the following methods:
[0004] Rule-based methods: This method usually uses human-defined rules to identify and normalize terms. For example, first, it is defined that lung cancer can be called "lung cancer", "lung malignancy", etc., and then these terms are uniformly converted to "lung cancer" through rules.
[0005] Dictionary or knowledge graph: This approach usually relies on a predefined and maintained dictionary or knowledge graph. During the normalization process, the input terms are matched with the terms in these predefined dictionaries to find the most appropriate standard terms.
[0006] Statistical method: This method uses statistical analysis of a large amount of literature and data to identify the most commonly used terms and use them as standard terms, and then normalize other related terms to them.
[0007] Semi-automatic method: This method combines the above methods while introducing a small amount of human intervention to improve the accuracy of normalization.
[0008] However, although these traditional methods are effective to some extent, they still have limitations, such as high cost of rule-making and maintenance, lack of flexibility to adapt to new terminology, and low accuracy when dealing with complex literature.
[0009] Therefore, a new method for medical terminology normalization is needed to solve the above technical problems. Summary of the invention
[0010] In view of the above problems, the present invention provides a large language model training method and training device for medical term normalization, a method and device for medical term normalization, and corresponding electronic devices and non-transitory computer-readable storage media. The present invention determines similar term representations of standard medical terms, obtains accuracy identification of the determined similar term representations, and then updates the standard term library associated with the large language model based on the standard medical terms and similar term representations whose accuracy meets predetermined conditions, and updates the model parameters of the large language model using the standard medical terms and similar term representations whose accuracy does not meet predetermined conditions as training sets, so that the trained large language model can realize the normalization of medical terms more accurately and efficiently, and reduces costs compared to traditional normalization methods.
[0011] According to one aspect of the present invention, a large language model training method for medical term normalization is provided, comprising: obtaining a first standard medical term; using the large language model to determine one or more similar term representations similar to the first standard medical term for the first standard medical term; obtaining an accuracy identifier for the determined one or more similar term representations; in response to the accuracy of a first similar term representation among the one or more similar term representations satisfying a predetermined condition, updating a standard term library associated with the large language model based on the first standard medical term and the first similar term representation whose accuracy satisfies the predetermined condition, wherein the standard term library indicates a correspondence between a standard medical term and a medical term similar to the standard medical term; and in response to the accuracy of a first similar term representation among the one or more similar term representations not satisfying the predetermined condition, updating model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not satisfy the predetermined condition as a training set.
[0012] According to some embodiments of the present invention, the method further includes: pre-training the large language model based on pre-training text data so that the large language model has the ability to identify medical terms in text data, wherein the types of the pre-training text data include two or more of basic medical knowledge, medical explanations, research papers, medical records, and medical records.
[0013] According to some embodiments of the present invention, pre-training the large language model based on pre-training text data includes: identifying vocabulary in the pre-training text data; determining one or more known medical terms in the vocabulary in the pre-training text data based on the standard terminology library; performing term replacement on the determined one or more known medical terms, wherein the term replacement includes replacing the corresponding known medical terms with standard medical terms in the standard terminology library and / or medical terms similar to the standard medical terms; and pre-training the large language model based on the one or more pre-training text data after term replacement.
[0014] According to some embodiments of the present invention, performing term replacement on the determined multiple known medical terms includes: performing term replacement on the determined multiple known medical terms in different arrangement and combination modes.
[0015] According to some embodiments of the present invention, the method also includes: tokenizing the pre-trained text data through a word segmenter to generate a minimum semantic unit of the pre-trained text data, wherein the minimum semantic unit has complete and independent semantics; and identifying vocabulary in the pre-trained text data based on the minimum semantic unit of the pre-trained text data.
[0016] According to some embodiments of the present invention, acquiring the first standard medical term includes: acquiring medical text data; and identifying the first standard medical term in the medical text data based on the standard term library.
[0017] According to some embodiments of the present invention, pre-training the large language model based on pre-trained text data also includes: performing data cleaning on the pre-trained text data to filter out garbled codes and special characters in the pre-trained text data; and pre-training the large language model based on the filtered pre-trained text data.
[0018] According to some embodiments of the present invention, updating the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet a predetermined condition as a training set includes: updating the model parameters of the large language model based on a first task, wherein the first task indicates identifying the first standard medical term and medical terms in the standard term library that are similar to the first standard medical term as a first clustering category, and identifying the first similar term representation whose accuracy does not meet a predetermined condition as a second clustering category different from the first clustering category.
[0019] According to some embodiments of the present invention, updating the model parameters of the large language model includes updating the model parameters of the large language model based on low-rank adaptive LoRA.
[0020] According to some embodiments of the present invention, updating the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet a predetermined condition as a training set also includes updating the model parameters of the large language model based on a deep learning method.
[0021] According to some embodiments of the present invention, updating the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet a predetermined condition as a training set also includes: obtaining a modified second similar term representation for the first similar term representation whose accuracy does not meet a predetermined condition; and updating the model parameters of the large language model using the first standard medical term, the first similar term representation whose accuracy does not meet a predetermined condition, and the second similar term representation as a training set.
[0022] According to some embodiments of the present invention, updating the model parameters of the large language model using the first standard medical term, the first similar term representation whose accuracy does not meet a predetermined condition, and the second similar term representation as a training set also includes: updating the model parameters of the large language model based on a second task, wherein the second task indicates that the first standard medical term, the second similar term representation, and medical terms in the standard term library that are similar to the first standard medical term are identified as a first clustering category, and the first similar term representation whose accuracy does not meet a predetermined condition is identified as a second clustering category different from the first clustering category.
[0023] According to some embodiments of the present invention, obtaining an accuracy identification of the determined one or more similar term representations includes: obtaining an accuracy identification of the determined one or more similar term representations based on manual identification or obtaining an accuracy identification of the determined one or more similar term representations based on a knowledge graph.
[0024] According to some embodiments of the present invention, the method also includes obtaining a second standard terminology library different from the standard terminology library; using the large language model to determine one or more similar term representations similar to the first standard medical term for the first standard medical term also includes: based on the second standard terminology library, determining one or more additional similar term representations similar to the first standard medical term for the first standard medical term.
[0025] According to some embodiments of the present invention, obtaining an accuracy identification of the determined one or more similar term representations includes obtaining an accuracy identification indicating whether the determined one or more similar term representations are similar to the first standard medical term, and the predetermined condition includes that the determined one or more similar term representations are similar to the first standard medical term; or obtaining an accuracy identification of the determined one or more similar term representations includes obtaining an accuracy identification indicating a similarity value between the determined one or more similar term representations and the first standard medical term, and the predetermined condition includes that the similarity value between the determined one or more similar term representations and the first standard medical term is higher than a predetermined value.
[0026] According to another aspect of the present invention, a method for normalizing medical terms is provided. The method for normalizing medical terms comprises: inputting text data into a medical term normalization large language model obtained based on the large language model training method as described above; identifying medical terms in the text data; replacing the medical terms identified in the text data with corresponding standard medical terms based on a standard term library, wherein the standard term library indicates the correspondence between standard medical terms and similar medical terms; and outputting the text data containing the replaced corresponding standard medical terms.
[0027] According to another aspect of the present invention, a large language model training device for medical term normalization is provided. The large language model training device for medical term normalization includes: a medical term acquisition unit, configured to acquire a first standard medical term; a similar term representation determination unit, configured to use the large language model to determine one or more similar term representations similar to the first standard medical term for the first standard medical term; an accuracy identification unit, which obtains an accuracy identification of the determined one or more similar term representations; a standard term library update unit, configured to update the standard term library associated with the large language model based on the first standard medical term and the first similar term representation whose accuracy meets the predetermined condition in response to the accuracy of the first similar term representation in the one or more similar term representations meeting the predetermined condition, wherein the standard term library indicates the correspondence between the standard medical term and the medical term similar to the standard medical term; and a parameter update unit, configured to update the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet the predetermined condition as a training set in response to the accuracy of the first similar term representation in the one or more similar term representations not meeting the predetermined condition.
[0028] According to some embodiments of the present invention, the training device also includes a pre-training unit, which is configured to pre-train the large language model based on pre-training text data so that the large language model has the ability to identify medical terms in text data, wherein the types of the pre-training text data include two or more of basic medical knowledge, medical explanations, research papers, medical records, and medical records.
[0029] According to some embodiments of the present invention, the pre-training unit further includes: an identification component configured to identify vocabulary in the pre-training text data; a medical term determination component configured to determine one or more known medical terms in the vocabulary in the pre-training text data based on the standard term library; a medical term replacement component configured to perform term replacement on the determined one or more known medical terms, wherein the term replacement includes replacing the corresponding known medical terms with standard medical terms in the standard term library and / or medical terms similar to the standard medical terms; and a pre-training component configured to pre-train the large language model based on the one or more pre-training text data after term replacement.
[0030] According to some embodiments of the present invention, the medical term replacement component is further configured to: perform term replacement on the determined multiple known medical terms in different arrangement and combination modes.
[0031] According to some embodiments of the present invention, the pre-training unit also includes a word segmentation component: the word segmentation component is configured to: tokenize the pre-training text data through a word segmenter to generate a minimum semantic unit of the pre-training text data, wherein the minimum semantic unit has complete and independent semantics; and identify vocabulary in the pre-training text data based on the minimum semantic unit of the pre-training text data.
[0032] According to some embodiments of the present invention, the medical term acquisition unit is further configured to: acquire medical text data; and identify the first standard medical term in the medical text data based on the standard term library.
[0033] According to some embodiments of the present invention, the pre-training unit is further configured to: perform data cleaning on the pre-training text data to filter out garbled codes and special characters in the pre-training text data; and pre-train the large language model based on the filtered pre-training text data.
[0034] According to some embodiments of the present invention, the parameter updating unit is further configured to: update the model parameters of the large language model based on a first task, wherein the first task indicates that the first standard medical term and medical terms in the standard terminology library that are similar to the first standard medical term are identified as a first clustering category, and the first similar term representation whose accuracy does not meet a predetermined condition is identified as a second clustering category different from the first clustering category.
[0035] According to some embodiments of the present invention, the parameter updating unit is further configured to update the model parameters of the large language model based on low-rank adaptive LoRA.
[0036] According to some embodiments of the present invention, the parameter updating unit is further configured to update the model parameters of the large language model based on a deep learning device.
[0037] According to some embodiments of the present invention, the parameter updating unit further includes a term modification component configured to obtain a modified second similar term representation for the first similar term representation whose accuracy does not meet a predetermined condition; the parameter updating unit is further configured to update the model parameters of the large language model using the first standard medical term, the first similar term representation whose accuracy does not meet a predetermined condition, and the second similar term representation as a training set.
[0038] According to some embodiments of the present invention, the parameter updating unit is further configured to: update the model parameters of the large language model based on a second task, wherein the second task indicates that the first standard medical term, the second similar term representation, and the medical term in the standard term library that is similar to the first standard medical term are identified as a first clustering category, and the first similar term representation whose accuracy does not meet a predetermined condition is identified as a second clustering category different from the first clustering category.
[0039] According to some embodiments of the present invention, the accuracy identification unit is further configured to obtain an accuracy identification of the determined one or more similar term representations based on manual identification or to obtain an accuracy identification of the determined one or more similar term representations based on a knowledge graph.
[0040] According to some embodiments of the present invention, the training device also includes a terminology library acquisition unit, which is configured to acquire a second standard terminology library different from the standard terminology library; the similar term representation determination unit is also configured to determine, based on the second standard terminology library, one or more additional similar term representations similar to the first standard medical term using the large language model for the first standard medical term.
[0041] According to some embodiments of the present invention, the accuracy identification unit is also configured to obtain an accuracy identification indicating whether the determined one or more similar term representations are similar to the first standard medical term, and the predetermined condition includes that the determined one or more similar term representations are similar to the first standard medical term; or the accuracy identification unit is also configured to obtain an accuracy identification for the determined one or more similar term representations, including obtaining an accuracy identification indicating a similarity value between the determined one or more similar term representations and the first standard medical term, and the predetermined condition includes that the similarity value between the determined one or more similar term representations and the first standard medical term is higher than a predetermined value.
[0042] According to another aspect of the present invention, a device for normalizing medical terms is provided. The device for normalizing medical terms includes: a text data input unit configured to input text data into a medical term normalization large language model obtained based on the large language model training method as described above; a term identification unit configured to identify medical terms in the text data; a term replacement unit configured to replace the medical terms identified in the text data with corresponding standard medical terms based on a standard term library, wherein the standard term library indicates the correspondence between the standard medical terms and their similar medical terms; and a text data output unit configured to output the text data containing the text data replaced with the corresponding standard medical terms.
[0043] According to another aspect of the present invention, there is provided an electronic device, comprising: a processor; and a memory, wherein the memory stores a computer-readable code, and when the computer-readable code is executed by the processor, the large language model training method for medical terminology normalization or the method for medical terminology normalization is implemented.
[0044] According to another aspect of the present invention, a non-transitory computer-readable storage medium is provided, storing computer-readable instructions, wherein when the computer-readable instructions are executed by a processor, the aforementioned large language model training method for medical terminology normalization or the aforementioned method for medical terminology normalization is implemented.
[0045] Therefore, according to the large language model training method and training device for medical terminology normalization, the method and device for medical terminology normalization, and the corresponding equipment and medium of the embodiments of the present invention, after obtaining the first standard medical term, one or more similar term representations similar to the first standard medical term can be determined for the first standard medical term; then, by obtaining the accuracy identification of the determined one or more similar term representations, based on whether the accuracy of the first similar term representation in the one or more similar term representations meets the predetermined condition, it is determined to update the standard term library associated with the large language model based on the first standard medical term and the first similar term representation whose accuracy meets the predetermined condition, or to update the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet the predetermined condition as the training set, so that the trained large language model can normalize the medical terms more efficiently and accurately, thereby improving the output accuracy of the large language model, and providing medical workers with correct medical logical reasoning and expected medical information. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some exemplary embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative work.
[0047] Figure 1 An architectural diagram of a large language model for medical term normalization according to some embodiments of the present invention is shown;
[0048] Figure 2 A flowchart of a large language model training method for medical term normalization according to some embodiments of the present invention is shown;
[0049] Figure 3 A flowchart of a method for normalizing medical terms according to some embodiments of the present invention is shown;
[0050] Figure 4 A block diagram of a large language model training apparatus for medical term normalization according to some embodiments of the present invention is shown;
[0051] Figure 5 A block diagram of an apparatus for normalizing medical terms according to some embodiments of the present invention is shown;
[0052] Figure 6 A structural diagram of an electronic device according to some embodiments of the present invention is shown. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. "First", "second" and similar words used in the present invention do not represent any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present invention clear and concise, the present invention omits the detailed description of some known functions and known components.
[0055] The present invention uses a flow chart to illustrate the steps of the method according to an embodiment of the present invention. It should be understood that the preceding or following steps are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more steps may be removed from these processes.
[0056] In the specification and drawings of the present invention, elements are described in singular or plural form, depending on the embodiment. However, the singular and plural forms are appropriately selected for the proposed situation only for the convenience of explanation and are not intended to limit the present invention thereto. Therefore, the singular form may include the plural form, and the plural form may also include the singular form, unless the context clearly indicates otherwise.
[0057] The following will describe in detail the large language model training method and training device for medical term normalization, the method and device for medical term normalization, and the corresponding equipment and medium provided by the present invention with reference to the accompanying drawings.
[0058] First embodiment
[0059] Large language models (LLMs) are artificial intelligence models designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, reasoning, and more. Large language models currently have emerging capabilities that other machine learning models do not have - including, for example, context, instruction following, and step-by-step reasoning capabilities, making them ideal tools for medical terminology normalization.
[0060] However, there is currently no large language model for normalizing medical terminology. Figure 1 and Figure 2 The following is an architecture diagram and a flow chart of a large language model training method for medical term normalization according to some embodiments of the present invention. Figure 1 and Figure 2 The large language model training method for medical terminology normalization of the present invention is described in detail.
[0061] First, in step S202, a first standard medical term may be acquired.
[0062] According to an example of the present invention, the first standard medical term may be one or more standard medical terms. Standard medical terms refer to medical terms that are well known and / or commonly used by those skilled in the medical field.
[0063] In one example, the first standard term may be a directly acquired standard term, such as a single standard term input by an external interface. In another example, the first standard term may be one or more standard terms identified from text data.
[0064] According to one embodiment of the present invention, a large language model can be pre-trained based on pre-trained text data so that the large language model has the ability to identify medical terms in text data. The types of pre-trained text data may include two or more of basic medical knowledge, medical explanations, research papers, medical records, and medical records. Training based on multiple types of pre-trained texts can enable the large language model to fully include synonyms that may appear in different contexts, ensuring that the model has a deep understanding of the terms.
[0065] According to one embodiment of the present invention, pre-training a large language model based on pre-training text data may include identifying vocabulary in the pre-training text data; determining one or more known medical terms in the vocabulary in the pre-training text data based on a standard terminology library; performing term replacement on the determined one or more known medical terms, wherein the term replacement includes replacing the corresponding known medical terms with standard medical terms in the standard terminology library and / or medical terms similar to the standard medical terms; and pre-training a large language model based on the one or more pre-training text data after term replacement. The standard terminology library includes standard medical terms and indicates the correspondence between standard medical terms and medical terms similar to the standard medical terms. By replacing the training terms, the pre-trained large language model's understanding of synonymous terms can be enhanced, thereby more accurately determining similar term representations of standard medical terms.
[0066] According to one embodiment of the present invention, term replacement may be performed on the determined multiple known medical terms in different arrangements and combinations to further enhance the pre-trained large language model's understanding of synonymous terms.
[0067] According to one embodiment of the present invention, the pre-trained text data can also be tokenized by a tokenizer to generate the smallest semantic unit of the pre-trained text data, wherein the smallest semantic unit has complete and independent semantics; then the vocabulary in the pre-trained text data can be identified based on the smallest semantic unit of the pre-trained text data. By tokenizing the training text data, it can be ensured that each word has relatively complete and independent semantics, thereby improving the recognition ability of the pre-trained large language model for text.
[0068] According to one embodiment of the present invention, in order to obtain the first standard medical term, medical text data may be obtained, and then the first standard medical term in the medical text data may be identified based on a standard term library.
[0069] In one example, the first standard medical term may be a standard medical term from a standard medical library.
[0070] According to one embodiment of the present invention, pre-training a large language model based on pre-trained text data may also include: performing data cleaning on the pre-trained text data; pre-training the large language model based on filtered pre-trained text data to filter out garbled codes and special characters in the pre-trained text data to avoid interference of garbled codes and special characters in pre-training, thereby improving the pre-trained large language model's ability to recognize text.
[0071] After the first standard medical term is obtained, in step S204 , a large language model may be used to determine one or more similar term representations similar to the first standard medical term for the first standard medical term.
[0072] In one example, when determining one or more similar term representations similar to the first standard medical term using a large language model, it can be further determined whether the standard term library associated with the large language model has described the relationship between the first standard medical term and a specific similar term in the one or more similar term representations. In response to the standard term library describing the relationship between the first standard medical term and a specific similar term in the one or more similar term representations, the specific similar term can be removed from the one or more similar terms, and the accuracy identification of the updated one or more similar term representations can be obtained to avoid the similar term representations already recorded in the standard term library from affecting the training effect.
[0073] According to one embodiment of the present invention, the large language model can also be used to determine one or more additional similar term expressions similar to the first standard medical term based on the second standard term library. For example, the substandard term library is a plug-in knowledge base attached to the large language model in a plug-in form, so that the large language model can more flexibly process new terms that have not appeared in the training data.
[0074] The large language model in training can be made to determine one or more similar term representations similar to the standard medical term based on the acquired standard medical term. However, one of the one or more similar term representations may be a true similar term representation of the standard medical term, or may be an erroneous similar term representation of the standard medical term. In order to enable the large language model to accurately and quickly determine the true similar term representation of the standard medical term and avoid determining the erroneous similar term representation of the standard medical term, an accuracy identification of the determined one or more similar term representations can be obtained in step S206, and in steps S208-S210, the large language model will be further trained in response to whether the accuracy of the first similar term representation in the one or more similar term representations meets a predetermined condition, so as to more quickly and accurately determine the similar term representation of the standard medical term.
[0075] Specifically, in step S206, an accuracy mark of the determined one or more similar term representations is obtained. In one example, the accuracy mark of the determined one or more similar term representations may come from feedback from an external source. According to one embodiment of the present invention, the accuracy mark of the determined one or more similar term representations may be obtained based on manual identification or based on a knowledge graph.
[0076] In one example, the accuracy mark can be a binary judgment indicating whether the similarity term indicates that the medical term is similar to the standard medical term. In one example, the accuracy mark can also be a confidence score indicating whether the similarity term indicates that the medical term is similar to the standard medical term.
[0077] According to one embodiment of the present disclosure, obtaining an accuracy identification of the determined one or more similar term representations may include obtaining an accuracy identification indicating whether the determined one or more similar term representations are similar to a first standard medical term, and the predetermined condition may include that the determined one or more similar term representations are similar to the first standard medical term. According to another embodiment of the present disclosure, obtaining an accuracy identification of the determined one or more similar term representations may include obtaining an accuracy identification indicating a similarity value between the determined one or more similar term representations and the first standard medical term, and the predetermined condition may include that the similarity value between the determined one or more similar term representations and the first standard medical term is higher than a predetermined value.
[0078] In step S208, in response to the accuracy of the first similar term representation among the one or more similar term representations satisfying a predetermined condition, the standard term library associated with the large language model may be updated based on the first standard medical term and the first similar term representation whose accuracy satisfies the predetermined condition. The standard term library of the large language model may be improved by adding similar term representations similar to the standard medical term to the standard term library, so that similar term representations of standard medical terms may be determined more quickly and accurately during subsequent training and use, and / or standard medical terms may be determined through similar term representations.
[0079] In one example, the standard term base associated with the large language model may be a standard term base included in the large language model.
[0080] In one example, different predetermined conditions may be set according to different accuracy identifiers. For example, in the case where the accuracy identifier is a binary judgment of whether the similarity term representation is similar to the standard medical term, the predetermined condition may be whether the similarity term representation is similar to the standard medical term. In the case where the accuracy identifier is a confidence score of whether the similarity term representation is similar to the standard medical term, the predetermined condition may be a specific confidence score of whether the similarity term representation is similar to the standard medical term. For example, a confidence score greater than or greater than or equal to the predetermined condition may be considered to satisfy the condition.
[0081] In step S210, in response to the accuracy of the first similar term representation among the one or more similar term representations not satisfying a predetermined condition, the model parameters of the large language model are updated using the first standard medical term and the first similar term representation whose accuracy does not satisfy the predetermined condition as a training set. After the dissimilar terms are identified, they can be used as training sets to train the large language model and update the parameters of the large language model, so that similar term representations of standard medical terms can be determined more quickly and accurately in subsequent training and use, or standard medical terms can be determined through similar term representations, avoiding determining similar term representations for standard medical terms whose accuracy does not satisfy a predetermined condition (the predetermined condition may be the same or different from that in the training process), and / or avoiding determining incorrect standard medical terms through similar term representations.
[0082] According to one embodiment of the present invention, updating the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet the predetermined condition as a training set may include updating the model parameters of the large language model based on a first task, wherein the first task indicates that the first standard medical term and the medical terms similar to the first standard medical term in the standard term library are identified as a first cluster category, and the first similar term representation whose accuracy does not meet the predetermined condition is identified as a second cluster category different from the first cluster category. By classifying the correct term identification and the identification term whose accuracy does not meet the predetermined condition into different cluster categories, similar terms and similar terms generated by erroneous output can be separated to the greatest extent, thereby further updating the parameters of the large language model so that the large language model can more accurately identify the similar term representations of the annotated medical terms.
[0083] According to one embodiment of the present invention, updating the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet the predetermined condition as a training set may also include: obtaining a modified second similar term representation for the first similar term representation whose accuracy does not meet the predetermined condition; updating the model parameters of the large language model using the first standard medical term, the first similar term representation whose accuracy does not meet the predetermined condition, and the second similar term representation as a training set.
[0084] According to one embodiment of the present invention, for a first standard medical term, a first similar term representation whose accuracy does not meet a predetermined condition, and a second similar term representation, the model parameters of the large language model can be updated based on a second task, wherein the second task indicates that the first standard medical term, the second similar term representation, and medical terms in the standard term library that are similar to the first standard medical term are identified as a first clustering category, and the first similar term representation whose accuracy does not meet a predetermined condition is identified as a second clustering category that is different from the first clustering category. By classifying the corrected and correct term identifications and the identified terms whose accuracy does not meet a predetermined condition into different clustering categories, similar terms and similar terms generated by erroneous outputs can be separated to the greatest extent, thereby further updating the parameters of the large language model so that the large language model can more accurately identify similar term representations of annotated medical terms.
[0085] In one example, the modified term representations may be obtained from an external source, and model parameters of the large language model may be further trained based on the modified term representations.
[0086] According to one embodiment of the present invention, updating the model parameters of the large language model can be based on low-rank adaptive LoRA to update the model parameters of the large language model. Updating the model parameters of the large language model based on LoRa can increase the large model's understanding of synonyms / synonyms while reducing training costs.
[0087] In one example, the large language model training method for medical term normalization of the present invention can be used in the fine-tuning stage of the large language model. When preparing fine-tuning training data, the standard term library is updated.
[0088] According to one embodiment of the present invention, the model parameters of the large language model can be updated based on a deep learning method by using the first standard medical term and the first similar term representation whose accuracy does not meet the predetermined condition as a training set. Based on deep learning, the large model can automatically identify and normalize the terms while maintaining a high accuracy rate.
[0089] It should be noted that although the above steps are described in a sequential form, the order in which they are executed during the training process is not necessarily to be carried out in exact order. Instead, these steps can be processed in reverse order or in parallel.
[0090] Figure 3 A method for normalizing medical terms according to the present invention is shown. Figure 3 As shown, in step S302, the text data can be input based on the above-mentioned Figure 1 and Figure 2The large language model training method for obtaining the large language model for normalization of medical terms shown in the figure obtains the large language model for normalization of medical terms. In step S304, the medical terms in the text data can be identified. In step S306, the medical terms identified in the text data are replaced with corresponding standard medical terms based on the standard terminology library, wherein the standard terminology library indicates the corresponding relationship between the standard medical terms and their similar medical terms. Finally, in step S308, the text data containing the replaced corresponding standard medical terms is output.
[0091] In one example, the output may be normalized term data output, for example, in JSON format.
[0092] The steps of the method for normalizing medical terms will be described below by way of example.
[0093] An example of text data to be input into a medical term normalization large language model obtained by a large language model training method for obtaining a medical term normalization large language model is as follows:
[0094] "The patient underwent an MRI scan, which showed a cystic lesion with a diameter of about 2.5 cm in the pancreas, suspected to be a mucinous cystadenoma. In addition, the chest CT results showed that there were multiple small nodules with a diameter of less than 0.8 cm in the upper lobe of the lungs, suspected to be metastatic lesions. Please identify all tumor professional terms, normalize them, and restate them using normalized terms, and output them in JSON format."
[0095] According to the input text data, the following input can be obtained:
[0096]
[0097] "Code": "R93.5"
[0098] },
[0099] {
[0100] "Original text": "Mucinous cystadenoma",
[0101] "Normalized name": "mucinous cystadenoma",
[0102] "Code": "D13.6"
[0103] },
[0104] {
[0105] "Original text": "Upper lobe of lung",
[0106] "Normalized name": "Upper lobe of lung",
[0107] "Code": "J98.4"
[0108] },
[0109] {
[0110] "Original text": "small nodules",
[0111] "Normalized name": "pulmonary nodules",
[0112] "Code": "R91.1"
[0113] },
[0114] {
[0115] "Original text": "metastatic lesions",
[0116] "Normalized name": "Tumor metastasis",
[0117] "Code": "C78.01"
[0118] }
[0119] ]}
[0120] Above, combined Figure 1-Figure 3 A large language model training method for medical term normalization and a method for medical term normalization are described. The training method can determine one or more similar term representations similar to the first standard medical term for the first standard medical term using a large language model after obtaining a first standard medical term; then, by obtaining an accuracy identifier for the determined one or more similar term representations, based on whether the accuracy of the first similar term representation in the one or more similar term representations meets a predetermined condition, determine to update the standard term library associated with the large language model based on the first standard medical term and the first similar term representation whose accuracy meets the predetermined condition, or update the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet the predetermined condition as a training set, so that the trained large language model can normalize the medical terms more efficiently and accurately, thereby improving the output accuracy of the large language model, and providing medical workers with correct medical logical reasoning and expected medical information. At the same time, using a large language model to normalize terms reduces costs compared to traditional normalization methods.
[0121] Second embodiment
[0122] In addition to providing the above-mentioned large language model training method for medical terminology normalization and the method for medical terminology normalization, the present invention also provides a large language model training device for medical terminology normalization and a device for medical terminology normalization. Figure 4 and Figure 5 This is described in detail.
[0123] Figure 4 FIG. 4 is a block diagram of a large language model training apparatus for medical term normalization according to some embodiments of the present invention. Figure 4 As shown, the large language model training device 400 for medical term normalization described in the present invention may include a medical term acquisition unit 410, a similar term representation determination unit 420, an accuracy identification unit 430, a standard term library updating unit 440 and a parameter updating unit 450.
[0124] According to some embodiments of the present invention, the medical term acquisition unit 410 may be configured to acquire a first standard medical term.
[0125] The similar term representation determining unit 420 may be configured to determine, for the first standard medical term, one or more similar term representations that are similar to the first standard medical term using a large language model.
[0126] The accuracy identification unit 430 may be configured to obtain an accuracy identification of the determined one or more similar term expressions.
[0127] The standard term library updating unit 440 can be configured to update the standard term library associated with the large language model based on the first standard medical term and the first similar term representation whose accuracy meets the predetermined condition in response to the accuracy of the first similar term representation among the one or more similar term representations meeting the predetermined condition, wherein the standard term library indicates the correspondence between the standard medical term and the medical terms similar to the standard medical term.
[0128] The parameter updating unit 450 may be configured to update the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet the predetermined condition as a training set in response to the accuracy of the first similar term representation among the one or more similar term representations not meeting the predetermined condition.
[0129] According to some embodiments of the present invention, the large language model training device 400 for medical terminology normalization may further include a pre-training unit, which is configured to pre-train the large language model based on pre-training text data so that the large language model has the ability to identify medical terms in text data, wherein the types of the pre-training text data include two or more of basic medical knowledge, medical explanations, research papers, medical records, and medical records.
[0130] According to some embodiments of the present invention, the pre-training unit may further include: an identification component configured to identify vocabulary in the pre-training text data; a medical term determination component configured to determine one or more known medical terms in the vocabulary in the pre-training text data based on a standard term library; a medical term replacement component configured to perform term replacement on the determined one or more known medical terms, wherein the term replacement includes replacing the corresponding known medical terms with standard medical terms in the standard term library and / or medical terms similar to the standard medical terms; and a pre-training component configured to pre-train a large language model based on the one or more pre-training text data after term replacement.
[0131] According to some embodiments of the present invention, the medical term replacement component may also be configured to perform term replacement on the determined multiple known medical terms in different arrangements and combinations.
[0132] According to some embodiments of the present invention, the pre-training unit may further include a word segmentation component: the word segmentation component is configured to: tokenize the pre-training text data through a word segmenter to generate a minimum semantic unit of the pre-training text data, wherein the minimum semantic unit has complete and independent semantics; and identify vocabulary in the pre-training text data based on the minimum semantic unit of the pre-training text data.
[0133] According to some embodiments of the present invention, the medical term acquisition unit 410 may also be configured to: acquire medical text data; and identify a first standard medical term in the medical text data based on a standard term library.
[0134] According to some embodiments of the present invention, the pre-training unit is further configured to: perform data cleaning on the pre-training text data to filter out garbled codes and special characters in the pre-training text data; and pre-train a large language model based on the filtered pre-training text data.
[0135] According to some embodiments of the present invention, the parameter updating unit 450 can also be configured to: update the model parameters of the large language model based on a first task, wherein the first task indicates that a first standard medical term and a medical term in a standard term library that is similar to the first standard medical term are identified as a first clustering category, and a first similar term representation whose accuracy does not meet a predetermined condition is identified as a second clustering category different from the first clustering category.
[0136] According to some embodiments of the present invention, the parameter updating unit 450 may also be configured to update the model parameters of the large language model based on the low-rank adaptive LoRA.
[0137] According to some embodiments of the present invention, the parameter updating unit 450 may also be configured to update the model parameters of the large language model based on the deep learning device.
[0138] According to some embodiments of the present invention, the parameter updating unit 450 may further include a term modification component configured to obtain a modified second similar term representation for a first similar term representation whose accuracy does not meet a predetermined condition; the parameter updating unit is further configured to update the model parameters of the large language model using the first standard medical term, the first similar term representation whose accuracy does not meet a predetermined condition, and the second similar term representation as a training set.
[0139] According to some embodiments of the present invention, the parameter updating unit 450 can also be configured to: update the model parameters of the large language model based on a second task, wherein the second task indicates that the first standard medical term, the second similar term representation, and the medical term in the standard term library that is similar to the first standard medical term are identified as a first clustering category, and the first similar term representation whose accuracy does not meet a predetermined condition is identified as a second clustering category different from the first clustering category.
[0140] According to some embodiments of the present invention, the accuracy identification unit 430 may also be configured to obtain an accuracy identification of the determined one or more similar term representations based on manual identification or to obtain an accuracy identification of the determined one or more similar term representations based on a knowledge graph.
[0141] According to some embodiments of the present invention, the large language model training device 400 for medical terminology normalization may further include a term base acquisition unit, which is configured to acquire a second standard term base that is different from the standard term base; the similar term representation determination unit is further configured to determine, based on the second standard term base, one or more additional similar term representations similar to the first standard medical term using the large language model for the first standard medical term.
[0142] According to some embodiments of the present invention, the accuracy identification unit 430 may also be configured to obtain an accuracy identification indicating whether the determined one or more similar term representations are similar to the first standard medical term, and the predetermined condition includes that the determined one or more similar term representations are similar to the first standard medical term.
[0143] According to some embodiments of the present invention, the accuracy identification unit 430 may also be configured to obtain an accuracy identification of the determined one or more similar term representations, including obtaining an accuracy identification indicating a similarity value between the determined one or more similar term representations and the first standard medical term, and the predetermined condition includes that the similarity value between the determined one or more similar term representations and the first standard medical term is higher than a predetermined value.
[0144] Figure 5 FIG. 4 is a block diagram of an apparatus for normalizing medical terms according to some embodiments of the present invention. Figure 5As shown, the apparatus 500 for normalizing medical terms of the present invention may include a text data input unit 510 , a term identification unit 520 , a term replacement unit 530 , and a text data output unit 540 .
[0145] According to some embodiments of the present invention, the data input unit 510 may be configured to input text data into a normalized large language model of medical terms obtained based on the large language model training method as described above.
[0146] The term identification unit 520 may be configured to identify medical terms in the text data.
[0147] The term replacement unit 530 may be configured to replace the medical terms identified in the text data with corresponding standard medical terms based on a standard term library, wherein the standard term library indicates a correspondence relationship between the standard medical terms and similar medical terms thereof.
[0148] The text data output unit 540 may be configured to output the text data including the medical terms replaced with the corresponding standard medical terms.
[0149] about Figures 4 to 5 Some specific details of the medical large language model training and logical reasoning device shown can also be referred to Figures 1 to 3 The content of the medical large language model training and logical reasoning methods shown in.
[0150] Figure 6 A structural diagram of an electronic device according to some embodiments of the present invention is shown.
[0151] See also Figure 6 , the electronic device 600 may include a processor 601 and a memory 602. The processor 601 and the memory 602 may be connected via a bus 603. The electronic device 600 may be any type of portable device (such as a smart camera, a smart phone, a tablet computer, etc.) or any type of fixed device (such as a desktop computer, a server, etc.).
[0152] The processor 601 can perform various actions and processes according to the program stored in the memory 602. Specifically, the processor 601 can be an integrated circuit chip with signal processing capabilities. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can be an X86 architecture or an ARM architecture.
[0153] The memory 602 stores computer executable instructions, and when the computer executable instructions are executed by the processor 601, the above-mentioned large language model training method for medical term normalization and the method for medical term normalization are implemented. The memory 602 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0154] In addition, the large language model training method for medical term normalization and the method for medical term normalization according to the present invention can be recorded in a computer-readable recording medium. Specifically, according to the present invention, a computer-readable recording medium storing computer-executable instructions can be provided, and when the computer-executable instructions are executed by a processor, the processor can be prompted to execute the large language model training method for medical term normalization and the method for medical term normalization as described above.
[0155] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code contains at least one executable instruction for realizing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0156] In general, various example embodiments of the present invention may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of embodiments of the present invention are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.
[0157] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.
[0158] The above is an explanation of the present invention and should not be considered as a limitation thereof. Although several exemplary embodiments of the present invention have been described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined in the claims. It should be understood that the above is an explanation of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.
Claims
1. A large language model training method for medical term normalization, comprising: Acquire first standard medical terminology; determining, for the first standard medical term, one or more similar term representations similar to the first standard medical term using the large language model; obtaining an accuracy indication of the identified one or more similar term representations; In response to the accuracy of a first similar term representation among the one or more similar term representations satisfying a predetermined condition, updating a standard term library associated with the large language model based on the first standard medical term and the first similar term representation whose accuracy satisfies the predetermined condition, wherein the standard term library indicates a correspondence between a standard medical term and a medical term similar to the standard medical term; as well as In response to the accuracy of a first similar term representation among the one or more similar term representations not satisfying the predetermined condition, the model parameters of the large language model are updated using the first standard medical term and the first similar term representation whose accuracy does not satisfy the predetermined condition as a training set.
2. The method according to claim 1, further comprising: Pre-training the large language model based on pre-training text data so that the large language model has the ability to identify medical terms in the text data, The types of the pre-training text data include two or more of basic medical knowledge, medical explanations, research papers, medical records, and medical records.
3. The method according to claim 2, wherein: Pre-training the large language model based on the pre-training text data includes: Identifying vocabulary in the pre-trained text data; Based on the standard terminology library, determining one or more known medical terms in the vocabulary in the pre-trained text data; Performing term replacement on the determined one or more known medical terms, wherein the term replacement includes replacing the corresponding known medical term with a standard medical term in the standard term library and / or a medical term similar to the standard medical term; and The large language model is pre-trained based on one or more pre-training text data replaced with terms.
4. The method according to claim 3, wherein: Term replacements for multiple known medical terms identified include: The determined multiple known medical terms are replaced with different permutations and combinations.
5. The method according to claim 3, further comprising: Tokenizing the pre-trained text data by a word segmenter to generate a minimum semantic unit of the pre-trained text data, wherein the minimum semantic unit has complete and independent semantics; The vocabulary in the pre-training text data is identified based on the smallest semantic unit of the pre-training text data.
6. The method according to claim 2, wherein: Access to first standard medical terminology including: Obtain medical text data; The first standard medical term in the medical text data is identified based on the standard term library.
7. The method according to claim 2, wherein: Pre-training the large language model based on the pre-training text data further includes: Performing data cleaning on the pre-trained text data to filter out garbled characters and special characters in the pre-trained text data; The large language model is pre-trained based on the filtered pre-training text data.
8. The method according to claim 1, wherein: Updating the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet a predetermined condition as a training set includes: The model parameters of the large language model are updated based on a first task, wherein the first task indicates that the first standard medical term and medical terms in the standard term library that are similar to the first standard medical term are identified as a first clustering category, and the first similar term representations whose accuracy does not meet a predetermined condition are identified as a second clustering category different from the first clustering category.
9. The method according to claim 8, wherein: Updating the model parameters of the large language model includes updating the model parameters of the large language model based on low-rank adaptive LoRA.
10. The method according to claim 1, wherein: Updating the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet a predetermined condition as a training set also includes updating the model parameters of the large language model based on a deep learning method.
11. The method according to claim 1, wherein: Updating the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet a predetermined condition as a training set also includes: For the first similar term representation whose accuracy does not meet a predetermined condition, obtaining a modified second similar term representation; The model parameters of the large language model are updated using the first standard medical term, the first similar term representation whose accuracy does not meet a predetermined condition, and the second similar term representation as a training set.
12. The method according to claim 11, wherein updating the model parameters of the large language model using the first standard medical term, the first similar term representation whose accuracy does not meet a predetermined condition, and the second similar term representation as a training set further comprises: The model parameters of the enlarged language model are updated based on a second task, wherein the second task indicates that the first standard medical term, the second similar term representation, and medical terms in the standard term library that are similar to the first standard medical term are identified as a first clustering category, and the first similar term representation whose accuracy does not meet a predetermined condition is identified as a second clustering category different from the first clustering category.
13. The method according to claim 1, obtaining an accuracy indicator of the determined one or more similar term expressions comprises: The accuracy identification of the one or more similar term representations determined is obtained based on manual identification or the accuracy identification of the one or more similar term representations determined is obtained based on a knowledge graph.
14. The method according to claim 1, further comprising obtaining a second standard terminology database different from the standard terminology database; Determining one or more similar term representations similar to the first standard medical term using the large language model for the first standard medical term further includes: Based on the second standard term base, the large language model is used to determine one or more additional similar term representations similar to the first standard medical term for the first standard medical term.
15. The method according to claim 1, wherein: Obtaining an accuracy indication for the determined one or more similar term expressions includes obtaining an accuracy indication indicating whether the determined one or more similar term expressions are similar to the first standard medical term, and the predetermined condition includes that the determined one or more similar term expressions are similar to the first standard medical term; or Obtaining an accuracy identification of the determined one or more similar term representations includes obtaining an accuracy identification indicating a similarity value between the determined one or more similar term representations and the first standard medical term, and the predetermined condition includes that the similarity value between the determined one or more similar term representations and the first standard medical term is higher than a predetermined value.
16. A method for normalizing medical terminology, comprising: Inputting text data into a normalized large language model of medical terms obtained based on the large language model training method according to claim 1; identifying medical terms in the text data; replacing the medical terms identified in the text data with corresponding standard medical terms based on a standard term library, wherein the standard term library indicates a correspondence between a standard medical term and a similar medical term; The output includes the text data replaced with corresponding standard medical terms.
17. A large language model training device for medical term normalization, comprising: a medical term acquisition unit configured to acquire a first standard medical term; a similar term representation determining unit configured to determine, for the first standard medical term, one or more similar term representations similar to the first standard medical term using the large language model; An accuracy identification unit, which obtains an accuracy identification of the determined one or more similar terms; a standard term base updating unit configured to update a standard term base associated with the large language model based on the first standard medical term and the first similar term base whose accuracy satisfies the predetermined condition in response to the accuracy of a first similar term base among the one or more similar term bases satisfying a predetermined condition, wherein the standard term base indicates a correspondence between a standard medical term and a medical term similar to the standard medical term; as well as A parameter updating unit is configured to update the model parameters of the large language model using the first standard medical term and the first similar term representation whose accuracy does not meet the predetermined condition as a training set in response to the accuracy of the first similar term representation among the one or more similar term representations not meeting the predetermined condition.
18. The apparatus according to claim 17, further comprising a pre-training unit, The pre-training unit is configured to pre-train the large language model based on pre-training text data so that the large language model has the ability to identify medical terms in the text data. The types of the pre-training text data include two or more of basic medical knowledge, medical explanations, research papers, medical records, and medical records.
19. The device according to claim 18, wherein: The pre-training unit also includes: an identification component configured to identify vocabulary in the pre-trained text data; a medical term determination component configured to determine one or more known medical terms in the vocabulary in the pre-trained text data based on the standard term base; a medical term replacement component configured to perform term replacement on the determined one or more known medical terms, wherein the term replacement includes replacing the corresponding known medical term with a standard medical term in the standard term library and / or a medical term similar to the standard medical term; and The pre-training component is configured to pre-train the large language model based on one or more pre-training text data replaced by terms.
20. The device according to claim 19, wherein The medical term replacement component is further configured to: The determined multiple known medical terms are replaced with different permutations and combinations.
21. The device according to claim 19, wherein The pre-training unit also includes a word segmentation component: The word segmentation component is configured as follows: Tokenizing the pre-trained text data by a word segmenter to generate a minimum semantic unit of the pre-trained text data, wherein the minimum semantic unit has complete and independent semantics; The vocabulary in the pre-training text data is identified based on the smallest semantic unit of the pre-training text data.
22. The device according to claim 18, wherein The medical term acquisition unit is further configured to: Obtain medical text data; The first standard medical term in the medical text data is identified based on the standard term library.
23. The device according to claim 18, wherein The pre-training unit is further configured to: Performing data cleaning on the pre-trained text data to filter out garbled characters and special characters in the pre-trained text data; The large language model is pre-trained based on the filtered pre-training text data.
24. The device according to claim 17, wherein: The parameter updating unit is further configured to: The model parameters of the large language model are updated based on a first task, wherein the first task indicates that the first standard medical term and medical terms in the standard term library that are similar to the first standard medical term are identified as a first clustering category, and the first similar term representations whose accuracy does not meet a predetermined condition are identified as a second clustering category different from the first clustering category.
25. The device according to claim 24, wherein: The parameter updating unit is also configured to update the model parameters of the large language model based on low rank adaptive LoRA.
26. The device according to claim 17, wherein: The parameter updating unit is also configured to update the model parameters of the large language model based on a deep learning device.
27. The device according to claim 17, wherein: The parameter updating unit further comprises a term modification component configured to obtain a modified second similar term representation for the first similar term representation whose accuracy does not meet a predetermined condition; The parameter updating unit is further configured to update the model parameters of the large language model using the first standard medical term, the first similar term representation whose accuracy does not meet a predetermined condition, and the second similar term representation as a training set.
28. The apparatus according to claim 27, wherein the parameter updating unit is further configured to: The model parameters of the enlarged language model are updated based on a second task, wherein the second task indicates that the first standard medical term, the second similar term representation, and medical terms in the standard term library that are similar to the first standard medical term are identified as a first clustering category, and the first similar term representation whose accuracy does not meet a predetermined condition is identified as a second clustering category different from the first clustering category.
29. According to the device according to claim 17, the accuracy identification unit is also configured to obtain the accuracy identification of the determined one or more similar term representations based on manual identification or to obtain the accuracy identification of the determined one or more similar term representations based on a knowledge graph.
30. The apparatus according to claim 17, further comprising a term base acquisition unit, wherein the term base acquisition unit is configured to acquire a second standard term base different from the standard term base; The similar term representation determining unit is further configured to determine, based on the second standard term base, one or more additional similar term representations similar to the first standard medical term using the large language model for the first standard medical term.
31. The apparatus of claim 17, wherein: The accuracy identification unit is further configured to obtain an accuracy identification indicating whether the determined one or more similar term representations are similar to the first standard medical term, and the predetermined condition includes that the determined one or more similar term representations are similar to the first standard medical term; or The accuracy identification unit is also configured to obtain an accuracy identification of the determined one or more similar term representations, including obtaining an accuracy identification indicating a similarity value between the determined one or more similar term representations and the first standard medical term, and the predetermined condition includes that the similarity value between the determined one or more similar term representations and the first standard medical term is higher than a predetermined value.
32. A device for normalizing medical terms, comprising: A text data input unit configured to input text data into a normalized large language model of medical terms obtained based on the large language model training method according to claim 1; a term identification unit configured to identify medical terms in the text data; a term replacement unit configured to replace the medical terms identified in the text data with corresponding standard medical terms based on a standard term library, wherein the standard term library indicates a corresponding relationship between a standard medical term and a similar medical term; The text data output unit is configured to output the text data including the text data replaced with the corresponding standard medical terms.
33. An electronic device comprising: processor; as well as A memory, wherein the memory stores computer-readable codes, which, when executed by the processor, implement the large language model training method for medical terminology normalization described in any one of claims 1 to 15 or the method for medical terminology normalization described in claim 16.
34. A non-transitory computer-readable storage medium storing computer-readable instructions, wherein: When the computer-readable instructions are executed by a processor, the large language model training method for medical terminology normalization described in any one of claims 1 to 15 or the method for medical terminology normalization described in claim 16 is implemented.
Citation Information
Patent Citations
Code mapping method and related device thereof, electronic equipment and storage medium
CN114048313A
Method and device for training generative large language model based on knowledge base feedback
CN117009490A
Medical term standardization auxiliary diagnosis method based on large language model
CN118296121A
Cited By
Matching method and device for medical terms
CN121434403A
