Method and system for icd automatic coding based on large language model transfer learning

Abstract

The application provides an ICD automatic coding method and system based on large language model transfer learning, comprising: inputting electronic medical record data of a low-resource language, using a large language model to identify the type of the text to be coded; based on the content to be coded, constructing a multilingual ICD automatic coding instruction; using a language transfer ICD automatic coding large model to perform ICD automatic coding on the content to be coded, and outputting an ICD coding result; based on the coding rules of the low-resource language, verifying and aligning the output ICD coding result, and outputting a final coding result. The application utilizes the multilingual capability of the large language model, effectively reduces the artificial labeling workload and reduces the landing cost of the large language model in the ICD coding task for the ICD automatic coding task which is difficult to obtain artificial labeling data in a low-resource language environment.

Description

Technical Field

[0001] This invention belongs to the field of medical event recognition and entity linking technology, and particularly relates to an ICD automatic encoding method based on large language model transfer learning. Background Technology

[0002] In the medical field, standardized coding of diseases and procedures is a core component of information management. The International Classification of Diseases (ICD) coding system, as a widely used global medical coding standard, has played a positive role in promoting the standardization of disease diagnosis and treatment processes, data analysis, and international communication. It also provides fundamental support for medical insurance reimbursement, health management, and medical research. ICD coding comprises two main parts: disease diagnosis and surgical procedures. Each ICD code typically consists of letters and numbers, matched with a corresponding medical name. Due to its importance in medical informatization, the ICD coding system is widely used in major hospitals, medical insurance institutions, and the public health sector.

[0003] To address these issues, automated coding technology has become an important direction for improving the efficiency and accuracy of medical information processing. With the rapid development of Natural Language Processing (NLP) and Artificial Intelligence, automated coding technology has gradually gained widespread application. Traditional deep learning-based automated coding methods typically require a large amount of manually labeled, high-quality data for training, which makes them less adaptable and more costly to train. Especially in low-resource language environments, the lack of sufficient task-relevant data makes building an effective automated coding system particularly difficult.

[0004] Existing public datasets are mostly for high-resource languages ​​(such as English), with a lack of research on low-resource languages ​​(such as Chinese). Particularly in some regions, ICD-encoded data annotation still relies on manual work, leading to inefficiency and a high risk of errors. Therefore, overcoming data scarcity in low-resource language environments and improving the accuracy and universality of automatic encoding has become a pressing technical challenge.

[0005] To address the aforementioned issues, we propose a method and system for ICD auto-encoding based on large language model transfer learning. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for ICD automatic encoding based on large language model transfer learning, which aims to solve the problem of lack of task-related training data in low-resource language environments.

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0008] A method for automatic ICD encoding based on large language model transfer learning is used for data annotation of ICD encoding of low-resource language electronic medical record data. The method includes:

[0009] Obtain a medical data sample containing diagnostic text and a set of ICD codes, wherein the medical data sample has a corresponding correct ICD code;

[0010] For input low-resource language electronic medical record data, a large language model is used to identify the type of text to be encoded. The step guides the large language model to classify text by assembling prompt words using templates. The templates include role definition labels, encoding type labels, example labels, and input low-resource language electronic medical record data labels.

[0011] Based on the content to be encoded, a multilingual ICD automatic encoding instruction is constructed. The instruction includes the content to be encoded in a low-resource language and its corresponding high-resource language translation content. The translated high-resource language content is placed first, followed by the input low-resource language content, and each is labeled with a language tag.

[0012] Based on the content to be encoded, construct multilingual ICD automatic encoding instructions; use a language transfer ICD automatic encoding model to perform ICD automatic encoding on the content to be encoded and output the ICD encoding result; based on the encoding rules of low-resource languages, verify and align the output ICD encoding result and output the final encoding result.

[0013] On the one hand, an ICD automatic encoding system based on large language model transfer learning is provided, including:

[0014] The electronic medical record classification module is used to determine which type of encoding is required based on the input low-resource language electronic medical records.

[0015] The translation module is used to translate input low-resource language electronic medical records into high-resource language;

[0016] The multilingual instruction building module is used to construct multilingual ICD automatic encoding instructions based on the input resource language electronic medical records;

[0017] The training module is used to perform cross-language transfer training on large language models;

[0018] The deployment module is used to deploy large language models, making them accessible via API;

[0019] The code mapping module is used to map the results generated by the large language model to obtain ICD encoding that conforms to the low-resource language application environment;

[0020] The user interaction module is used to provide coding services to doctors and coders after the large language model has been deployed.

[0021] On the one hand, a readable storage medium is provided, the storage medium storing a computer program for a method or system of ICD auto-encoding based on large language model transfer learning, the computer program being able to implement the method of ICD auto-encoding based on large language model transfer learning when executed in a computing device.

[0022] Beneficial effects:

[0023] This invention, based on natural language processing technology, performs medical event recognition and entity linking on original electronic medical records. Using a public dataset in a high-resource language, it translates the data into a low-resource language using a translation model to construct parallel corpora. Transfer learning is then used to train the large language model. In the practical application phase, certain transformations and alignments are required based on the actual rules of ICD encoding in different language environments. This method leverages the powerful language capabilities of large language models, significantly reducing the cost of training a model from scratch to acquire ICD encoding capabilities in low-resource language environments. Through transfer learning, the model can quickly acquire task capabilities without the need for annotation. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of the ICD auto-encoding method based on large language model transfer learning of the present invention;

[0026] Figure 2 This is an example of the process of collecting, organizing, translating, and constructing parallel corpora;

[0027] Figure 3 This is an example of the transformation and mapping process when the ICD versions of high-resource languages ​​and low-resource languages ​​are inconsistent;

[0028] Figure 4 This is an example of the transformation process of different encoding rules in this invention;

[0029] Figure 5 This is a diagram showing the relationship between the modules of the automatic coding system of this invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] In a specific embodiment of this application, an automatic ICD encoding method based on large language model transfer learning is proposed to address the application problem of traditional ICD encoding in low-resource language environments. This method aims to improve the accuracy and universality of ICD encoding by leveraging the technical advantages of transfer learning.

[0032] To facilitate readers' understanding of the entire contents of this disclosure, the following terms appearing in this disclosure are explained. It should be understood that the explanations of terms in this document are only to assist readers' understanding and do not constitute a limitation on the technical solutions of this disclosure.

[0033] Large language models: These are deep learning models trained on massive amounts of text data that can generate natural language text or understand the meaning of language text. They typically refer to language models with billions or more parameters.

[0034] Transfer learning: refers to a deep learning training method that applies knowledge or patterns learned in one domain or task to different but related domains or problems;

[0035] like Figure 1 As shown in some disclosures, the method steps include:

[0036] Step S1: Input electronic medical record data in a low-resource language and use a large language model to identify the type of text to be encoded;

[0037] This step uses specific template-assembled prompt words to guide the large language model in text classification. The structure of the template-assembled prompt words is as follows:

[0038] "{Role Definition Label}: {Role definition statement for setting the large language model as an encoding type classifier}\n{Encoding Type Label}: {Set or sequence of encoding types}\n{Example Label}:\n{Example text and classification result}\n{Input low-resource language e-health data label}: {Statement guiding the large model to perform classification}\n{Input low-resource language e-health data}\n{Classification result output label}:"

[0039] The {example text and classification result} can contain a single example or multiple examples, or no examples. When examples are present, each example is preceded by a statement that guides the large model to perform classification.

[0040] In some specific implementations, taking the migration from an English language environment (high resources) to a Chinese language environment (low resources) as an example, the specific content of the template is determined based on the structural composition of the above-mentioned template assembly prompt words as follows:

[0041] "You are an excellent medical text classification expert, and you are familiar with various types of electronic medical records. Now you need to select the type of electronic medical record data in the 'Data to be Classified' from the classifications provided in 'Encoding Type'. You can refer to the examples provided in 'Examples' to complete your task."

[0042] [Encoding Type]

[0043] Disease diagnosis code, surgical procedure code

[0044]

Example

[0045] Example 1: {Example Text 1}

[0046] Output 1: {Classification result}

[0047] Example 2: {Example Text 2}

[0048] Output 2: {Classification result}

[0049] [Data to be categorized]

[0050] {Enter the data to be classified}

[0051]

Output

[0052] In step S2, the system constructs a multilingual ICD automatic encoding instruction based on the content to be encoded. This instruction includes the content to be encoded in the low-resource language input, and together with its corresponding high-resource language translation, forms the instruction. The specific construction method is as follows:

[0053] In step S2, based on the content to be encoded, a multilingual ICD automatic encoding instruction is constructed. The instruction includes the content to be encoded input in a low-resource language and its corresponding multilingual translation in a high-resource language. In this multilingual instruction, the translated high-resource language content comes first, followed by the input low-resource language content, and each is labeled with a language tag, as follows:

[0054] "{High-Resource Language Command Label}:{High-Resource Language Encoded Command}\n{Low-Resource Language Command Label}:{Low-Resource Language Encoded Command}\n{High-Resource Language Text Label}:\n{High-Resource Language Electronic Medical Record Text}:{Low-Resource Language Text Label}:\n{Low-Resource Language Electronic Medical Record Text}\n"

[0055] In some specific implementations, taking the migration from an English language environment (high resources) to a Chinese language environment (low resources) as an example, the specific content of the template is determined based on the structural composition of the above-mentioned template assembly prompt words as follows:

[0056] Command template: "Please extract the name of the surgical operation from the following medical record text:\nPlease extract the name of the surgical operation from the following medical record text:\nEnglish Text:\n{English medical record translated from the input Chinese medical record}\nChinese Text:\n{Input Chinese medical record text}\n."

[0057] This construction approach ensures that during the encoding process, the system can understand and process medical data in low-resource languages ​​(such as Chinese) while referencing its translated high-resource language counterpart (such as English). This bilingual instruction helps the large language model accurately understand and generate ICD encoding results.

[0058] Step S3: Use the language transfer ICD auto-encoding large model to perform ICD auto-encoding on the content to be encoded and output the ICD encoding result;

[0059] The training process of the aforementioned language transfer ICD autoencoder large model employs a cross-language transfer learning approach. The training process includes data collection and translation, construction of parallel corpora, and model fine-tuning. Specifically, it includes the following:

[0060] Collect, organize, and translate data: Select suitable high-resource language datasets and translate the data into low-resource languages.

[0061] Constructing parallel corpora: Aligning ICD-coded data of high-resource languages ​​with low-resource language data using parallel corpora.

[0062] Model fine-tuning: Fine-tuning the large model using constructed parallel corpora to adapt it to low-resource language environments.

[0063] A further aspect of this invention is: in the process of collecting, organizing, and translating data, publicly available datasets in the ICD encoding domain of high-resource language environments are collected and organized, and closed-source or open-source large language models with strong translation capabilities are used to translate the source language domain task datasets into low-resource languages, including input and output;

[0064] In constructing the parallel corpus, each data point from the high-resource language dataset is organized together with its corresponding translated data to form training corpus for large language model transfer learning. The input part is organized in the same way as the multilingual instruction construction method in step S2, and the output part consists only of the translated low-resource language content, as follows:

[0065] Input: "{High-Resource Language Command Label}:{High-Resource Language Encoding Command}\n{Low-Resource Language Command Label}:{Low-Resource Language Encoding Command}\n{High-Resource Language Text Label}:\n{High-Resource Language Electronic Medical Record Text}:{Low-Resource Language Text Label}:\n{Low-Resource Language Electronic Medical Record Text}\n"

[0066] Output: "{Encoding result conforming to encoding rules in low-resource language environments}"

[0067] A further technical solution of the present invention is as follows: During the process of collecting, organizing, and translating data, it is necessary to determine the consistency between the ICD encoding version, industry requirements, and actual situation used in the ICD encoding task of medical records in high-resource language datasets and low-resource language environments. If they are consistent, the input data is directly translated, and the output results are directly mapped using a standard dictionary. If they are inconsistent, they need to be transformed according to the actual situation before mapping using a standard dictionary. The standard dictionary refers to a mapping dictionary formed by directly associating the ICD encoding set issued by the functional departments or industry associations of low-resource language countries with the ICD encoding set of high-resource languages ​​through ICD codes or indirectly associating them after transformation.

[0068] For example, such as Figure 2 As shown, the process of collecting, organizing, translating, and constructing parallel corpora includes:

[0069] The English dataset used is the MIMIC dataset, which contains anonymized patient data collected since 2001 in the Intensive Care Unit (ICU) of Beth Israel Deaconess Medical Center. This database includes patient discharge records and their corresponding ICD codes. These discharge records were translated into Chinese, and the corresponding ICD codes were converted into Chinese encoding.

[0070] Specifically, when processing ICD encoding in the MIMIC library, since the current ICD encoding version in the Chinese environment is not completely consistent with the English encoding, it is necessary to map the Chinese ICD encoding version to the English ICD encoding according to certain rules to form a mapping dictionary.

[0071] For example, such as Figure 3The diagram shows a mapping between Chinese and English ICD codes. Taking surgical procedure codes as an example, the Chinese ICD code is the National Clinical Version 3.0 of Surgical Procedure Codes issued by the National Health Commission of China, while the English ICD code is ICD-9-CM-3 issued by the Centers for Disease Control and Prevention of the United States.

[0072] Step S4: Based on the encoding rules of the low-resource language, verify and align the output ICD encoding results, and output the final encoding results. The specific steps are as follows:

[0073] Based on the encoding rules of low-resource language, the output ICD encoding results are verified and aligned, and the final encoding result is output. If there are differences between the dataset in the high-resource language environment and the actual application in the low-resource language environment, then according to the transformation method in step 3, the consistency of the ICD encoding version, industry requirements, and actual situation used in the medical record ICD encoding task between the high-resource language dataset and the low-resource language environment is judged. If they are consistent, the model can be used directly to generate the result. If they are inconsistent, then according to the correspondence of encoding rules in different language environments, the result conforming to the encoding rules of the high-resource language environment is mapped or transformed to the result conforming to the encoding rules of the low-resource language environment. The output result is post-processed to make it conform to the actual application of the low-resource language environment.

[0074] For example, such as Figure 4 As shown, since the ICD encoding rules in the Chinese environment are different from those in the English environment, it is necessary to post-process the encoding results according to the correspondence between the encoding rules in the Chinese and English environments, and map the encoding results to the Chinese application environment for practical application or further processing.

[0075] Specifically, the process involves handling the inconsistencies between the output ICD encoding results and the encoding rules of the high-resource language. The method is to map the ICD encoding versions of the Chinese language to the ICD encoding of the high-resource language according to certain rules, thus mapping the high-resource language encoding results to the low-resource language application environment.

[0076] like Figure 5 As shown, this invention provides an ICD automatic coding system based on large language model transfer learning, aiming to solve the application problem of traditional ICD coding in low-resource language environments. This system improves the accuracy and universality of ICD coding through cross-language transfer learning technology, supporting automatic coding of electronic medical records in low-resource language environments. The core functions of the system include: electronic medical record classification, translation, instruction construction, transfer learning training, and result mapping modules.

[0077] Detailed description of system modules;

[0078] The electronic medical record classification module is used to determine which type of encoding is required based on the input low-resource language electronic medical records.

[0079] The translation module is used to translate input low-resource language electronic medical records into high-resource language;

[0080] The multilingual instruction building module is used to construct multilingual ICD automatic encoding instructions based on the input resource language electronic medical records;

[0081] The training module is used to perform cross-language transfer training on large language models.

[0082] The deployment module is used to deploy large language models, making them accessible via API;

[0083] The code mapping module is used to map the results generated by the large language model to obtain ICD encoding that conforms to the low-resource language application environment;

[0084] The user interaction module is used to provide coding services to doctors and coders after the large language model has been deployed.

[0085] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0086] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for automatic ICD encoding based on large language model transfer learning, used for data annotation of ICD encoding of electronic medical record data in low-resource languages, characterized in that, For input electronic medical record data in low-resource language, the first language model is used to identify the type of content to be encoded in the electronic medical record data in low-resource language. The first language model is guided to perform text classification by assembling prompt words using templates. The templates include role definition labels, encoding type labels, example labels, and labels of the input electronic medical record data in low-resource language. Based on the content to be encoded, a multilingual ICD automatic encoding instruction is constructed. The instruction includes the content to be encoded in a low-resource language and its corresponding high-resource language translation content. The translated high-resource language content is placed first, followed by the input low-resource language content, and each is labeled with a language tag. The multilingual ICD auto-encoding instructions are input into the language transfer ICD auto-encoding model, which then performs ICD auto-encoding on the content to be encoded and outputs the ICD encoding result. Based on the encoding rules of the low-resource language, the output ICD encoding results are verified and aligned, and the final encoding result is output. The training of the language transfer ICD auto-encoding large model adopts cross-language transfer learning. The training process includes collecting and organizing data and translating it, constructing parallel corpora, and fine-tuning the model. The collection, organization, and translation of data includes: Select a dataset of a high-resource language and use the second-largest language model to translate the high-resource language data into low-resource language data; The construction of the parallel corpus includes: Each aligned data point is used as training corpus for transfer learning of the basic large language model. The input part of the basic large language model is organized in the same way as the multilingual ICD automatic encoding instruction construction method, and the output part is the ICD encoding of the low-resource language. The model fine-tuning includes: The basic large language model is fine-tuned using the constructed parallel corpus to adapt it to low-resource language environments, resulting in a large language transfer ICD auto-encoding model.

2. The method for ICD auto-encoding based on large language model transfer learning according to claim 1, characterized in that, The structure of the template assembly prompt words consists of the following parts: {Set the role definition statement for the large language model as the encoding type classifier}\n{Encoding type label}\n{Encoding type shape set or sequence}\n{Example label}\n{Example text and classification result}\n{Input low-resource language electronic medical record data label}\n{Input low-resource language electronic medical record data}\n{Classification result output label}\n; The {Example Text and Classification Results} may contain a single example, multiple examples, or no examples. When examples are present, each example is preceded by a statement that guides the first language model to perform classification.

3. The method for ICD auto-encoding based on large language model transfer learning according to claim 1, characterized in that, The specific format of the multilingual ICD automatic encoding instructions is as follows: {High-resource language encoding instructions}\n{Low-resource language encoding instructions}\n{High-resource language text tags}:\n{High-resource language electronic medical record text}:{Low-resource language text tags}:\n{Low-resource language electronic medical record text}\n.

4. The method for ICD auto-encoding based on large language model transfer learning according to claim 1, characterized in that, The training process also includes: A consistency check is performed on the ICD encoding versions and industry requirements used by high-resource language data and low-resource language data in the ICD encoding task of medical record. If they are consistent, the output results of the basic large language model are directly mapped using a standard dictionary. If there is a discrepancy, the output of the basic large language model is first transformed, and then mapped using a standard dictionary.

5. The method for ICD auto-encoding based on large language model transfer learning according to claim 1, characterized in that, Based on the encoding rules of the low-resource language, the output ICD encoding results are verified and aligned, and the final output encoding results include: The consistency of ICD coding versions and industry requirements used for high-resource language data and low-resource language data in medical record ICD coding tasks is judged. If they are consistent, the language transfer ICD automatic coding big model is used to generate output results to obtain the final coding results. If they are inconsistent, the output result that conforms to the encoding rules of the high-resource language environment will be mapped or transformed to the result that conforms to the encoding rules of the low-resource language environment according to the correspondence of the encoding rules in different language environments, so as to obtain the final encoding result.

6. An ICD auto-encoding system based on large language model transfer learning, wherein the system employs the ICD auto-encoding method based on large language model transfer learning as described in claim 1, characterized in that, include: The electronic medical record classification module is used to determine what type of encoding is required based on the input low-resource language electronic medical record data. The translation module is used to translate the content to be encoded in the input low-resource language electronic medical record data into high-resource language data; The multilingual instruction building module is used to build multilingual ICD automatic encoding instructions based on the input content to be encoded; The training module is used to perform cross-language transfer training on the basic large language model. The deployment module is used to deploy the trained language transfer ICD autoencoder large model, making it accessible via API; The code mapping module is used to map the results generated by the language transfer ICD auto-encoding large model to obtain ICD encoding that conforms to the low-resource language application environment. The user interaction module is used to provide coding services to doctors and coders after the language transfer ICD auto-coding big model is deployed.

7. A readable storage medium, characterized in that, The storage medium stores a computer program that, when executed in a computing device, implements the ICD auto-encoding method based on large language model transfer learning as described in any one of claims 1 to 5.

Images