Text information and laboratory result feature extraction method and device based on large language model

Abstract

The invention relates to a text information and laboratory result feature extraction method and device based on a large language model. The method comprises the following steps: obtaining text information to be subjected to feature extraction and a laboratory result; converting a laboratory result into a laboratory result text through a preset format, and taking the laboratory result text and the text information as a text to be subjected to feature extraction; performing text preprocessing on the text to be subjected to feature extraction to obtain a preprocessed text; and inputting the preprocessed text into a pre-trained LLM model, and outputting a word-level vector and a sentence-level vector. According to the text information and laboratory result feature extraction method and device based on the large language model, text information and laboratory result feature extraction is carried out based on the LLM model, and the efficiency of related workers can be greatly improved. The invention belongs to the application of machine learning in the field of text recognition.

Description

Technical Field

[0001] The present invention relates to the technical field related to text information extraction, and in particular to a method and device for extracting features of text information and laboratory results based on a large language model. Background Art

[0002] Laboratory result and text feature extraction is designed to extract key information from data, simplify the analysis process, and improve efficiency and accuracy. Laboratory result extraction ensures the scientific nature and reliability of the data, while text feature extraction helps identify language patterns and key points, facilitating subsequent support for natural language processing and information retrieval.

[0003] Large Language Model (LLM) is a natural language processing technology based on deep learning, which aims to simulate the human ability to understand and generate language.

[0004] Current feature extraction methods mostly rely on manual analysis, which requires a lot of effort. If large language models can be applied to feature extraction from laboratory results and text information, the efficiency of related personnel will be greatly improved. Summary of the Invention

[0005] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a method and apparatus for extracting features of text information and laboratory results based on a large language model.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] Specifically, a method for extracting features from text information and laboratory results based on a large language model is proposed, including the following:

[0008] Obtain text information to be feature extracted and laboratory results;

[0009] Converting the laboratory result into a laboratory result text using a preset format, and using the laboratory result text and the text information together as text to be feature extracted;

[0010] Perform text preprocessing on the feature extraction text to obtain the preprocessed text;

[0011] The preprocessed text is input into the pre-trained LLM model, and word-level vectors and sentence-level vectors are output.

[0012] Furthermore, it is characterized in that, specifically, the laboratory result is converted into a laboratory result text by a preset format, including:

[0013] extracting relevant structured data from the laboratory results;

[0014] Performing missing value processing and outlier detection processing on the structured data to obtain processed data;

[0015] Perform unit conversion on the processed data to unify the units and obtain the data to be used;

[0016] Formatting the data to be used by a preset template to obtain a preliminary laboratory result text;

[0017] Marking the preliminary laboratory result text as abnormal according to the preset normal range of each indicator, and finding the abnormal content;

[0018] The abnormal content is corrected through manual verification and the laboratory result text is finally obtained.

[0019] Further, specifically, the data to be used is formatted using a preset template to obtain a preliminary laboratory result text, including:

[0020] Define template variables based on the type and field of laboratory results;

[0021] Design templates suitable for various laboratory results based on defined template variables;

[0022] The data to be used are formatted using the designed template to obtain a preliminary laboratory result text.

[0023] Furthermore, the method further comprises:

[0024] When the data to be used cannot be formatted using a preset template, auxiliary text generation is performed using NLG.

[0025] Furthermore, specifically, the feature extraction text is subjected to text preprocessing to obtain preprocessed text, including:

[0026] Removing irrelevant information from the text to be feature extracted, wherein the irrelevant information refers to spaces, line breaks, and HTML tags to obtain a first text;

[0027] Lowercase the first text to obtain a second text, that is, convert all English letters in the text into lowercase;

[0028] Removing punctuation marks and stop words from the second text to obtain a third text;

[0029] Splitting the third text into tokens to obtain a fourth text;

[0030] Performing stem extraction and lemmatization on the fourth text to obtain a fifth text;

[0031] Correcting the spelling errors in the fifth text yields the sixth text;

[0032] Removing emoticons, URLs, and special characters of a preset type from the sixth text to obtain a seventh text;

[0033] The seventh text is normalized to obtain a preprocessed text.

[0034] Furthermore, the specific training process of the pre-trained LLM model includes:

[0035] Collect text data for training the LLM model, and collect, clean, segment, and encode the text data to obtain processed text data;

[0036] Choose a Transformer architecture based on self-attention and multi-head attention mechanisms, and determine the number of layers, hidden units, and attention heads.

[0037] Train the model through forward propagation, calculating loss, backpropagation and gradient updates, and use batch training and optimizer to update parameters;

[0038] Learning is performed through unsupervised pre-training and supervised fine-tuning during model training;

[0039] Evaluate model performance and perform optimization;

[0040] Adjust the model hyperparameters to prevent overfitting and obtain the pre-trained LLM model.

[0041] Furthermore, specifically, model training is performed through forward propagation, loss calculation, backpropagation, and gradient update, and parameter updates are performed using batch training and optimizers, including:

[0042] Predetermine the weights for model training to initialize the model;

[0043] The processed text data is input into the initialized model, and the output of the model, i.e. the predicted probability of each token, is calculated through a multi-layer Transformer structure.

[0044] The cross entropy is used as the loss function of the model to calculate the difference between the model prediction value and the true value;

[0045] Calculate the gradient through the back-propagation algorithm and update the model parameters;

[0046] Multiple epochs of batch training are performed using the mini-batch training method.

[0047] Further, specifically, evaluate the model performance and perform tuning, including,

[0048] During the training process, the model is evaluated regularly, and the evaluation indicators used include accuracy, precision, recall, F1 score, and perplexity;

[0049] Among them, the accuracy rate reaches more than 80%, the precision reaches more than 70%, the recall rate is more than 60%, the F1 score reaches more than 70%, and the perplexity is less than 10, indicating that the model evaluation is qualified.

[0050] The learning rate scheduler is used to gradually adjust the learning rate to avoid overfitting or oscillation; gradient clipping is performed to prevent gradient explosion, and regularization is used to prevent overfitting.

[0051] The present invention also proposes a device for extracting features of text information and laboratory results based on a large language model, comprising the following:

[0052] A data acquisition module is used to obtain text information and laboratory results to be extracted;

[0053] A text conversion module, configured to convert the laboratory result into a laboratory result text using a preset format, and use the laboratory result text and the text information together as text to be feature extracted;

[0054] A preprocessing module is used to perform text preprocessing on the text to be feature extracted to obtain preprocessed text;

[0055] The feature extraction module is used to input the preprocessed text into the pre-trained LLM model and output word-level vectors and sentence-level vectors.

[0056] The beneficial effects of the present invention are:

[0057] This invention proposes a method and device for extracting features from text information and laboratory results based on a large language model. First, the laboratory results are converted into laboratory result text according to a preset format. This text and the text information are then used together as the text to be feature extracted. The text is then preprocessed to obtain preprocessed text suitable for LLM model recognition. Finally, a specifically designed and trained LLM model is used for rapid feature extraction. The proposed method and device for extracting features from text information and laboratory results based on a large language model significantly improves the efficiency of relevant personnel by extracting features from the text information and laboratory results based on the LLM model. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The above and other features of the present disclosure will become more apparent through a detailed description of the embodiments shown in conjunction with the accompanying drawings. The same reference numerals in the drawings of the present disclosure represent the same or similar elements. Obviously, the drawings described below are only some embodiments of the present disclosure. It is possible for a person skilled in the art to derive other drawings based on these drawings without inventive effort. In the drawings:

[0059] Figure 1 Shown is a flow chart of the method for extracting features of text information and laboratory results based on a large language model of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other unless there is a conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0061] Example 1, with reference to Figure 1 The present invention proposes a method for extracting features of text information and laboratory results based on a large language model, including the following:

[0062] Step 110: Obtain text information to be feature extracted and laboratory results;

[0063] Step 120: converting the laboratory result into a laboratory result text using a preset format, and using the laboratory result text and the text information together as the text to be feature extracted;

[0064] Step 130: preprocess the text to be feature-extracted to obtain a preprocessed text;

[0065] Step 140: Input the preprocessed text into the pre-trained LLM model, and output word-level vectors and sentence-level vectors.

[0066] In this first embodiment, laboratory results are first converted into laboratory result text according to a preset format. This laboratory result text and the text information are then used together as the text to be feature extracted. Subsequently, text preprocessing is performed to obtain preprocessed text suitable for LLM model recognition. Finally, a specifically designed and trained LLM model is used for rapid feature extraction. The proposed method and device for extracting features from text information and laboratory results based on a large language model, which extracts features from text information and laboratory results based on the LLM model, can significantly improve the efficiency of relevant personnel.

[0067] As a preferred embodiment of the present invention, it is characterized in that, specifically, the laboratory result is converted into a laboratory result text by a preset format, including:

[0068] extracting relevant structured data from the laboratory results;

[0069] Performing missing value processing and outlier detection processing on the structured data to obtain processed data;

[0070] Perform unit conversion on the processed data to unify the units and obtain the data to be used;

[0071] Formatting the data to be used by a preset template to obtain a preliminary laboratory result text;

[0072] Marking the preliminary laboratory result text as abnormal according to the preset normal range of each indicator, and finding the abnormal content;

[0073] The abnormal content is corrected through manual verification and the laboratory result text is finally obtained.

[0074] As a preferred embodiment of the present invention, specifically, the preliminary laboratory result text is obtained by formatting the data to be used using a preset template, including:

[0075] Define template variables based on the type and field of laboratory results;

[0076] Design templates suitable for various laboratory results based on defined template variables;

[0077] The data to be used are formatted using the designed template to obtain a preliminary laboratory result text.

[0078] As a preferred embodiment of the present invention, the method further comprises:

[0079] When the data to be used cannot be formatted using a preset template, auxiliary text generation is performed using NLG.

[0080] In this preferred embodiment, the steps of converting the laboratory results into laboratory result text using a preset format are as follows:

[0081] 1. Data preparation and cleaning

[0082] First, relevant structured data needs to be extracted from laboratory results, and necessary data cleaning and preprocessing must be performed.

[0083] 1.1 Data format standardization

[0084] Laboratory results are usually stored in tabular or structured formats (e.g., CSV, JSON, database tables, etc.). Ensure that the data is formatted consistently, with clear fields and standardized units. For example:

[0085] Test name (such as blood sugar, white blood cell count, etc.)

[0086] Measurement value (such as 5.2, 3.5, etc.)

[0087] Unit of measurement (e.g., mg / dL, mmol / L, etc.)

[0088] Measurement time (e.g., 2025-02-12 14:30)

[0089] Normal range (eg, 4.0-6.0 mg / dL)

[0090] 1.2 Dealing with Missing Values and Outliers

[0091] Missing value handling: Fill in missing data or remove incomplete records as needed.

[0092] Outlier Detection: Detect and handle extreme or unreasonable measurements to ensure the generated text is not misleading.

[0093] 1.3 Unit conversion (if necessary)

[0094] Standardize units and ensure that all results use the same measurement unit. For example, if the unit of blood glucose is standardized as mg / dL, if some of the original data is in mmol / L, it needs to be converted.

[0095] 2. Determine the template structure

[0096] To convert lab results into natural language text, you can use a preset template (rule-based or template-based NLG) to format structured data. A template typically consists of the following components:

[0097] 2.1 Defining template variables

[0098] Define template variables based on the type and fields of the lab results. For example:

[0099] Test name: blood sugar, blood pressure, cholesterol, etc.

[0100] Test results: 5.2, 120, 200, etc.

[0101] Normal range: 4.0-6.0, 90-120, etc.

[0102] Unit: mg / dL, mmHg, etc.

[0103] Timestamp: Test date and time

[0104] 2.2 Creating a general template

[0105] Design templates that are suitable for a variety of lab results. For example, a general template for blood glucose testing could be:

[0106] "Your blood sugar test result is {result}{unit}, which is within the normal range {normal_range}."

[0107] For other types of tests, you can define different templates based on your needs:

[0108] Blood sugar result: "Your blood sugar result is {result}{unit}, the normal range is {normal_range}, and the result is normal."

[0109] White blood cell count: "Your white blood cell count is {result}x10^9 / L, the normal range is {normal_range}, and the result is normal."

[0110] Liver function: "Your liver function test results show that the AST is {result} U / L, and the normal range is {normal_range}. Please consult a doctor for further interpretation."

[0111] 2.3 Design dynamic templates based on conditions

[0112] Different results may require different text to be generated depending on how their value compares to the normal range. For example:

[0113] If the result is within the normal range, a text similar to "Your test result is normal" may be generated.

[0114] If the result is outside the normal range, a warning or suggestion needs to be generated: "Your test result is outside the normal range, further examination is recommended."

[0115] 3. Data filling template

[0116] Populate the variables in the template based on lab data. The values in the lab data (such as test results, units, normal ranges, etc.) need to be matched with the variables defined in the template and formatted accordingly.

[0117] 3.1 Filling static variables

[0118] Populate static variables in the template directly with lab result data. For example:

[0119] result = 5.2

[0120] unit="mg / dL"

[0121] normal_range="4.0-6.0"

[0122] #Fill data using template

[0123] Your blood sugar test result is {result}{unit}, which is within the normal range {normal_range}.

[0124] "print(sentence)#Output: Your blood sugar test result is 5.2 mg / dL, which is between 4.0 and 6.0.

[0125] 3.2 Dynamic filling based on conditions

[0126] For text that needs to be dynamically generated based on the data range (such as abnormal result prompts), you can use conditional judgment to generate text:

[0127] result=8.5

[0128] normal_range = (4.0, 6.0)

[0129] unit="mg / dL"

[0130] if result<normal_range[0]or result> normal_range[1]:

[0131] sentence=f"Your blood sugar result is {result}{unit}, which is outside the normal range.

[0132] {normal_range[0]}-{normal_range[1]}, please consult a doctor. "else:

[0133] sentence=f"Your blood sugar test result is {result}{unit}, which is within the normal range

[0134] The result is normal if it is between {normal_range[0]} and {normal_range[1]}.

[0135] print(sentence)#Output: Your blood sugar result is 8.5mg / dL, which is outside the normal range of 4.0-6.0. Please consult a doctor.

[0136] 3.3 Multiple test result text generation

[0137] If a lab report contains multiple test results, you can generate separate text for each test item, or generate a comprehensive summary report:

[0138] lab_results=[

[0139] {"test_name":"Blood sugar","result":5.2,"unit":"mg / dL","normal_range":"4.0-6.0"},

[0140] {"test_name":"White blood cell count","result":4.5,"unit":"x10^9 / L","normal_range":

[0141] "4.0-11.0"},

[0142] {"test_name":"Liver function (AST)","result":35,"unit":"U / L","normal_range":"15-40"} ]

[0144] #Generate report for result in lab_results:

[0145] if result["result"]<float(result["normal_range"].split('-')[0])orresult["result"]>

[0146] float(result["normal_range"].split('-')[1]):

[0147] print(f"{result['test_name']}The test result is {result['result']}{result['unit']}, which is beyond the normal range {result['normal_range']}. Further inspection is recommended.")

[0148] else:

[0149] print(f"{result['test_name']}The test result is {result['result']}{result['unit']}, which is within the normal range {result['normal_range']}. The result is normal.")

[0150] Output:

[0151] The blood sugar test result was 5.2 mg / dL, which is between 4.0 and 6.0, so the result is normal.

[0152] The white blood cell count test result was 4.5x10^9 / L, which is between the normal range of 4.0-11.0, and the result is normal.

[0153] The liver function (AST) test result was 35U / L, which is within the normal range of 15-40, and the result is normal.

[0154] 4. Advanced Generation Methods: Natural Language Generation (NLG)

[0155] If the template method is not flexible enough, or if you need to generate more personalized and complex text, you can use NLG-based generation methods, such as deep learning-based text generation models (such as GPT, T5, etc.) to automatically generate laboratory reports. This method can generate more natural and fluent reports with more contextual information.

[0156] For example, using models like T5 or GPT, you can convert structured data into natural language text:

[0157] from transformers import T5ForConditionalGeneration,T5Tokenizer

[0158] #Load the pre-trained T5 model

[0159] model=T5ForConditionalGeneration.from_pretrained('t5-small')

[0160] tokenizer=T5Tokenizer.from_pretrained('t5-small')

[0161] #Input structured data and generate natural language text

[0162] input_text="Blood sugar: 5.2 mg / dL, normal range: 4.0-6.0"

[0163] input_ids=tokenizer.encode(input_text,return_tensors="pt")

[0164] #Generate text

[0165] generated_ids=model.generate(input_ids)

[0166] generated_text=tokenizer.decode(generated_ids[0],skip_special_tokens=True)

[0167] print(generated_text)

[0168] 5. Generate final report

[0169] Ultimately, the text of all test results can be stitched together into a complete lab report or formatted into an appropriate document (e.g. PDF, Word, etc.).

[0170] As a preferred embodiment of the present invention, specifically, the text to be feature extracted is preprocessed to obtain a preprocessed text, including:

[0171] Removing irrelevant information from the text to be feature extracted, wherein the irrelevant information refers to spaces, line breaks, and HTML tags to obtain a first text;

[0172] Lowercase the first text to obtain a second text, that is, convert all English letters in the text into lowercase;

[0173] Removing punctuation marks and stop words from the second text to obtain a third text;

[0174] Splitting the third text into tokens to obtain a fourth text;

[0175] Performing stem extraction and lemmatization on the fourth text to obtain a fifth text;

[0176] Correcting the spelling errors in the fifth text yields the sixth text;

[0177] Removing emoticons, URLs, and special characters of a preset type from the sixth text to obtain a seventh text;

[0178] The seventh text is normalized to obtain a preprocessed text.

[0179] In this preferred embodiment, considering that text preprocessing is an important step in natural language processing (NLP), it involves cleaning and formatting raw text data for further analysis or modeling. The preprocessing steps may vary depending on the specific task, so text preprocessing is performed through the following steps:

[0180] 1. Remove irrelevant information

[0181] First, it is necessary to remove unnecessary content from the text, which may not be helpful for subsequent processing and analysis.

[0182] 1.1 Remove spaces and line breaks

[0183] By deleting redundant spaces, line breaks and other whitespace characters in the text, the text format can be unified.

[0184] text = text.replace("\n","").replace("\r","").strip()

[0185] 1.2 Remove HTML tags

[0186] If the text contains HTML tags (such as web scraping data), regular expressions or a dedicated library (such as BeautifulSoup) can be used to remove these tags.

[0187] from bs4 import BeautifulSoup

[0188] text = " This is a test paragraph. "

[0189] clean_text = BeautifulSoup(text,"html.parser").get_text()

[0190] 2. Lowercase conversion

[0191] Many NLP tasks do not care about case differences, so converting the text to lowercase uniformly can avoid duplicate problems caused by different cases.

[0192] text = text.lower()

[0193] 3. Remove punctuation marks

[0194] Punctuation marks may have no direct impact on certain tasks (such as sentiment analysis, text classification), so punctuation marks can be removed.

[0195] import string

[0196] text = text.translate(str.maketrans('','',string.punctuation))

[0197] 4. Remove stop words

[0198] Stop words are words that appear frequently in a language but contribute little to semantic analysis, such as "的", "是", "在", "for", etc. Removing stop words can reduce the computational load and improve the accuracy of the model. Note: There is a small error in the original code for removing punctuation marks. In the line `text=text.translate(str.maketrans(”,”,string.punctuation))`, the double quotes around the comma in the first argument of `str.maketrans` should be single quotes to match the Python syntax. The corrected code is `text = text.translate(str.maketrans('','',string.punctuation))` in the translation.

[0199] You can use libraries such as NLTK and spaCy to remove stop words.

[0200] from nltk.corpus import stopwords

[0201] stop_words=set(stopwords.words('english'))

[0202] text="This is an example sentence."

[0203] filtered_text="".join([word for word in text.split()if word not instop_words])

[0204] 5. Participle

[0205] Tokenization is the process of breaking text into words or subwords, and is often a fundamental operation in text preprocessing. Tokenization can be performed at the word, character, or subword level.

[0206] Word-level tokenization: Split text into words (common in English).

[0207] Character-level tokenization: Split text into characters (common in languages like Chinese).

[0208] Subword segmentation: For some low-frequency words, subword units are used for word segmentation (such as the word segmenters of BERT and GPT).

[0209] For example, use NLTK to perform word-level segmentation on English text:

[0210] from nltk.tokenize import word_tokenize

[0211] text="This is an example sentence."

[0212] tokens = word_tokenize(text)

[0213] 6. Stemming and Lemmatization

[0214] Both stemming and lemmatization are processes that reduce different word forms to their base form. Stemming simplifies words by removing endings, while lemmatization reduces words to their root form by using lexical and part-of-speech information.

[0215] Stemming: For example, simplifying "running" to "run".

[0216] Lemmatization: Lemmatizes "running" to "run," but takes part of speech into account.

[0217] 6.1 Stemming

[0218] from nltk.stem import PorterStemmer

[0219] stemmer = PorterStemmer()

[0220] text="running runner runs"

[0221] tokens = text.split()

[0222] stemmed_tokens=[stemmer.stem(word)for word in tokens]

[0223] 6.2 Lemmatization

[0224] from nltk.stem import WordNetLemmatizerfrom nltk.corpus importwordnet

[0225] lemmatizer=WordNetLemmatizer()

[0226] text="running runner runs"

[0227] tokens = text.split()

[0228] lemmatized_tokens=[lemmatizer.lemmatize(word,pos=wordnet.VERB)forword in tokens]

[0229] 7. Remove numbers

[0230] Depending on the needs of the task, it is sometimes necessary to remove numbers from the text. Especially in tasks such as text classification and sentiment analysis, numbers may not be very meaningful.

[0231] import re

[0232] text="There are 10apples in the basket."

[0233] cleaned_text=re.sub(r'\d+',”,text)

[0234] 8. Spelling Correction

[0235] For some tasks, if there are spelling errors in the text, it may affect the accuracy of the analysis. Spelling correction can be done by some libraries (such as pyspellchecker).

[0236] from spellchecker import SpellChecker

[0237] spell=SpellChecker()

[0238] text="This is an exmaple sentence."

[0239] corrected_text="".join([spell.correction(word)for word in text.split()])

[0240] 9. Handle emojis, special characters, and abbreviations

[0241] If the text contains emoticons, URLs, email addresses, special characters, etc., you may need to remove them or convert them to a standard format.

[0242] import re

[0243] #Remove emoticons and special characters

[0244] text="I love NLP! Visit us at http: / / example.com"

[0245] cleaned_text = re.sub(r'[^\w\s]',",text) # remove non-alphanumeric characters

[0246] 10. Text normalization (such as word standardization)

[0247] For example, "TV" is normalized to "television" and "USA" is normalized to "United States".

[0248] text=text.replace("TV","television").replace("USA","United States")

[0249] 11. Text Sentence Segmentation

[0250] In some tasks, it may be more appropriate to divide text into sentences rather than pure words (e.g. text summarization, sentiment analysis, etc.).

[0251] from nltk.tokenize import sent_tokenize

[0252] text="This is the first sentence. This is the second sentence."

[0253] sentences=sent_tokenize(text)

[0254] 12. Text formatting (optional)

[0255] When performing certain tasks, you may also need to further format the text, for example, converting the text to a specific structure (such as JSON) or making marked parts of the text bold or italic.

[0256] As a preferred embodiment of the present invention, specifically, the training process of the pre-trained LLM model includes:

[0257] Collect text data for training the LLM model, and collect, clean, segment, and encode the text data to obtain processed text data;

[0258] Choose a Transformer architecture based on self-attention and multi-head attention mechanisms, and determine the number of layers, hidden units, and attention heads.

[0259] Train the model through forward propagation, calculating loss, backpropagation and gradient updates, and use batch training and optimizer to update parameters;

[0260] Learning is performed through unsupervised pre-training and supervised fine-tuning during model training;

[0261] Evaluate model performance and perform optimization;

[0262] Adjust the model hyperparameters to prevent overfitting and obtain the pre-trained LLM model.

[0263] As a preferred embodiment of the present invention, specifically, model training is performed by forward propagation, loss calculation, back propagation and gradient update, and parameter update using batch training and optimizer, including:

[0264] Predetermine the weights for model training to initialize the model;

[0265] The processed text data is input into the initialized model, and the output of the model, i.e. the predicted probability of each token, is calculated through a multi-layer Transformer structure.

[0266] The cross entropy is used as the loss function of the model to calculate the difference between the model prediction value and the true value;

[0267] Calculate the gradient through the back-propagation algorithm and update the model parameters;

[0268] Multiple epochs of batch training are performed using the mini-batch training method.

[0269] As a preferred embodiment of the present invention, specifically, evaluating the model performance and performing optimization includes:

[0270] During the training process, the model is evaluated regularly, and the evaluation indicators used include accuracy, precision, recall, F1 score, and perplexity;

[0271] The learning rate scheduler is used to gradually adjust the learning rate to avoid overfitting or oscillation; gradient clipping is performed to prevent gradient explosion, and regularization is used to prevent overfitting.

[0272] The present invention also proposes a device for extracting features of text information and laboratory results based on a large language model, comprising the following:

[0273] A data acquisition module is used to obtain text information and laboratory results to be extracted;

[0274] A text conversion module, configured to convert the laboratory result into a laboratory result text using a preset format, and use the laboratory result text and the text information together as text to be feature extracted;

[0275] A preprocessing module is used to perform text preprocessing on the text to be feature extracted to obtain preprocessed text;

[0276] The feature extraction module is used to input the preprocessed text into the pre-trained LLM model and output word-level vectors and sentence-level vectors.

[0277] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0278] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0279] Although the present invention has been described in considerable detail and with particularity with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be construed as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively encompassing the intended scope of the invention. In addition, the invention has been described above in terms of embodiments foreseen by the inventors for the purpose of providing a useful description, and those insubstantial modifications of the invention that are not currently foreseen may still represent equivalent modifications of the invention.

[0280] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. As long as the technical effects of the present invention are achieved by the same means, they shall fall within the scope of protection of the present invention. Within the scope of protection of the present invention, various modifications and variations of the technical solutions and / or implementation methods may be made.

Claims

1. A method for extracting features from text information and laboratory results based on a large language model, characterized in that: These include: Obtain text information to be feature extracted and laboratory results; Converting the laboratory result into a laboratory result text using a preset format, and using the laboratory result text and the text information together as text to be feature extracted; Perform text preprocessing on the feature extraction text to obtain the preprocessed text; The preprocessed text is input into the pre-trained LLM model, and word-level vectors and sentence-level vectors are output.

2. The method for extracting features of text information and laboratory results based on a large language model according to claim 1, characterized in that: Specifically, the laboratory results are converted into laboratory result texts through a preset format, including: extracting relevant structured data from the laboratory results; Performing missing value processing and outlier detection processing on the structured data to obtain processed data; Perform unit conversion on the processed data to unify the units and obtain the data to be used; Formatting the data to be used by a preset template to obtain a preliminary laboratory result text; Marking the preliminary laboratory result text as abnormal according to the preset normal range of each indicator, and finding the abnormal content; The abnormal content is corrected through manual verification and the laboratory result text is finally obtained.

3. The method for extracting features of text information and laboratory results based on a large language model according to claim 2, characterized in that: Specifically, the data to be used is formatted using a preset template to obtain a preliminary laboratory result text, including: Define template variables based on the type and field of laboratory results; Design templates suitable for various laboratory results based on defined template variables; The data to be used are formatted using the designed template to obtain a preliminary laboratory result text.

4. The method for extracting features of text information and laboratory results based on a large language model according to claim 2, characterized in that: The method further comprises, When the data to be used cannot be formatted using a preset template, auxiliary text generation is performed using NLG.

5. The method for extracting features of text information and laboratory results based on a large language model according to claim 1, characterized in that: Specifically, the feature extraction text is subjected to text preprocessing to obtain the preprocessed text, including: Removing irrelevant information from the text to be feature extracted, wherein the irrelevant information refers to spaces, line breaks, and HTML tags to obtain a first text; Lowercase the first text to obtain a second text, that is, convert all English letters in the text into lowercase; Removing punctuation marks and stop words from the second text to obtain a third text; Splitting the third text into tokens to obtain a fourth text; Performing stem extraction and lemmatization on the fourth text to obtain a fifth text; Correcting the spelling errors in the fifth text yields the sixth text; Removing emoticons, URLs, and special characters of a preset type from the sixth text to obtain a seventh text; The seventh text is normalized to obtain a preprocessed text.

6. The method for extracting features of text information and laboratory results based on a large language model according to claim 1, characterized in that: Specifically, the training process of the pre-trained LLM model includes: Collect text data for training the LLM model, and collect, clean, segment, and encode the text data to obtain processed text data; Choose a Transformer architecture based on self-attention and multi-head attention mechanisms, and determine the number of layers, hidden units, and attention heads. Train the model through forward propagation, calculating loss, backpropagation and gradient updates, and use batch training and optimizer to update parameters; Learning is performed through unsupervised pre-training and supervised fine-tuning during model training; Evaluate model performance and perform optimization; Adjust the model hyperparameters to prevent overfitting and obtain the pre-trained LLM model.

7. The method for extracting features of text information and laboratory results based on a large language model according to claim 6, characterized in that: Specifically, the model is trained through forward propagation, loss calculation, back propagation and gradient update, and parameter update using batch training and optimizer, including: Predetermine the weights for model training to initialize the model; The processed text data is input into the initialized model, and the output of the model, i.e. the predicted probability of each token, is calculated through a multi-layer Transformer structure. The cross entropy is used as the loss function of the model to calculate the difference between the model prediction value and the true value; Calculate the gradient through the back-propagation algorithm and update the model parameters; Multiple epochs of batch training are performed using the mini-batch training method.

8. The method for extracting features of text information and laboratory results based on a large language model according to claim 6, characterized in that: Specifically, evaluate model performance and perform optimization, including: During the training process, the model is evaluated regularly, and the evaluation indicators used include accuracy, precision, recall, F1 score, and perplexity; The learning rate scheduler is used to gradually adjust the learning rate to avoid overfitting or oscillation; gradient clipping is performed to prevent gradient explosion, and regularization is used to prevent overfitting.

9. A device for extracting features from text information and laboratory results based on a large language model, characterized in that: These include: A data acquisition module is used to obtain text information and laboratory results to be extracted; A text conversion module, configured to convert the laboratory result into a laboratory result text using a preset format, and use the laboratory result text and the text information together as text to be feature extracted; A preprocessing module is used to perform text preprocessing on the text to be feature extracted to obtain preprocessed text; The feature extraction module is used to input the preprocessed text into the pre-trained LLM model and output word-level vectors and sentence-level vectors.

Images