Large language model training method, device, equipment and medium

Abstract

The present invention provides a large language model training method, device, equipment and medium. The method includes determining multiple candidate configurations for a large language model, each candidate configuration freezes the parameters of one or more layers and has different frozen layers and / or numbers of layers; based on the same original training parameters, for each candidate configuration, using one of multiple training data groups for training to obtain a training group loss function and update the parameters of unfrozen layers, wherein the frozen layers in each candidate configuration retain the original training parameters and the parameters of the unfrozen layers are allowed to change with training; inputting another training data group to train the large language model until multiple training data groups are traversed, and determining a training set loss function based on the training group loss function; after traversing multiple training data groups, using a test data set for testing to obtain a test set loss function; and determining a target configuration based on the training set loss function and the test set loss function.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and more specifically, to a training method, device, equipment and medium for a large language model (LLM) based on the medical field. Background Art

[0002] With the development of the information industry, smart medical care using artificial intelligence technology is becoming more and more popular. In the case of limited high-quality medical resources, it is expected that artificial intelligence can reduce the workload of doctors, make medical resources more reasonably and efficiently allocated, and alleviate the shortage of medical resources.

[0003] A big language model is an artificial intelligence model designed to understand and generate human language. By training on a large amount of text data, a big language model can perform a wide range of tasks, including text summarization, reasoning, and more. The big language model currently has emerging capabilities that other machine learning models do not have - for example, context, instruction following, and step-by-step reasoning capabilities. Therefore, with the help of a big language model, key pathological information can be refined and extracted to achieve medical logical reasoning, thereby helping medical workers obtain the desired medical information more quickly, allowing medical workers to diagnose and treat quickly and accurately.

[0004] General large language models often have strong generalization characteristics, and their ability to understand general language logic has been fully demonstrated. However, general large language models lack the in-depth knowledge and business scenario-specific logic / knowledge required for medical scenarios (especially oncology), which makes it impossible for general large language models to effectively improve the judgment / diagnosis accuracy of medical scenarios. They will encounter more obstacles when judging the efficacy, medical history / situation, which will increase the burden on specialists and lead to more misjudgments.

[0005] Therefore, a large language model training method is needed to efficiently train the large language model while retaining the large language model's ability to understand general semantics to improve the large language model's understanding ability in medical scenarios. Summary of the invention

[0006] In response to the above problems, the present invention provides a large language model training method, apparatus, device and medium. The method determines multiple candidate configurations for the large language model to be trained, obtains a training set loss function and a test set loss function for each of the multiple candidate configurations based on the same original training parameters, and determines a target configuration based on the training set loss function and the test set loss function. This avoids the serious negative impact on the semantic understanding ability of the large language model itself due to the update of parameters of all layers when training a general large language model to understand medical terms, while improving the large language model's ability to understand medical scenarios.

[0007] According to one aspect of the present invention, a large language model training method is provided, comprising: determining a plurality of candidate configurations for a large language model to be trained, wherein in each of the plurality of candidate configurations, parameters of one or more layers of the large language model to be trained are frozen and each of the candidate configurations has a different frozen layer and / or number of layers; based on the same original training parameters, for each of the plurality of candidate configurations, using one of a plurality of training data groups included in a training data set to train the large language model to be trained to obtain a training group loss function and updating parameters of unfrozen layers in the large language model to be trained, wherein the frozen layers in each candidate configuration retain the original training parameters and the parameters of the unfrozen layers in each candidate configuration are allowed to change with training; in updating the After the parameters of the unfrozen layers in the large language model to be trained are input into another training data group among the multiple training data groups to train the large language model to be trained until the multiple training data groups are traversed, and a training set loss function is determined based on the training group loss functions corresponding to the multiple training data groups; after traversing the multiple training data groups, the large language model to be trained is tested using a test data set to obtain a test set loss function; for a first training set loss function and a first test set loss function corresponding to a first candidate configuration among the multiple candidate configurations, in response to a correlation between the first training set loss function and the first test set loss function being greater than a first predetermined threshold, and in response to the training set loss function and the test set loss function being less than a second predetermined threshold, the first candidate configuration is determined as the target configuration of the large language model to be trained.

[0008] According to some embodiments of the present invention, the large language model to be trained includes an input layer, an output layer, and a plurality of intermediate layers between the input layer and the output layer, and wherein the number of one or more layers frozen in each candidate configuration is less than the sum of the number of layers of the input layer, the output layer, and the intermediate layers.

[0009] According to some embodiments of the present invention, for a first training set loss function and a first test set loss function corresponding to a first candidate configuration among multiple candidate configurations, in response to a correlation between the first training set loss function and the first test set loss function being greater than a first predetermined threshold, and in response to the training set loss function and the test set loss function being less than a second predetermined threshold, determining the first candidate configuration as the target configuration of the large language model to be trained also includes: in response to the perplexity of the training set loss function and the test set loss function being less than a third predetermined threshold, determining the first candidate configuration as the target configuration of the large language model to be trained.

[0010] According to some embodiments of the present invention, determining a plurality of candidate configurations for the large language model to be trained further comprises determining the plurality of candidate configurations based on a grid search and / or a random search.

[0011] According to some embodiments of the present invention, the method further comprises: acquiring a training data set and a test data set.

[0012] According to some embodiments of the present invention, acquiring the training data set and the test data set further includes: acquiring medical text data, and extracting the training data set and the test data set from the medical text data.

[0013] According to some embodiments of the present invention, the medical text data includes basic tumor medical information, tumor medical scenario production data, and cutting-edge tumor medical research information.

[0014] According to some embodiments of the present invention, extracting a training data set and a test data set from the medical text data further includes: using a sliding window to obtain a plurality of fixed-length sentence groups with continuous sentences in the medical text data as training data or test data in the training data set or the test data set.

[0015] According to some embodiments of the present invention, the method further includes: in response to the text quality of the medical text data satisfying a predetermined condition, determining that the width of the sliding window is less than the number of sentences in the fixed-length sentence group; and in response to the text quality of the medical text data not satisfying the predetermined condition, determining that the width of the sliding window is greater than or equal to the number of sentences in the fixed-length sentence group.

[0016] According to some embodiments of the present invention, the predetermined condition includes one or more of a source for acquiring the medical text data, a text quality indicator of the medical text data, and a number of sentences contained in the medical text data.

[0017] According to some embodiments of the present invention, the method further includes: performing cluster analysis on the information in the medical text data to generate multiple categories based on the subject and / or features of the medical text data; and selecting a predetermined amount of data from each category in the generated multiple categories as training data or test data in the training data set or the test data set.

[0018] According to another aspect of the present invention, a large language model training device is provided, comprising: a candidate configuration determination unit, configured to determine a plurality of candidate configurations for a large language model to be trained, wherein in each of the plurality of candidate configurations, parameters of one or more layers of the large language model to be trained are frozen and each of the candidate configurations has a different frozen layer and / or number of layers; a training unit, configured to train the large language model to be trained based on the same original training parameters, for each of the plurality of candidate configurations, using one of the plurality of training data groups included in a training data set to obtain a training group loss function and update the parameters of the unfrozen layers in the large language model to be trained, wherein the frozen layers in each candidate configuration retain the original training parameters and the parameters of the unfrozen layers in each candidate configuration are allowed to change with training; the training unit is also configured to The method is configured to input another training data group among the multiple training data groups to train the large language model to be trained until the multiple training data groups are traversed, and determine the training set loss function based on the training group loss functions corresponding to the multiple training data groups; the testing unit is configured to test the large language model to be trained using the test data set to obtain the test set loss function after traversing the multiple training data groups; the target configuration determination unit is configured to determine the first configuration as the target configuration of the large language model to be trained for a first training set loss function and a first test set loss function corresponding to a first configuration among multiple candidate configurations in response to the correlation between the first training set loss function and the first test set loss function being greater than a first predetermined threshold, and in response to the training set loss function and the test set loss function being less than a second predetermined threshold.

[0019] According to some embodiments of the present invention, the large language model to be trained includes an input layer, an output layer, and a plurality of intermediate layers between the input layer and the output layer, and wherein the number of one or more layers frozen in each candidate configuration is less than the sum of the number of layers of the input layer, the output layer, and the intermediate layers.

[0020] According to some embodiments of the present invention, the target configuration determination unit is further configured to: in response to the perplexity of the training set loss function and the test set loss function being less than a third predetermined threshold, determine the first configuration as the target configuration of the large language model to be trained.

[0021] According to some embodiments of the present invention, the candidate configuration determining unit is further configured to determine the plurality of candidate configurations based on a grid search and / or a random search.

[0022] According to some embodiments of the present invention, the apparatus further comprises: a data acquisition unit configured to acquire a training data set and a test data set.

[0023] According to some embodiments of the present invention, the data acquisition unit is further configured to: acquire medical text data, and extract a training data set and a test data set from the medical text data.

[0024] According to some embodiments of the present invention, the medical text data includes basic tumor medical information, tumor medical scenario production data, and cutting-edge tumor medical research information.

[0025] According to some embodiments of the present invention, the data acquisition unit is further configured to: acquire multiple fixed-length sentence groups with continuous sentences in the medical text data using a sliding window as training data or test data in the training data set or the test data set.

[0026] According to some embodiments of the present invention, the data acquisition unit is further configured to: in response to the text quality of the medical text data satisfying a predetermined condition, determine that the width of the sliding window is less than the number of sentences in the fixed-length sentence group; and in response to the text quality of the medical text data not satisfying a predetermined condition, determine that the width of the sliding window is greater than or equal to the number of sentences in the fixed-length sentence group.

[0027] According to some embodiments of the present invention, the predetermined condition includes one or more of a source for acquiring the medical text data, a text quality indicator of the medical text data, and a number of sentences contained in the medical text data.

[0028] According to some embodiments of the present invention, the data acquisition unit is further configured to: perform cluster analysis on the information in the medical text data to generate multiple categories based on the subject and / or characteristics of the medical text data; and select a predetermined amount of data from each category in the generated multiple categories as training data or test data in the training data set or the test data set.

[0029] According to another aspect of the present invention, there is provided an electronic device, comprising: a processor; and a memory, wherein the memory stores a computer-readable code, and when the computer-readable code is executed by the processor, the large language model training method described above is implemented.

[0030] According to another aspect of the present invention, a non-transitory computer-readable storage medium is provided, storing computer-readable instructions, wherein when the computer-readable instructions are executed by a processor, the aforementioned large language model training method is implemented.

[0031] Therefore, according to the large language model training method, apparatus, device and medium of the embodiment of the present invention, multiple candidate configurations are determined for the large language model, each of the multiple candidate configurations freezes the parameters of one or more layers of the large language model and has different frozen layers and / or numbers of layers; based on the same original training parameters, for each candidate configuration, one of the multiple training data groups is used for training to obtain the training group loss function and update the parameters of the unfrozen layers, the frozen layers in each candidate configuration retain the original training parameters and the parameters of the unfrozen layers in each candidate configuration are allowed to change with the training; then another training data group is input to train the large language model until multiple training data groups are traversed, and the training set loss function is determined based on the training group loss function; after traversing multiple training data groups, a test data set is used for testing to obtain a test set loss function; and the target configuration is determined based on the training set loss function and the test set loss function. Obtaining the target configuration of the large language model by this method can avoid the general semantic understanding ability of the large language model itself from being seriously negatively affected when training for medical scenarios, and at the same time can efficiently improve the understanding ability of the large language model for medical scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some exemplary embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0033] Figure 1 A flowchart of a large language model training method according to some embodiments of the present invention is shown;

[0034] Figure 2 A block diagram of a large language model training apparatus according to some embodiments of the present invention is shown;

[0035] Figure 3 A structural diagram of an electronic device according to some embodiments of the present invention is shown. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. "First", "second" and similar words used in the present invention do not represent any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present invention clear and concise, the present invention omits the detailed description of some known functions and known components.

[0038] The present invention uses a flow chart to illustrate the steps of the method according to an embodiment of the present invention. It should be understood that the preceding or following steps are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more steps may be removed from these processes.

[0039] In the specification and drawings of the present invention, elements are described in singular or plural form, depending on the embodiment. However, the singular and plural forms are appropriately selected for the proposed situation only for the convenience of explanation and are not intended to limit the present invention thereto. Therefore, the singular form may include the plural form, and the plural form may also include the singular form, unless the context clearly indicates otherwise.

[0040] The following will describe in detail the method, device, equipment and medium for training and reasoning the medical large language model provided by the present invention with reference to the accompanying drawings.

[0041] First embodiment

[0042] In order to train a large language model for application in medical scenarios, the entire model of the large language model can be trained. However, through practice and experiments, it is found that when all layers of the large language model are updated with parameters, the semantic understanding ability of the large language model itself may be seriously negatively affected. In addition, when only the shallower layers of the large language model are updated, the large language model may not be able to effectively learn the medical scenario knowledge in the training data. In order to improve the understanding ability of the large language model in medical scenarios while retaining the semantic understanding ability of the pre-trained large language model, Figure 1 A flowchart of a large language model training method according to some embodiments of the present invention is shown.

[0043] First, in step S102, multiple candidate configurations can be determined for the large language model to be trained, wherein parameters of one or more layers of the large language model to be trained can be frozen in each of the multiple candidate configurations and each candidate configuration can have different frozen layers and / or numbers of layers.

[0044] According to one embodiment of the present invention, the large language model to be trained may include an input layer, an output layer, and a plurality of intermediate layers between the input layer and the output layer, and the number of layers of the one or more layers frozen in each candidate configuration is less than the sum of the number of layers of the input layer, the output layer, and the intermediate layers. By freezing some layers of the large language model instead of all layers, the training efficiency can be effectively improved.

[0045] For example, candidate configuration 1 can freeze the parameters of the bottom layer, candidate configuration 2 can freeze the parameters of layer X and layer Y, etc., where X and Y are arbitrary layers of the large language model. However, randomly freezing the parameters of some layers of the large language model does not necessarily improve the understanding ability of the large language model in medical scenarios, so it is necessary to further determine the configuration that can effectively improve the understanding ability of the large language model in medical scenarios.

[0046] According to one embodiment of the present invention, multiple candidate configurations can be determined based on grid search and / or random search. In one example, based on random search, a fixed total number of frozen layers can be selected, and a predetermined number of layers are unfrozen for each configuration, and a new predetermined number of layers are frozen at the same time. The predetermined number of layers is less than or equal to the total number of frozen layers. In one example, when the total number of layers of a large language model to be trained is small, multiple candidate configurations can be determined based on a grid search. Specifically, the frozen layers and the number of layers are arranged and combined, and all possible candidate configuration combinations are listed to generate a grid to determine multiple candidate configurations.

[0047] According to an embodiment of the present invention, before training, a training data set and a test data set may be obtained. In one example, the training data set and the test data set may be obtained before or after determining multiple candidate configurations.

[0048] The above steps are not necessarily performed in exact order. Instead, the steps may be processed in reverse order or simultaneously.

[0049] Similarly, the execution order of the methods or parts of the methods implemented in the subsequent steps S104, S106, S108 and S110 does not necessarily have to be carried out precisely in that order. Instead, these steps may be processed in reverse order or in parallel.

[0050] According to an example of the present invention, the training data set and the test data set may be directly obtained.

[0051] According to one embodiment of the present invention, medical text data can also be obtained, and training data sets and test data sets can be extracted from the medical text data. According to one embodiment of the present invention, medical text data can include basic tumor medical information, tumor medical scene production data, and cutting-edge tumor medical research information. These data play a great role in improving the ability / knowledge of the large language model in terms of data quality and the depth of knowledge of tumor medical scenes. According to practical experience, including the above three types of medical text data in the training data helps to improve the understanding of the medical scene by the large language model.

[0052] In one example, basic tumor medical information may include medical textbooks, encyclopedias, evaluation / diagnosis / treatment standards, drug instructions, etc. These data may provide explanations of basic medical terms, basic production rules, etc.

[0053] In one example, the tumor medical scenario production data may include, for example, desensitized medical texts, which may include medical records, medical history records, ward rounds records, etc. These data may provide real records / information related to tumors in the real world, thereby providing rich application scenario knowledge for the large language model.

[0054] In one example, cutting-edge tumor medical research information may include advanced medical research (academic papers, academic blogs, etc.), etc. These data can provide advanced medical research results and enhance the knowledge coverage of the large language model in the field of advanced tumor medicine.

[0055] In one example, after obtaining the medical text data, the data may be preprocessed to improve the data quality and / or quantity.

[0056] According to one embodiment of the present invention, a plurality of fixed-length sentence groups with continuous sentences can be obtained in medical text data using a sliding window as training data or test data in a training data set or a test data set. For very long paragraphs, more contextual knowledge and logical connections between knowledge can be retained by using a sliding window. In one example, for example, a fixed-length text group can be constructed as training data or test data based on the paragraph topic division of medical text data in combination with a sliding window. For example, [sentence 1, sentence 2, ..., sentence n] can be used as a sentence group, and for the next sentence group, [sentence x, sentence x+1, ..., sentence x+n] is used.

[0057] According to one embodiment of the present invention, in response to the text quality of the medical text data satisfying a predetermined condition, determining that the width of the sliding window is less than the number of sentences in the fixed-length sentence group; and in response to the text quality of the medical text data not satisfying the predetermined condition, determining that the width of the sliding window is greater than or equal to the number of sentences in the fixed-length sentence group. According to an embodiment of the present invention, the predetermined condition includes one or more of the acquisition source of the medical text data, the text quality identifier of the medical text data, and the number of sentences contained in the medical text data.

[0058] For example, when the amount of data (e.g., the number of sentences contained in the medical text data) is limited and / or the text quality is high (e.g., indicated as having high text quality determined according to the source of the medical text data), the number x (i.e., the width of the sliding window) can be less than n (i.e., the number of sentences in the fixed-length sentence group). For example, x=1. For cases where the amount of data is large and / or the data quality is average, n can be used as the width of the sliding window to ensure that the negative effects of the introduced noise will not be amplified due to data duplication, thereby reducing the training quality of the large language model.

[0059] According to one embodiment of the present invention, cluster analysis can also be performed on the information in the medical text data based on the subject and / or characteristics of the medical text data to generate multiple categories; a predetermined amount of data is selected from each category of the generated multiple categories as training data in the training data set or test data in the test data set.

[0060] In one example, for example, medical text topics, features, etc. can be defined (for example, they can be defined by medical experts / doctors, etc., or predetermined), combined with statistical parameters such as entropy, distribution, and other factors, and combined with simple machine learning unsupervised learning algorithms such as KNN, DBSCAN, etc.), a large amount of medical scene data can be clustered to achieve quality screening. In this way, a large amount of medical scene data can be filtered quickly and efficiently, and the parts with high repetition rates and similar contents can be classified, thereby reducing the repetition rate of training data and reducing the negative impact of noise, thereby improving the training effect.

[0061] After multiple candidate configurations are determined for the large language model to be trained, in step S104, based on the same original training parameters, for each of the multiple candidate configurations, the large language model to be trained can be trained using one of the multiple training data groups included in the training data set to obtain a training group loss function and update the parameters of unfrozen layers in the large language model to be trained, wherein the frozen layers in each candidate configuration retain the original training parameters and the parameters of the unfrozen layers in each candidate configuration are allowed to change with training.

[0062] By using the same original training parameters, a comparison based on the loss function for each candidate configuration can be achieved.

[0063] In one example, during the training process, for each of the multiple candidate configurations, one epoch of training may be performed based on one training data set. For example, in the first epoch for the first candidate configuration, multiple training data groups included in the training data set may be used for training. Specifically, a batch of training is performed using the first training data group among the multiple training data groups, and after the training, a loss function of the first training data group is obtained, and the parameters of the unfrozen layers in the large language model to be trained are updated based on the loss function or manual evaluation.

[0064] After using one training data group to train the large language model to be trained to obtain the training group loss function and updating the parameters of the unfrozen layers in the large language model to be trained, in step S106, another training data group from the multiple training data groups can be input to train the large language model to be trained until the multiple training data groups are traversed, and the training set loss function is determined based on the training group loss functions corresponding to the multiple training data groups.

[0065] Continuing with the above example, in the first cycle, the second training data group in the training data set is continued to be used for the next batch of training, and the loss function of the second training data group is obtained after the training, and the parameters of the unfrozen layers in the large language model to be trained are further updated based on the loss function or manual evaluation. In one example, the training set loss function for the first candidate configuration of the first candidate configuration can be obtained based on the average of the loss functions of the multiple training data groups obtained in the first cycle or by weighting.

[0066] In step S108, after traversing multiple training data sets, the test data set may be used to test the large language model to be trained to obtain a test set loss function.

[0067] Continuing with the above example, after completing all training of the first cycle based on multiple training data groups in the training data set, the test data set can be input into the trained large language model to obtain a test set loss function of the first candidate configuration.

[0068] Repeat multiple cycles of training to determine a training set loss function and a test set loss function for each candidate configuration in the multiple candidate configurations. In the multiple cycles of training, the training data set may be the same. In addition, the training data set used by the batches in each cycle and the order of use may also be the same.

[0069] In step S110, for the first training set loss function and the first test set loss function corresponding to the first candidate configuration among the multiple candidate configurations, in response to the correlation between the first training set loss function and the first test set loss function being greater than a first predetermined threshold, and in response to the training set loss function and the test set loss function being less than a second predetermined threshold, the first candidate configuration can be determined as the target configuration of the large language model to be trained, thereby selecting one or more optimal training configurations for the required medical field for subsequent further training, thereby avoiding serious negative impact on the general semantic understanding ability of the large language model itself when training for medical scenarios, and at the same time efficiently improving the large language model's ability to understand medical scenarios.

[0070] In one example, the target configuration finally determined for the large language model may be one or more target configurations that meet the conditions among multiple candidate configurations. The large language model is then further trained based on the one or more determined target configurations.

[0071] In one example, the correlation between the first training set loss function and the first test set loss function can be calculated based on the Pearson correlation coefficient. When the correlation is small, there may be overfitting during the training process. Therefore, only when the correlation between the first training set loss function and the first test set loss function is greater than the first predetermined threshold, the candidate configuration is considered to be determined as the target configuration of the large language model to be trained.

[0072] In one example, a lower loss function generally indicates a more accurate model. Therefore, when the training set loss function and the test set loss function are less than the second predetermined threshold, the candidate configuration is considered to be determined as the target configuration of the large language model to be trained.

[0073] According to one embodiment of the present invention, alternatively or additionally, in response to the perplexity of the training set loss function and the test set loss function being less than a third predetermined threshold, the first candidate configuration may be determined as the target configuration of the large language model to be trained.

[0074] The perplexity can be represented, for example, by the cross entropy loss power function of e. If the perplexity of the model is 5, it means that each time the model predicts the next word, it is equivalent to choosing from about 5 possible words. Therefore, the lower the perplexity, the more accurate the model. Therefore, when the training set loss function and the test set loss function are less than the second predetermined threshold, the candidate configuration is considered to be determined as the target configuration of the large language model to be trained.

[0075] In one example, those skilled in the art may determine the above-mentioned predetermined threshold value based on common knowledge in the art and experimental results.

[0076] After determining the target configuration of the large language model, the target configuration can be used for further training. In one example, the large language model can be fine-tuned based on the target configuration.

[0077] In one example, a large language model can be fine-tuned based on SFT, LORA, etc.

[0078] In one example, the fine-tuned training data can also be further modified to further enhance the fine-tuning effect:

[0079] 1. Specific tasks and indicators for oncology medical scenarios can be defined. For each type of problem, relevant medical definitions, medical indicators, production rules, and annotation standards can be summarized and defined. For example, for postoperative complications, the judgment logic of surgical complications and the key logic that needs to be defined / paid attention to can be defined, and the logic can be indexed.

[0080] 2. The basic large language model is used to combine the medical source text with the indicators and production rules defined in 1) to output the judgment of the large language model.

[0081] 3. By reviewing the output of the big language model of some desensitized medical texts (such as medical records, disease management, etc.), we can summarize the types of problems that occur in the big language model; and for typical texts, we provide example annotations and detailed production rules.

[0082] 4. Based on the example annotations in 3) and the indicators and production rules defined in 1), annotate all medical record texts and prepare relevant medical reasoning logic.

[0083] 5. For cases with doubts / uncertainties during the annotation process, repeat the process in 1 by performing batch / topic summarization to provide example annotations and production rule indicator updates. Repeat the process in 2) by combining the new example annotations and updated production rule indicators to generate a new large language model output, and repeat step 4 to annotate medical text.

[0084] After the above data annotation and fine-tuning of the large language model, a better configuration can be further selected from multiple candidate configurations based on the aforementioned large language model training method for determining the target configuration.

[0085] In one example, after selecting a better configuration, the large language model can be further trained using reinforcement learning with human feedback (RLHF).

[0086] For medical application scenarios, some differences in medical history / process management may lead to differences in conditions / diagnoses. At the same time, due to the differences between patients, there may also be large differences in the expression / diagnosis level. In order to make the large language model of the medical scenario closer to the doctor's ability / judgment, the present invention can also use reinforcement learning with human feedback (RLHF).

[0087] Specifically, the large language model for medical scenarios can be prepared using data from reinforcement learning with human feedback (RLHF):

[0088] Using the medical text categories / topics sorted in the fine-tuning data preparation process, we can filter out similar but differently processed medical texts; for example, text A: label A, text B: label B, and text C: label C. Texts A, B, and C are similar to each other, but they have some differences. We construct the following data for reinforcement learning with human feedback (RLHF):

[0089] For text A: Annotation A: 5 Annotation B: 4 Annotation C: 3

[0090] For text B: Annotation B: 5 Annotation C: 4 Annotation A: 3

[0091] etc.

[0092] By using the generative characteristics of the large language model and adjusting and fine-tuning the large language model output parameters, multiple outputs are generated for each medical text, and the medical annotation team evaluates and scores the results. The outputs of the large language model are ranked and scored based on relevance, connectivity, etc., to achieve reinforcement learning with human feedback (RLHF) training. In addition, after the RLHF stage, the large language model can be further optimized using the aforementioned method for determining the target configuration.

[0093] Combination of the above Figure 1 The method for training a large language model is described in detail. The method determines multiple candidate configurations for a large language model, each of the multiple candidate configurations freezes the parameters of one or more layers of the large language model and has different frozen layers and / or numbers of layers; based on the same original training parameters, for each candidate configuration, one of the multiple training data groups is used for training to obtain a training group loss function and update the parameters of the unfrozen layers, the frozen layers in each candidate configuration retain the original training parameters and the parameters of the unfrozen layers in each candidate configuration are allowed to change with the training; then another training data group is input to train the large language model until multiple training data groups are traversed, and a training set loss function is determined based on the training group loss function; after traversing multiple training data groups, a test data set is used for testing to obtain a test set loss function; and a target configuration is determined based on the training set loss function and the test set loss function. Obtaining the target configuration of the large language model by this method can avoid the general semantic understanding ability of the large language model itself from being seriously negatively affected when training for medical scenarios, and at the same time can efficiently improve the understanding ability of the large language model for medical scenarios.

[0094] Second embodiment

[0095] In addition to providing the above-mentioned large language model training method, the present invention also provides a large language model training device. Figure 2 This is described in detail.

[0096] Figure 2 FIG. 4 is a block diagram of a large language model training apparatus according to some embodiments of the present invention. Figure 2 As shown, the large language model training device 200 of the present invention may include a candidate configuration determination unit 210 , a training unit 220 , a testing unit 230 , and a target configuration determination unit 240 .

[0097] According to some embodiments of the present invention, the data acquisition unit 210 may be configured to determine a plurality of candidate configurations for a large language model to be trained, wherein parameters of one or more layers of the large language model to be trained are frozen in each of the plurality of candidate configurations and each candidate configuration has a different frozen layer and / or number of layers.

[0098] The training unit 220 can be configured to train the large language model to be trained based on the same original training parameters, for each candidate configuration in the multiple candidate configurations, using one training data group from the multiple training data groups included in the training data set to obtain a training group loss function and update the parameters of the unfrozen layers in the large language model to be trained, wherein the frozen layers in each candidate configuration retain the original training parameters and the parameters of the unfrozen layers in each candidate configuration are allowed to change with training.

[0099] The training unit 220 can also be configured to, after updating the parameters of the unfrozen layers in the large language model to be trained, input another training data group from the multiple training data groups to train the large language model to be trained until the multiple training data groups are traversed, and determine the training set loss function based on the training group loss functions corresponding to the multiple training data groups.

[0100] The testing unit 230 may be configured to test the large language model to be trained using the test data set after traversing multiple training data sets to obtain a test set loss function.

[0101] The data filtering unit 240 can be configured to determine, for a first training set loss function and a first test set loss function corresponding to a first configuration among multiple candidate configurations, the first configuration as the target configuration of the large language model to be trained in response to a correlation between the first training set loss function and the first test set loss function being greater than a first predetermined threshold, and in response to the training set loss function and the test set loss function being less than a second predetermined threshold.

[0102] According to some embodiments of the present invention, the large language model to be trained includes an input layer, an output layer, and a plurality of intermediate layers between the input layer and the output layer, and wherein the number of one or more layers frozen in each candidate configuration is less than the sum of the number of input layers, output layers, and intermediate layers.

[0103] According to some embodiments of the present invention, the target configuration determination unit 240 may also be configured to determine the first configuration as the target configuration of the large language model to be trained in response to the perplexity of the training set loss function and the test set loss function being less than a third predetermined threshold.

[0104] According to some embodiments of the present invention, the candidate configuration determining unit 210 may also be configured to determine multiple candidate configurations based on a grid search and / or a random search.

[0105] According to some embodiments of the present invention, the apparatus may further include: a data acquisition unit configured to acquire a training data set and a test data set.

[0106] According to some embodiments of the present invention, the data acquisition unit may also be configured to: acquire medical text data, and extract a training data set and a test data set from the medical text data.

[0107] According to some embodiments of the present invention, medical text data may include basic tumor medical information, tumor medical scenario production data, and cutting-edge tumor medical research information.

[0108] According to some embodiments of the present invention, the data acquisition unit may also be configured to acquire multiple fixed-length sentence groups with continuous sentences in the medical text data using a sliding window as training data or test data in a training data set or a test data set.

[0109] According to some embodiments of the present invention, the data acquisition unit may also be configured to, in response to the text quality of the medical text data satisfying a predetermined condition, determine that the width of the sliding window is less than the number of sentences in the fixed-length sentence group; and in response to the text quality of the medical text data not satisfying the predetermined condition, determine that the width of the sliding window is greater than or equal to the number of sentences in the fixed-length sentence group.

[0110] According to some embodiments of the present invention, the predetermined condition may include one or more of a source for obtaining the medical text data, a text quality indicator of the medical text data, and a number of sentences contained in the medical text data.

[0111] According to some embodiments of the present invention, the data acquisition unit may also be configured to perform cluster analysis on the information in the medical text data to generate multiple categories based on the subject and / or features of the medical text data; and select a predetermined amount of data from each category in the generated multiple categories as training data or test data in a training data set or a test data set.

[0112] about Figure 2 Some specific details of the large language model training device shown can also be referred to Figure 1 The content of the large language model training method shown in .

[0113] Figure 3 A structural diagram of an electronic device according to some embodiments of the present invention is shown.

[0114] See also Figure 3 , the electronic device 300 may include a processor 301 and a memory 302. The processor 301 and the memory 302 may be connected via a bus 303. The electronic device 300 may be any type of portable device (such as a smart camera, a smart phone, a tablet computer, etc.) or any type of fixed device (such as a desktop computer, a server, etc.).

[0115] The processor 301 can perform various actions and processes according to the program stored in the memory 302. Specifically, the processor 301 can be an integrated circuit chip with signal processing capabilities. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can be an X86 architecture or an ARM architecture.

[0116] The memory 302 stores computer executable instructions, and the above-mentioned large language model training method is implemented when the computer executable instructions are executed by the processor 301. The memory 302 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting explanation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that the memory of the method described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0117] In addition, the large language model training method according to the present invention can be recorded in a computer-readable recording medium. Specifically, according to the present invention, a computer-readable recording medium storing computer-executable instructions can be provided, and when the computer-executable instructions are executed by a processor, the processor can be prompted to execute the large language model training method as described above.

[0118] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code contains at least one executable instruction for realizing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] In general, various example embodiments of the present invention may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of embodiments of the present invention are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0120] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0121] The above is an explanation of the present invention and should not be considered as a limitation thereof. Although several exemplary embodiments of the present invention have been described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined in the claims. It should be understood that the above is an explanation of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.

Claims

1. A large language model training method for the medical field, comprising: Acquire medical text data, and extract a training data set and a test data set from the medical text data; Determine a plurality of candidate configurations for a large language model to be trained, wherein parameters of one or more layers of the large language model to be trained are frozen in each of the plurality of candidate configurations and each of the candidate configurations has a different frozen layer and / or number of layers; Based on the same original training parameters, for each candidate configuration in the multiple candidate configurations, use one training data group in the multiple training data groups included in the training data set to train the large language model to be trained to obtain a training group loss function and update the parameters of the unfrozen layers in the large language model to be trained, wherein the frozen layers in each candidate configuration retain the original training parameters and the parameters of the unfrozen layers in each candidate configuration are allowed to change with training; After updating the parameters of the unfrozen layers in the large language model to be trained, inputting another training data group from the multiple training data groups to train the large language model to be trained until the multiple training data groups are traversed, and determining the training set loss function based on the training group loss functions corresponding to the multiple training data groups; After traversing the plurality of training data sets, using the test data set to test the large language model to be trained to obtain a test set loss function; For a first training set loss function and a first test set loss function corresponding to a first candidate configuration among the multiple candidate configurations, in response to a correlation between the first training set loss function and the first test set loss function being greater than a first predetermined threshold, and in response to the training set loss function and the test set loss function being less than a second predetermined threshold, the first candidate configuration is determined as the target configuration of the large language model to be trained.

2. The large language model training method according to claim 1, wherein: The large language model to be trained includes an input layer, an output layer, and a plurality of intermediate layers between the input layer and the output layer, and wherein the number of the one or more layers frozen in each candidate configuration is less than the sum of the number of the input layer, the output layer, and the intermediate layers.

3. The large language model training method according to claim 1, wherein: For a first training set loss function and a first test set loss function corresponding to a first candidate configuration among the multiple candidate configurations, in response to a correlation between the first training set loss function and the first test set loss function being greater than a first predetermined threshold, and in response to the training set loss function and the test set loss function being less than a second predetermined threshold, determining the first candidate configuration as a target configuration of the large language model to be trained further includes: In response to the perplexity of the training set loss function and the test set loss function being less than a third predetermined threshold, the first candidate configuration is determined as the target configuration of the large language model to be trained.

4. The large language model training method according to claim 1, wherein: Determining a plurality of candidate configurations for the large language model to be trained further comprises determining the plurality of candidate configurations based on a grid search and / or a random search.

5. The large language model training method according to claim 1, wherein: The medical text data includes basic tumor medical information, tumor medical scenario production data, and cutting-edge tumor medical research information.

6. The large language model training method according to claim 1, wherein: Extracting a training data set and a test data set from the medical text data also includes: A plurality of fixed-length sentence groups with continuous sentences are obtained from the medical text data using a sliding window as training data or test data in the training data set or the test data set.

7. The large language model training method according to claim 6, further comprising: In response to the text quality of the medical text data satisfying a predetermined condition, determining that the width of the sliding window is smaller than the number of sentences in the fixed-length sentence group; as well as In response to the text quality of the medical text data not satisfying a predetermined condition, determining that the width of the sliding window is greater than or equal to the number of sentences in the fixed-length sentence group.

8. The large language model training method according to claim 7, wherein: The predetermined condition includes one or more of a source for acquiring the medical text data, a text quality indicator of the medical text data, and the number of sentences contained in the medical text data.

9. The large language model training method according to claim 1, further comprising: Based on the subject and / or feature of the medical text data, performing cluster analysis on the information in the medical text data to generate a plurality of categories; A predetermined amount of data is selected from each category of the generated multiple categories as training data or test data in the training data set or the test data set.

10. A large language model training device for the medical field, comprising: A data acquisition unit is configured to acquire medical text data and extract a training data set and a test data set from the medical text data; a candidate configuration determining unit configured to determine a plurality of candidate configurations for a large language model to be trained, wherein parameters of one or more layers of the large language model to be trained are frozen in each of the plurality of candidate configurations and each of the candidate configurations has a different frozen layer and / or number of layers; a training unit configured to, based on the same original training parameters, train the large language model to be trained using one of the multiple training data groups included in the training data set for each candidate configuration in the multiple candidate configurations to obtain a training group loss function and update parameters of unfrozen layers in the large language model to be trained, wherein the frozen layers in each candidate configuration retain the original training parameters and the parameters of the unfrozen layers in each candidate configuration are allowed to change with training; The training unit is further configured to, after updating the parameters of the unfrozen layers in the large language model to be trained, input another training data group from the multiple training data groups to train the large language model to be trained until the multiple training data groups are traversed, and determine the training set loss function based on the training group loss functions corresponding to the multiple training data groups; A testing unit, configured to, after traversing the plurality of training data groups, use a test data set to test the large language model to be trained to obtain a test set loss function; The target configuration determination unit is configured to, for a first training set loss function and a first test set loss function corresponding to a first configuration among the multiple candidate configurations, determine the first configuration as the target configuration of the large language model to be trained in response to a correlation between the first training set loss function and the first test set loss function being greater than a first predetermined threshold, and in response to the training set loss function and the test set loss function being less than a second predetermined threshold.

11. The large language model training device according to claim 10, wherein: The large language model to be trained includes an input layer, an output layer, and a plurality of intermediate layers between the input layer and the output layer, and wherein the number of the one or more layers frozen in each candidate configuration is less than the sum of the number of the input layer, the output layer, and the intermediate layers.

12. The large language model training device according to claim 10, wherein: The target configuration determining unit is further configured to: In response to the perplexity of the training set loss function and the test set loss function being less than a third predetermined threshold, the first configuration is determined as the target configuration of the large language model to be trained.

13. The large language model training device according to claim 10, wherein: The candidate configuration determining unit is further configured to determine the plurality of candidate configurations based on a grid search and / or a random search.

14. The large language model training device according to claim 10, wherein: The medical text data includes basic tumor medical information, tumor medical scenario production data, and cutting-edge tumor medical research information.

15. The large language model training device according to claim 10, wherein: The data acquisition unit is further configured to: A plurality of fixed-length sentence groups with continuous sentences are obtained from the medical text data using a sliding window as training data or test data in the training data set or the test data set.

16. The large language model training device according to claim 15, wherein: The data acquisition unit is further configured to: In response to the text quality of the medical text data satisfying a predetermined condition, determining that the width of the sliding window is smaller than the number of sentences in the fixed-length sentence group; as well as In response to the text quality of the medical text data not satisfying a predetermined condition, determining that the width of the sliding window is greater than or equal to the number of sentences in the fixed-length sentence group.

17. The large language model training device according to claim 16, wherein: The predetermined condition includes one or more of a source for acquiring the medical text data, a text quality indicator of the medical text data, and the number of sentences contained in the medical text data.

18. The large language model training device according to claim 10, wherein: The data acquisition unit is further configured to: Based on the subject and / or feature of the medical text data, performing cluster analysis on the information in the medical text data to generate a plurality of categories; A predetermined amount of data is selected from each category of the generated multiple categories as training data or test data in the training data set or the test data set.

19. An electronic device comprising: processor; as well as A memory, wherein the memory stores a computer-readable code, and when the computer-readable code is executed by the processor, the large language model training method for the medical field described in any one of claims 1 to 9 is implemented.

20. A non-transitory computer-readable storage medium storing computer-readable instructions, wherein: When the computer-readable instructions are executed by a processor, the large language model training method for the medical field described in any one of claims 1 to 9 is implemented.

Images