Large language model training method, system and device

Abstract

The invention provides a large language model training method, system and device, and relates to the technical field of electric digital processing, and the method comprises the steps: dividing a domain text training set into a plurality of training sample sets; according to the semantic change and frequency of the segmented words in each training sample set, calculating to obtain a training attention degree corresponding to each training sample set; performing maximum and minimum normalization based on all the training attention degrees to obtain a demand coefficient corresponding to each sample set; and performing multiple rounds of fine tuning training on a preset large language model through a LoRA method and all the training sample sets to obtain a target large language model, and adjusting a rank in the multiple rounds of fine tuning training by a demand coefficient. The field text and the general text are used for comparison. By analyzing the occurrence frequency and the semantic difference of the segmented words in the two texts, whether the segmented words are more used in the professional field is judged, so that the specialization degree of the training sample is determined, and the calculation resources are distributed in a targeted manner for learning, so that the calculation resource distribution accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the field of electronic digital processing technology, specifically to a method, system, and apparatus for training large language models. Background Technology

[0002] In the training and use of large language models, a general model is first obtained through pre-training. This general model has strong generalization ability. Then, the general model is fine-tuned using training sets in specific scenarios to adapt it to a particular professional field. For example, in the field of smart grids, the general model is usually fine-tuned and trained using a training set rich in power grid text, so that the model has better adaptability in power grid scenarios.

[0003] Currently, LoRA (Local Algorithm for Refining General Models) is often used for fine-tuning general models. This method approximates the effect of full-parameter fine-tuning by introducing a low-rank fine-tuning matrix into specific layers of the general model. LoRA typically applies a fixed rank in each training epoch. However, this fixed-rank approach cannot effectively utilize the rich information in the training set of the power grid. Therefore, there is an urgent need for a training method that can effectively utilize the information-rich training set. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention aims to provide a method, system, and apparatus for training large language models.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, this application provides a method for training a large language model, in one possible implementation, in conjunction with the first aspect. The method includes: acquiring domain text; preprocessing the domain text to obtain a domain text training set; dividing the domain text training set into several training sample sets, wherein the training sample sets are divided into initialization sample sets or adaptive sample sets; calculating the training attention corresponding to each training sample set based on the semantic changes and frequencies of word segmentation within each training sample set; performing maximum-minimum normalization based on all training attention to obtain the demand coefficient corresponding to each sample set; and performing multiple rounds of fine-tuning training on a preset large language model using the LoRA method and all the training sample sets to obtain a target large language model. In the multiple rounds of fine-tuning training, initialization training is first performed using the initialization sample set, and then adaptive training is performed using the adaptive sample set. The rank of the initialization training is a preset initial value, and the rank of each round of adaptive training is adjusted by the demand coefficient.

[0006] In conjunction with the first aspect, in one possible implementation, the preprocessing includes data cleaning, word segmentation, and vectorization.

[0007] In conjunction with the first aspect, in one possible implementation, the training attention corresponding to each training sample set is calculated based on the semantic changes and frequencies of the word segments within each training sample set. This includes: calculating the domain tendency based on the difference in the number of occurrences of each target word in the domain text and the general text, where the target word is a word segment corresponding to any word vector within a training sample set, and the general text is a pre-defined training dataset of the large language model; calculating the semantic consistency evaluation value based on the semantic changes of each target word in the domain text and the general text; multiplying the domain tendency and semantic consistency evaluation value of each target word to obtain the attention corresponding to each target word; and calculating the training attention by averaging the attention corresponding to each target word.

[0008] In conjunction with the first aspect, in one possible implementation, the domain tendency degree is calculated based on the difference in the frequency of occurrence of each target word in the domain text and the general text. This includes: counting the frequency of occurrence of each target word in the general text and the domain text respectively to obtain the general frequency and domain frequency corresponding to each target word; calculating the average of the general frequency and domain frequency corresponding to each target word to obtain the average general frequency corresponding to the general text and the average domain frequency corresponding to the domain text; calculating the domain performance degree based on the difference between the domain frequency and the average domain frequency of each target word; calculating the general performance degree based on the difference between the general frequency and the average general frequency of each target word; and calculating the domain trend degree based on the domain performance degree and the general performance degree.

[0009] In conjunction with the first aspect, in one possible implementation, the semantic consistency evaluation value is calculated based on the meaning changes of each target word in the domain text and the general text, including: extracting all sentence segments containing the target word to obtain at least two target sentence segments, and obtaining the positional encoding of the target word in each target sentence segment; grouping the target sentence segments according to the positional encoding to obtain at least one sentence segment group, each sentence segment group including at least two target sentence segments, and the positional encoding of the target word is the same in each sentence segment group; calculating the mean word vector of each target sentence segment in each sentence segment group; pairing the target sentence segments in each sentence segment group to obtain several sentence segment pairs; calculating the cosine similarity of the mean word vectors of the two target sentence segments in each sentence segment pair, and calculating the semantic consistency evaluation value based on the mean and normalization of all cosine similarities.

[0010] In conjunction with the first aspect, in one possible implementation, the preset initial value is calculated as follows: the preset initial value corresponding to each round of initial training is calculated based on the demand coefficient and preset rated rank corresponding to the initial sample set in each round.

[0011] In conjunction with the first aspect, in one possible implementation, the rank calculation method for adaptive training is as follows: Based on the first sample set extracted from all trained sample sets, a second sample set and several similar training sample sets are obtained. The difference in training attention between the similar training sample sets and the first sample set is less than a preset threshold. The first sample set is an adaptive sample set that has not been used for adaptive training, and the second sample set is an adaptive sample set that has been used for adaptive training. The adaptive training rounds of the first sample set and the second sample set are adjacent. A trend value is calculated based on the difference in changes between the fine-tuning matrices corresponding to the second sample set and each similar training sample set. An adjustment magnitude is calculated based on the trend value, the training attention corresponding to the second sample set and the first sample set. The training attention corresponding to the first sample set is corrected based on the adjustment magnitude to obtain the corrected training attention of the first sample set. The rank of the first sample set in adaptive training is calculated based on the corrected training attention of the first sample set and the preset rank.

[0012] In conjunction with the first aspect, in one possible implementation, the trend value is calculated based on the difference in change between the second sample set and the fine-tuning matrix corresponding to each similar training sample set, including: extracting the fine-tuning matrix corresponding to each of the similar training sample sets to obtain a set of fine-tuning matrices, and calculating the mean of a first weight parameter based on the mean of each weight parameter in the set of fine-tuning matrices; extracting the fine-tuning matrix of the second sample set, and calculating the mean of the weight parameters to obtain a second weight parameter mean; and calculating the trend value based on the mean of the first weight parameter and the mean of the second weight parameter.

[0013] Secondly, this application also provides a large language model training system, comprising: an acquisition module for acquiring domain text; a preprocessing module for preprocessing the domain text to obtain a domain text training set; a partitioning module for partitioning the domain text training set into several training sample sets, wherein the training sample sets are divided into an initialization sample set or an adaptive sample set; an attention calculation module for calculating the training attention corresponding to each training sample set based on the semantic changes and frequencies of word segmentation within each training sample set; a demand coefficient calculation module for performing maximum-minimum normalization based on all the training attention to obtain the demand coefficient corresponding to each sample set; and a fine-tuning module for performing multiple rounds of fine-tuning training on a preset large language model using the LoRA method and all the training sample sets to obtain a target large language model. In the multiple rounds of fine-tuning training, initialization training is first performed using the initialization sample set, and then adaptive training is performed using the adaptive sample set. The rank of the initialization training is a preset initial value, and the rank of each round of adaptive training is adjusted by the demand coefficient.

[0014] Secondly, this application also provides a large language model training apparatus, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the large language model training method described above.

[0015] The present invention has the following beneficial effects: This invention uses domain-specific text as a training set for a specific scenario and general text used by large language models as a comparison text. First, based on the different frequency of word segmentation in domain-specific and general texts and their semantic performance, it determines whether each word segmentation is more inclined to be used within a specialized domain. Based on this, it determines whether each training sample set is more information-rich and whether more computational resources should be invested in learning. Compared to the fixed-value method used in traditional LoRA fine-tuning, this invention improves the accuracy of computational resource allocation. Attached Figure Description

[0016] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a large language model training method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart illustrating step S4 provided in one embodiment 1 of the present invention; Figure 3 This is a flowchart illustrating step S6 provided in one embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the structure of the large language model training system described in Embodiment 2 of the present invention.

[0018] Figure 5 This is a schematic diagram of the structure of the large language model training device described in an embodiment of the present invention. Detailed Implementation

[0019] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a large language model training method, system, and apparatus proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0021] Example 1: The specific scheme of the large language model training method provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0022] Please see Figure 1 The diagram illustrates a flowchart of a large language model training method according to an embodiment of the present invention, specifically including steps S1-S6.

[0023] S1. Obtain the domain text.

[0024] It should be noted that the domain text mentioned in this embodiment refers to the text data corresponding to a specific professional field. The professional field mentioned in this step could be the power grid, railway tracks, or other industries with specialized and standardized terminology, such as energy and chemical engineering, finance, and healthcare.

[0025] When acquiring texts related to the field, it is essential to ensure the authority and diversity of the data sources. Therefore, technical specifications, academic research papers, industry analysis reports, and other textual materials in the field can be selected.

[0026] S2. Preprocess the domain text to obtain a domain text training set.

[0027] Meanwhile, to improve the subsequent fine-tuning training effect of the large language model, the preprocessing involved in this step includes data cleaning, word segmentation, and vectorization. The data cleaning in this step targets non-textual symbols (such as HTML tags, special characters, etc.) and noisy data (including garbled characters, repeated paragraphs, etc.). In the word segmentation stage, Jieba word segmentation is used to segment the text data within the domain text. Vectorization uses the Word2Vec algorithm to convert each word segment into a distributed representation of word vectors.

[0028] S3. Divide the domain text training set into several training sample sets, which are either initialization sample sets or adaptive sample sets.

[0029] It should be noted that the number of samples included in the training sample set in this embodiment can be selected by those skilled in the art based on actual circumstances to complete the partitioning operation. Furthermore, based on the partitioning result, this embodiment also divides the training sample set into an initialization sample set and an adaptive sample set for subsequent fine-tuning training. Specifically, in this embodiment, 20 samples are selected from the training sample set as the initialization sample set for subsequent initialization training, and the remaining samples are used as the adaptive sample set for subsequent adaptive training. It is understood that those skilled in the art can choose an integer greater than 20 for the number of samples in the initialization sample set. This embodiment does not impose specific limitations on this.

[0030] S4. Calculate the training attention corresponding to each training sample set based on the semantic changes and frequencies of word segmentation within each training sample set.

[0031] In this embodiment, although the domain text training set originates from technical specifications, academic research papers, industry analysis reports, and other textual materials within a specific professional field, it also involves a significant amount of conventional language expression. Therefore, in order to concentrate more computational resources on the more specialized training sample set, a higher rank is assigned to the more specialized training sample set. Thus, in this embodiment, the training focus is determined from two perspectives: semantic variation and word frequency.

[0032] For details, please refer to Figure 2 Let's take calculating the training attention of any training sample set as an example to illustrate this.

[0033] S41. Calculate the domain tendency degree based on the difference in the number of occurrences of each target word in the domain text and the general text. The target word is the word corresponding to any word vector in a training sample set, and the general text is the training dataset of the pre-set large language model.

[0034] This embodiment takes the power grid field as an example for illustration: In the power grid field, frequently occurring word segments may be professional terms in the power field or names of power equipment, such as "power factor" and "hub". Such high-frequency word segments are the focus of analysis in the training of large language models. Therefore, it is particularly important to evaluate the high-frequency performance of each word segment based on the frequency characteristics of the word segment in the corresponding set.

[0035] Specifically, this process includes steps S411-S415.

[0036] S411. Count the number of times each target word appears in the general text and the domain text respectively, and obtain the general number and domain number corresponding to each target word.

[0037] S412. Calculate the average of the general frequency and the domain frequency corresponding to each target word segmentation to obtain the average general frequency corresponding to the general text and the average domain frequency corresponding to the domain text.

[0038] S413. Calculate the domain performance degree based on the difference between the domain frequency and the average domain frequency of each target word.

[0039] Specifically, the domain performance calculation function in this step is as follows: ; in, Indicates the first training sample set Domain representation of each word segment; Represents the maximum-minimum normalization function; Indicates the first training sample set The number of times each word is in its domain; This represents the mean number of times in the training sample set.

[0040] In the above calculation function, if the training sample set... The greater the frequency of a word segment compared to the average frequency of all word segments, the higher the probability that the word segment is being used. The stronger the domain representation of each word segment.

[0041] S414. The general performance degree is calculated based on the difference between the general frequency and the average general frequency of each target word.

[0042] The calculation method for general monetization is exactly the same as the calculation method for the aforementioned domain performance.

[0043] S415. Calculate the domain trend degree based on the domain performance degree and the general performance degree.

[0044] Specifically, in this embodiment, the calculation function for the domain trend degree is as follows: ; in, Indicates the first training sample set Domain trend of each word segmentation; Indicates the first training sample set Domain trend degree of each word segment in the domain text; Indicates the first training sample set The general expressiveness of each word in general text.

[0045] In the above calculation function, this embodiment considers that word segmentation commonly used in the power grid field but less common in the general field is more likely to belong to the professional vocabulary of the power grid field. Therefore, if the frequency of certain word segmentation in the domain text is more significant in the domain text training set, but the frequency is lower in the general text of the general set, it indicates that the word segmentation is more likely to be a professional word segmentation in the power grid field.

[0046] In this embodiment, it is considered that regardless of the professional field, they all belong to the same discipline, and within the same discipline, the use and expression of professional terms are standardized and consistent. Therefore, in a single power grid field, the usage of highly specialized word segments in different sentence segments often has similarities, while the usage of less specialized word segments is more diverse. Therefore, in order to further identify the specialization of word segments in the power grid field, this embodiment also includes step S42 to analyze the consistency of contextual usage of each word segment in the training set of domain texts, as detailed in step S42.

[0047] S42. Calculate the semantic consistency evaluation value based on the meaning changes of each target word in the domain text and general text.

[0048] Specifically, this process includes steps S421 to S41.

[0049] S421. Extract all sentence segments containing the target word to obtain at least two target sentence segments, and obtain the position code of the target word in each target sentence segment.

[0050] It should be noted that the positional encoding in the steps is calculated using the Positional Encoding algorithm. Positional Encoding is an algorithm that assigns a unique position vector to each element in a sequence. These position vectors help the model understand the relative or absolute position of elements in the sequence. In this embodiment, the Positional Encoding algorithm is used to encode the position of the target word within the sentence segment, thus obtaining the positional encoding for each target word. The introduction of positional encoding enables the model to better understand the contextual information of the target word within the sentence segment, thereby improving the accuracy of analyzing the consistency of contextual usage among the sentence segments to which each word belongs in the domain text training set.

[0051] Furthermore, in this embodiment, the influence of the position encoding of word segmentation in a sentence segment on semantics is considered. For example, in the sentence segment "Current / Overload / Causes / Circuit Breaker / Trip", "Current" appears as the subject, which directly causes the "Trip" phenomenon of the "Circuit Breaker". In another sentence segment "Engineer / Needs / To Measure / Current / At / The / Circuit Breaker", "Current" appears as the object, which is the object that the "Engineer" needs to measure. In these two sentence segments, the positions of "Current" are different, resulting in significant differences in its semantics in the description. Therefore, in this embodiment, not only the word segmentation itself is concerned, but further classification and analysis are also carried out according to the specific position of each word segmentation in the sentence segment to more accurately capture and understand its semantic content. See steps S422 - S424 for details.

[0052] S422. Group the target sentence segments according to the position encoding to obtain at least one sentence segment group, and each sentence segment group includes at least two target sentence segments, and the position encodings of the target word segmentations in each sentence segment group are the same.

[0053] Through the above grouping method, in this embodiment, the error influence caused by the difference in position encoding can be better reduced by analyzing each sentence segment group separately.

[0054] In this embodiment, another problem is further considered: Even if the position encodings of the word segmentations in two sentences are exactly the same, the contextual usages represented by the word segmentations may still be different. For example, in the sentences "Spirit / Is In / Overload / State" and "In / The Circuit / Part Of / The Branch / Current / Is In / Overload / State", in the first sentence, "Overload" is used to describe a person's mental state, indicating an unbearable psychological pressure; while in the second sentence, "Overload" describes a phenomenon in the power system, referring to the current exceeding the safe range. Although the position encodings of the word "Overload" in these two sentences are the same, their contextual usages are significantly different. Therefore, in this embodiment, a method based on word vectors is adopted to analyze the contextual differences of word segmentations to more accurately understand and process natural language.

[0055] S423. Calculate the word vector mean of each target sentence segment in each sentence segment group respectively.

[0056] S424. Pair up the target sentence segments in each sentence segment group pairwise to obtain several sentence segment pairs.

[0057] It should be noted that in the following processing, due to the necessary vector calculations, the pairwise pairings described in this step actually refer to repeatable pairings. For example, the pairing between sentence segment A and sentence segment B, and the pairing between sentence segment B and sentence segment A, are both considered valid sentence segment pairs. In other words, sentence segment A can be paired with sentence segment B, and sentence segment B can also be paired with sentence segment A; these two cases are considered two distinct sentence segment pairs.

[0058] S425. Calculate the cosine similarity of the word vectors of the two target sentence segments in each sentence segment pair, and calculate the semantic consistency evaluation value based on the mean and normalization of all cosine similarities.

[0059] In this embodiment, the higher the cosine similarity of the mean word vectors of the two target sentence segments in each sentence segment pair, the more similar the contexts of the two sentence segments are. Furthermore, averaging and normalizing all cosine similarities can reflect the semantic complexity of the word segmentation. The higher the semantic consistency evaluation value, the more uniform and standardized the usage of the word segmentation is in different sentence segment pairs, and vice versa.

[0060] In this embodiment, the cosine similarity of the mean word vectors of the two target sentence segments in each pair of sentence segments can be used to quantify the contextual similarity between sentences. A higher cosine similarity value indicates that the two sentence segments are semantically close. Subsequently, the mean of all cosine similarity values ​​is calculated to obtain an overall similarity assessment. The final semantic consistency assessment value, obtained through normalization, can effectively reveal the standardization of word segmentation usage in different sentence segment pairs. A higher semantic consistency assessment value indicates that the usage of word segmentation in different sentence segment pairs is relatively uniform and standardized; a lower assessment value indicates that the usage of word segmentation in different sentence segment pairs has higher complexity and diversity.

[0061] S43. Calculate the attention level corresponding to each target word based on the domain tendency and semantic consistency evaluation value of each target word.

[0062] It should be noted that, in this embodiment, the attention level corresponding to the target word segment can be obtained by directly multiplying the domain tendency degree and semantic consistency evaluation value of the target word segment.

[0063] However, this embodiment also considers the standardized expression of specialized terms in the power grid field, which needs to be considered not only in domain-specific texts but also in general texts. Therefore, the calculation function for attention in this embodiment is as follows: ; in, Indicates the first training sample set The attention given to each word segment; Represents the maximum-minimum normalization function; Indicates the first training sample set Domain trend of each word segmentation; Indicates the first training sample set The semantic consistency evaluation value of each word segmentation; Indicates the first training sample set The semantic consistency evaluation value of each word segment is calculated in the general text. The calculation method is the same as in step S42. Therefore, it will not be repeated in this embodiment.

[0064] The above calculation function can effectively reflect the standardized expression of specialized word segments in the power grid field across two specialized domains and everyday contexts. If a single word segment has a higher semantic consistency evaluation value in the domain-specific text and a lower semantic consistency evaluation value in the general text, and also a higher domain trend, then that word segment will receive more attention during training.

[0065] S44. The training attention is obtained by averaging the attention corresponding to each target word.

[0066] Through the above steps, in this embodiment, during the fine-tuning of the preset large language model, the domain trend of each word segment can be evaluated by the frequency of its occurrence in each training sample set. Simultaneously, the semantic consistency evaluation value of each word segment is obtained by combining the contextual similarity between the sentence segments to which the word belongs. Finally, the relative differences between each subsequent training sample set are confirmed by combining the domain trend and semantic consistency evaluation values, so as to assign different ranks to different training sample sets.

[0067] In this embodiment, by performing the above steps, during the fine-tuning of the preset large language model, the frequency of word segmentation in each training sample set can be statistically analyzed to obtain the trend degree of each word segmentation in a specific domain, that is, their importance and relevance in that domain. Furthermore, this embodiment also considers the contextual similarity between the sentence segments to which the word segments belong, thereby obtaining an evaluation value for the semantic consistency of the word segments. The semantic consistency evaluation value reflects the ability of word segments to maintain the same or similar meaning in different contexts. Finally, the domain trend degree and the semantic consistency evaluation value are combined to determine the relative differences between subsequent training sample sets. Based on the obtained relative differences, different ranks can be assigned specifically, thereby achieving a more refined and personalized training strategy.

[0068] S5. Based on all the training attention, perform max-min normalization to obtain the demand coefficient corresponding to each of the sample sets.

[0069] S6. The preset large language model is fine-tuned multiple times using the LoRA method and all the training sample sets to obtain the target large language model. In the multiple rounds of fine-tuning training, the initialization sample set is used for initialization training first, and the adaptive sample set is used for adaptive training second. The rank of the initialization training is a preset initial value, and the rank of each round of adaptive training is adjusted by the demand coefficient.

[0070] In this embodiment, the total number of fine-tuning training rounds depends on the number of training sample sets, but no specific limitation is imposed. Similarly, the number of initial training rounds depends on the number of initial sample sets; as mentioned above, the initial sample set in this embodiment is 20 sets. That is, a total of 20 initial training rounds are required in this embodiment. Furthermore, in this embodiment, the rank of the initial training rounds can be either 2 or 3.

[0071] Meanwhile, those skilled in the art can calculate the preset initial value by combining the training attention obtained from the above steps. Specifically, a preset rated rank is set in advance, and the value of the preset rated rank is set to 100. Based on the demand coefficient corresponding to the initialization sample set in each round and the preset rated rank, the preset initial value corresponding to the initialization training in each round is calculated. This preset initial value is the product of the demand coefficient and the preset rated rank. It should be noted that in this embodiment, other values ​​can also be used for the preset rated rank, and no specific restrictions are made on this in this embodiment.

[0072] Furthermore, the large language model mentioned in this step can be a pre-trained BERT, LLaMA2, BLOOM, etc., and BERT is used as an example in this embodiment. In this embodiment, the LoRA fine-tuning matrix is ​​applied to the Query and Key matrices of the self-attention layer in the BERT Transformer architecture for subsequent fine-tuning training.

[0073] Furthermore, the large language model mentioned in this step can be a pre-trained typical model with a multi-layer Transformer architecture, such as BERT, LLaMA2, or BLOOM. In this embodiment, the widely used BERT-base version (12 Transformer layers, 768 hidden layer dimensions) is used as an example. Specifically, in the calculation process of the self-attention mechanism of the large language model, the Query matrix is ​​responsible for constructing the relevance weights between the target word and other words, while the Key matrix stores the feature representation of the sequence elements. Therefore, in this embodiment, the LoRA fine-tuning matrix is ​​applied to the Query and Key matrices of the self-attention layer in the BERT Transformer architecture in a parallel adaptation manner, and the rank is set using a Gaussian distribution initialization strategy for parameter injection.

[0074] First, in this embodiment, the initial training process is completed by setting initial values. Then, the rank of the adaptive training is adjusted based on the fine-tuning training already performed. See details below. Figure 3 The figure shows that it also includes steps S61-S65.

[0075] S61. Based on the first sample set, extract from all trained sample sets to obtain a second sample set and several similar training sample sets. The difference in training attention between the similar training sample sets and the first sample set is less than a preset threshold. The first sample set is an adaptive sample set that has not been used for adaptive training, and the second sample set is an adaptive sample set that has been used for adaptive training. The adaptive training rounds of the first sample set and the second sample set are adjacent.

[0076] It should be noted that the first sample set mentioned in this step refers to the training sample set that will be applied to adaptive training, i.e., the adaptive sample set. Meanwhile, the corresponding second sample set refers to the training sample set that has already been applied to fine-tuning training in the previous stage. In other words, if the 21st round of fine-tuning training is about to begin, then the first sample set will be applied to the 21st round of fine-tuning training. The second sample set corresponds to the 20th round of fine-tuning training, which can also be called the 20th round of initialization training. In this case, the already trained sample set will include all the training sample sets used in rounds 1 to 19. If the 22nd round of fine-tuning training is about to begin, then the first sample set will be applied to the 22nd round of fine-tuning training. Accordingly, the second sample set corresponds to the 21st round of fine-tuning training, which can also be called the 21st round of adaptive training. In this case, the already trained sample set will include all the training sample sets used in rounds 1 to 20.

[0077] Meanwhile, the difference in training attention between the similar training sample set and the first sample set mentioned in the above steps is less than a preset threshold, which means that the difference in training attention between the similar training sample set and the first sample set is not greater than 10% of the training attention corresponding to the first sample set.

[0078] S62. The trend value is calculated based on the difference in change between the second sample set and the fine-tuning matrix corresponding to each similar training sample set.

[0079] Specifically, in this embodiment, the trend value is calculated as follows: S621. Extract the fine-tuning matrix corresponding to each of the similar training sample sets to obtain a set of fine-tuning matrices, and calculate the mean of the first weight parameter based on the mean of each weight parameter in the set of fine-tuning matrices.

[0080] It should be noted that the fine-tuning matrices mentioned in this step refer to the fine-tuning matrix A and fine-tuning matrix B of the LoRA method. The set of fine-tuning matrices consists of the fine-tuning matrices A and B after fine-tuning training on all similar training sample sets.

[0081] S622. Extract the fine-tuning matrix of the second sample set and calculate the mean of the weight parameters to obtain the mean of the second weight parameters.

[0082] S623. The trend value is calculated based on the mean of the first weight parameter and the mean of the second weight parameter.

[0083] Specifically, the trend value calculation function in this step is as follows: ; in, Indicates the The trend value corresponding to each round of fine-tuning training; Represents the maximum-minimum normalization function; Indicates the The mean of the second weight parameter after each round of fine-tuning training; Indicates proceeding to the first The mean of the first weight parameter calculated before each round of fine-tuning training.

[0084] In this embodiment, if the first The higher the mean value of the weight parameters in each round of fine-tuning training compared to previous rounds with similar levels of attention, the better. The higher the degree of tendency of each round towards specialized vocabulary, the better the large language model performs in the [number] round. The closer the training trends for specialized word segmentation are in each round, the more suitable the rank becomes. The higher the confidence level of the round rank, the better the analysis process described above can be.

[0085] S63. The adjustment range is calculated based on the trend value, the training attention corresponding to the second sample set and the first sample set.

[0086] Specifically, the adjustment amplitude calculation function in this step is as follows: ; in, Indicates the The adjustment range during rounds of fine-tuning training; Represents the maximum-minimum normalization function; Indicates the The training attention in each round of fine-tuning training, i.e. the training attention corresponding to the first sample set; Indicates the The training attention corresponding to each round of fine-tuning training, and the training attention corresponding to the second sample set; Indicates the The trend value corresponding to each round of fine-tuning training.

[0087] In the above calculation function, Reflects the first The difference in the required rank of the fine-tuning matrix between each round of fine-tuning training compared to the previous round is significant. The greater the difference in requirements between two adjacent rounds of fine-tuning training, the higher the required rank becomes. The more rounds of fine-tuning training are done, the more adjustments should be made; at the same time... This reflects that if the trend value of the previous round is lower, the confidence of the rank of the previous round is lower. Therefore, in the fine-tuning training of the y-th round, the adjustment range should be reduced compared to the previous round to reduce the consumption of unnecessary computing resources caused by the unreasonable increase of rank.

[0088] S64. Based on the adjustment range, the training attention corresponding to the first sample set is corrected to obtain the corrected training attention of the first sample set.

[0089] Specifically, the adjustment amplitude calculation function in this step is as follows: ; in, Indicates the Training focus after fine-tuning training rounds. Indicates the The training attention in each round of fine-tuning training, i.e. the training attention corresponding to the first sample set; Indicates the The training attention corresponding to each round of fine-tuning training, and the training attention corresponding to the second sample set; Indicates the The range of adjustments during each round of fine-tuning training.

[0090] In the above calculation function, by comparing the first... Rounds of fine-tuning training and the first The differences in training focus between different rounds of fine-tuning serve as the basis for dynamically adjusting the adjustment range, thereby achieving adaptive focus updates based on historical training states. When the... The training attention in each round is greater than or equal to that in the first round. In the round, it indicates the number of rounds. When fine-tuning training on the training sample set in each round, more resources should be allocated to train the large language model, thereby improving the model training effect. Therefore, a positive number is used. To amplify; conversely, to reduce by using negative numbers - Suppression is performed. This differential dynamic adjustment method can effectively balance the sample weight distribution at different training stages, prevent the waste of resources by constantly adjusting the direction of the large rank, and at the same time avoid model overfitting or getting trapped in local optima, thus helping the model to learn and generalize better in subsequent training processes.

[0091] S65. Calculate the rank of the first sample set in adaptive training based on the corrected training attention and the preset rated rank of the first sample set.

[0092] In this embodiment, the preset nominal rank is 100 as mentioned above. Other values ​​can also be used for the preset nominal rank in this embodiment. The product of the corrected training attention of the first sample set and the preset nominal rank is calculated, and this product is used as the rank of the first sample set in adaptive training.

[0093] In this embodiment, domain-specific text is used as the training set, while general text used by the large language model is used as the comparison set. First, word segmentation is used to analyze the frequency differences of words in the domain-specific and general texts to determine whether these words are more likely to be used within a specific professional domain. Simultaneously, the semantic changes of each word segment within the sentences of the domain-specific and general texts are combined, and the semantic consistency is calculated to comprehensively evaluate whether each word segment is specifically used in that domain. Based on these analyses, the training attention of each training sample set is further confirmed, ensuring that the large model can allocate more learning resources to these highly relevant samples in each round of fine-tuning training. Furthermore, during LoRA fine-tuning training, the rank of each round of fine-tuning training is dynamically adjusted based on the parameter changes within the fine-tuning matrix and the training attention. Compared to the traditional method of using a fixed rank value in LoRA fine-tuning, this method significantly improves the accuracy of computational resource allocation, thereby optimizing the model's training performance.

[0094] Example 2: like Figure 4 As shown, this embodiment provides a large language model training system, the system comprising: The acquisition module is used to acquire domain text.

[0095] The preprocessing module is used to preprocess the domain text to obtain a domain text training set.

[0096] The partitioning module is used to divide the domain text training set into several training sample sets, which are divided into an initialization sample set or an adaptive sample set.

[0097] The attention calculation module is used to calculate the training attention corresponding to each training sample set based on the semantic changes and frequencies of word segmentation within each training sample set.

[0098] The demand coefficient calculation module is used to obtain the demand coefficient corresponding to each of the sample sets by performing maximum and minimum normalization based on all the training attention.

[0099] The fine-tuning module is used to perform multiple rounds of fine-tuning training on the preset large language model using the LoRA method and all the training sample sets to obtain the target large language model. In the multiple rounds of fine-tuning training, initial training is first performed using the initialization sample set, and then adaptive training is performed using the adaptive sample set. The rank of the initial training is a preset initial value, and the rank of each round of adaptive training is adjusted by the demand coefficient.

[0100] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0101] Example 3: Corresponding to the above method embodiments, this embodiment also provides a large language model training device. The large language model training device described below and the large language model training method described above can be referred to each other.

[0102] Figure 5 This is a block diagram illustrating a large language model training device 800 according to an exemplary embodiment. Figure 5 As shown, the large language model training device 800 may include a processor 801 and a memory 802. The large language model training device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0103] The processor 801 controls the overall operation of the large language model training device 800 to complete all or part of the steps in the aforementioned large language model training method. The memory 802 stores various types of data to support the operation of the large language model training device 800. This data may include, for example, instructions for any application or method operating on the large language model training device 800, and application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 802 or transmitted via communication component 805. The audio component also includes at least one speaker for outputting audio signals. I / O interface 804 provides an interface between processor 801 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 805 is used for wired or wireless communication between the large language model training device 800 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 805 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0104] In an exemplary embodiment, the large language model training device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the large language model training method described above.

[0105] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the large language model training method described above. For example, the computer-readable storage medium may be the memory 802 including program instructions, which may be executed by the processor 801 of the large language model training apparatus 800 to complete the large language model training method described above.

[0106] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A method for training a large language model, characterized in that, The method comprises: Retrieve domain text; The domain text is preprocessed to obtain a domain text training set; The domain text training set is divided into several training sample sets, which are either initialization sample sets or adaptive sample sets. The training attention level corresponding to each training sample set is calculated based on the semantic changes and frequencies of word segmentation within each training sample set. The demand coefficient for each sample set is obtained by performing max-min normalization based on all the training attention; The target large language model is obtained by performing multiple rounds of fine-tuning training on the preset large language model using the LoRA method and all the training sample sets. In the multiple rounds of fine-tuning training, the initialization training is performed first using the initialization sample set, and then the adaptive training is performed using the adaptive sample set. The rank of the initialization training is a preset initial value, and the rank of each round of adaptive training is adjusted by the demand coefficient.

2. The large language model training method according to claim 1, characterized in that, The preprocessing includes data cleaning, word segmentation, and vectorization.

3. The large language model training method according to claim 1, characterized in that, The training attention level for each training sample set is calculated based on the semantic changes and frequencies of word segmentation within each training sample set, including: The domain tendency is calculated based on the difference in the number of occurrences of each target word in the domain text and the general text. The target word is the word corresponding to any word vector in a training sample set, and the general text is the training dataset of the pre-set large language model. A semantic consistency evaluation value is calculated based on the meaning changes of each target word in the domain text and the general text. The attention level corresponding to each target word is calculated by multiplying its domain tendency and semantic consistency evaluation value. The training attention is calculated by averaging the attention corresponding to each target word.

4. The large language model training method according to claim 3, characterized in that, The domain tendency is calculated based on the difference in the frequency of each target word in the domain text and the general text, including: The occurrence count of each target word in the general text and the domain text is counted separately to obtain the general occurrence count and the domain occurrence count for each target word; The average values ​​of the general frequency and the domain frequency corresponding to each target word are calculated separately to obtain the average general frequency corresponding to the general text and the average domain frequency corresponding to the domain text. The domain performance degree is calculated based on the difference between the domain frequency and the mean domain frequency for each target word segmentation. The general performance degree is calculated based on the difference between the general frequency and the mean general frequency of each target word segmentation. The domain trend degree is calculated based on the domain performance degree and the general performance degree.

5. The large language model training method according to claim 3, characterized in that, A semantic consistency evaluation value is calculated based on the meaning changes of each target word in the domain text and the general text, including: Extract all sentence segments containing the target word to obtain at least two target sentence segments, and obtain the positional encoding of the target word in each target sentence segment; The target sentence segments are grouped according to the positional encoding to obtain at least one sentence segment group. Each sentence segment group includes at least two target sentence segments, and the positional encoding of the target word segment is the same in each sentence segment group. Calculate the mean word vector for each target sentence segment within each sentence segment group; Pair the target segments within each of the aforementioned segment groups to obtain several segment pairs; The cosine similarity of the word vectors of the two target sentence segments in each sentence segment pair is calculated separately, and the semantic consistency evaluation value is obtained by averaging and normalizing all cosine similarities.

6. The large language model training method according to claim 1, characterized in that, The method for calculating the preset initial value is as follows: The preset initial value corresponding to each round of initial training is calculated based on the demand coefficient and preset rated rank corresponding to the initial sample set in each round.

7. The large language model training method according to claim 1, characterized in that, The rank is calculated as follows in adaptive training: Based on the first sample set extracted from all trained sample sets, a second sample set and several similar training sample sets are obtained. The difference in training attention between the similar training sample sets and the first sample set is less than a preset threshold. The first sample set is an adaptive sample set that has not been used for adaptive training, and the second sample set is an adaptive sample set that has been used for adaptive training. The adaptive training rounds of the first sample set and the second sample set are adjacent. The trend value is calculated based on the difference in change between the second sample set and the fine-tuning matrix corresponding to each similar training sample set. The adjustment range is calculated based on the trend degree value, the training attention corresponding to the second sample set and the first sample set; Based on the adjustment range, the training attention corresponding to the first sample set is corrected to obtain the corrected training attention of the first sample set. The rank of the first sample set during adaptive training is calculated based on the corrected training attention and the preset rank of the first sample set.

8. The large language model training method according to claim 7, characterized in that, The trend value is calculated based on the difference in changes between the second sample set and the fine-tuning matrix corresponding to each similar training sample set, including: Extract the fine-tuning matrix corresponding to each of the similar training sample sets to obtain a set of fine-tuning matrices, and calculate the mean of the first weight parameter based on the mean of each weight parameter in the set of fine-tuning matrices. Extract the fine-tuning matrix of the second sample set and calculate the mean of the weight parameters to obtain the mean of the second weight parameters; The trend value is calculated based on the mean of the first weight parameter and the mean of the second weight parameter.

9. A large language model training system, characterized in that, include: The acquisition module is used to acquire domain text; The preprocessing module is used to preprocess the domain text to obtain a domain text training set; The partitioning module is used to divide the domain text training set into several training sample sets, wherein the training sample sets are divided into an initialization sample set or an adaptive sample set; The attention calculation module is used to calculate the training attention corresponding to each training sample set based on the semantic changes and frequencies of word segmentation within each training sample set. The demand coefficient calculation module is used to obtain the demand coefficient corresponding to each of the sample sets by performing maximum and minimum normalization based on all the training attention; The fine-tuning module is used to perform multiple rounds of fine-tuning training on the preset large language model using the LoRA method and all the training sample sets to obtain the target large language model. In the multiple rounds of fine-tuning training, initial training is first performed using the initialization sample set, and then adaptive training is performed using the adaptive sample set. The rank of the initial training is a preset initial value, and the rank of each round of adaptive training is adjusted by the demand coefficient.

10. A large language model training device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the large language model training method as described in any one of claims 1 to 8.

Images