A large language model training method, system and device

Abstract

The application provides a large language model training method, system and device, and relates to the technical field of electric digital processing, which comprises the following steps: dividing a domain text training set into several training sample sets; calculating the training attention degree corresponding to each training sample set according to the semantic change and frequency of word segmentation in each training sample set; performing maximum minimum normalization based on all training attention degrees to obtain the demand coefficient corresponding to each sample set; and performing multi-round fine-tuning training on a preset large language model through the LoRA method and all training sample sets to obtain a target large language model, wherein the rank in the multi-round fine-tuning training is adjusted by the demand coefficient. The application uses domain text and general text as a comparison. By analyzing the appearance frequency and semantic difference of word segmentation in the two kinds of texts, it is determined whether word segmentation is more used in a professional field, so as to determine the professional degree of the training sample, and learning by targeted allocation of computing resources can improve the accuracy of computing resource allocation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital processing, in particular to a large language model training method, system and device. BACKGROUND

[0002] In the training and use process of a large language model, a general model is first obtained through pre-training, the general model has strong generalization ability, and then the general model is fine-tuned through a training set in a specific scene to adapt to a certain professional field. In the field of smart power grids, the general model is usually fine-tuned and trained through a text training set rich in power grids, so that the model has better adaptability in the power grid scene.

[0003] At present, the LoRA method is often used when fine-tuning the general model. The method approximates the effect of full-parameter fine-tuning by introducing a low-rank fine-tuning matrix in a specific layer of the general model. However, in the LoRA method, a fixed rank is usually applied in each training round. However, the above-mentioned fixed rank application method cannot effectively utilize the data of the text training set rich in power grids. Therefore, there is an urgent need for a training method that can effectively utilize the information-rich training set. SUMMARY

[0004] To solve the above technical problems, the purpose of the present application is to provide a large language model training method, system and device.

[0005] To achieve the above-mentioned purpose, the technical solutions adopted by the present application are as follows: In a possible implementation manner, the present application provides a large language model training method, which comprises: obtaining a domain text; preprocessing the domain text to obtain a domain text training set; dividing the domain text training set into a plurality of training sample sets, the training sample sets being divided into initialization sample sets or adaptive sample sets; calculating the training attention degree corresponding to each training sample set according to the semantic changes and frequencies of the word segmentation in each training sample set; performing maximum and minimum normalization based on all the training attention degrees to obtain a demand coefficient corresponding to each sample set; performing multi-round fine-tuning training on a preset large language model through the LoRA method and all the training sample sets to obtain a target large language model, wherein the initialization training is first performed using the initialization sample sets, and then the adaptive training is performed using the adaptive sample sets, the rank of the initialization training is a preset initial value, and the rank of each round of adaptive training is adjusted by the demand coefficient.

[0006] In a possible implementation manner, the preprocessing comprises data cleaning processing, word segmentation processing and vectorization processing.

[0007] In a possible implementation manner of the first aspect, the training attention degree corresponding to each training sample set is calculated according to semantic changes and frequencies of word segmentation in each training sample set, including: calculating a field trend degree according to a difference between occurrence frequencies of each target word segmentation in the field text and the general text, the target word segmentation being a word segmentation corresponding to any word vector in a training sample set, and the general text being a preset training data set of the large language model; calculating a semantic consistency evaluation value according to a semantic change of each target word segmentation in the field text and the general text; multiplying the field trend degree and the semantic consistency evaluation value of each target word segmentation to obtain an attention degree corresponding to each target word segmentation; and calculating a training attention degree based on the attention degrees corresponding to each target word segmentation.

[0008] In a possible implementation manner of the first aspect, the field trend degree is calculated according to a difference between occurrence frequencies of each target word segmentation in the field text and the general text, including: respectively counting occurrence frequencies of each target word segmentation in the general text and the field text to obtain a general frequency corresponding to each target word segmentation and a field frequency corresponding to each target word segmentation; respectively calculating a general frequency average corresponding to the general text and a field frequency average corresponding to the field text based on the general frequencies corresponding to each target word segmentation and the field frequencies corresponding to each target word segmentation; calculating a field performance degree based on a difference between the field frequency of each target word segmentation and the field frequency average; calculating a general performance degree based on a difference between the general frequency of each target word segmentation and the general frequency average; and calculating the field trend degree based on the field performance degree and the general performance degree.

[0009] In a possible implementation manner of the first aspect, the semantic consistency evaluation value is calculated according to a semantic change of each target word segmentation in the field text and the general text, including: extracting all sentence segments containing the target word segmentation to obtain at least two target sentence segments, and obtaining a position code of the target word segmentation in each target sentence segment; grouping the target sentence segments according to the position code to obtain at least one sentence group, each sentence group including at least two target sentence segments, and the position codes of the target word segment in each sentence group being the same; calculating a word vector average of each target sentence segment in each sentence group; pairing the target sentence segments in each sentence group two by two to obtain a plurality of sentence pairs; calculating a cosine similarity of the word vector averages of the two target sentence segments in each sentence pair, and calculating a semantic consistency evaluation value based on all cosine similarities.

[0010] In a possible implementation manner of the first aspect, the preset initial value is calculated based on a demand coefficient corresponding to each round of the initialization sample set and a preset rated rank.

[0011] In a possible implementation manner of the first aspect, the rank calculation method in the adaptive training is as follows: a second sample set and a plurality of similar training sample sets are extracted according to the first sample set in all trained sample sets, the similar training sample sets have a training attention difference less than a preset threshold with the first sample set, the first sample set is an adaptive sample set that has not been used for adaptive training, the second sample set is an adaptive sample set that has been used for adaptive training, and the adaptive training rounds of the first sample set and the second sample set are adjacent; a trend value is calculated according to the change difference between the fine-tuning matrix corresponding to the second sample set and each similar training sample set; an adjustment amplitude is calculated based on the trend value, the training attention corresponding to the second sample set, and the training attention corresponding to the first sample set; the training attention corresponding to the first sample set is modified based on the adjustment amplitude to obtain the training attention of the first sample set after modification; and the rank of the adaptive training of the first sample set is calculated based on the training attention of the first sample set after modification and a preset rated rank.

[0012] In a possible implementation manner of the first aspect, the trend value is calculated according to the change difference between the fine-tuning matrix corresponding to the second sample set and each similar training sample set, including: the fine-tuning matrix corresponding to each similar training sample set is extracted respectively to obtain a fine-tuning matrix set, and a first weight parameter mean value is calculated based on each weight parameter in the fine-tuning matrix set; the fine-tuning matrix of the second sample set is extracted, and a second weight parameter mean value is calculated based on the weight parameters; and the trend value is calculated based on the first weight parameter mean value and the second weight parameter mean value.

[0013] In a possible implementation manner of the first aspect, the trend value is calculated according to the change difference between the fine-tuning matrix corresponding to the second sample set and each similar training sample set, including: the fine-tuning matrix corresponding to each similar training sample set is extracted respectively to obtain a fine-tuning matrix set, and a first weight parameter mean value is calculated based on each weight parameter in the fine-tuning matrix set; the fine-tuning matrix of the second sample set is extracted, and a second weight parameter mean value is calculated based on the weight parameters; and the trend value is calculated based on the first weight parameter mean value and the second weight parameter mean value. In a possible implementation manner of the first aspect, the trend value is calculated according to the change difference between the fine-tuning matrix corresponding to the second sample set and each similar training sample set, including: the fine-tuning matrix corresponding to each similar training sample set is extracted respectively to obtain a fine-tuning matrix set, and a first weight parameter mean value is calculated based on each weight parameter in the fine-tuning matrix set; the fine-tuning matrix of the second sample set is extracted, and a second weight parameter mean value is calculated based on the weight parameters; and the trend value is calculated based on the first weight parameter mean value and the second weight parameter mean value.

[0014] In a second aspect, the present application also provides a large language model training device, comprising: a memory for storing a computer program; and a processor for implementing the steps of the large language model training method when executing the computer program.

[0015] The present application has the following advantages:

[0016] The present application uses domain text as a specific scenario training set, and uses general text used by a large language model as a comparison text. First, according to the different occurrence times and semantic expressions of the domain text and the general text, it is confirmed whether each word is more inclined to be used in a professional field. Based on this, it is confirmed whether each training sample set is more professional and whether more computing resources should be allocated for learning. Compared with the method of using a fixed value in the traditional LoRA fine-tuning process, the present application improves the accuracy of computing resource allocation. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0018] Figure 1 A flowchart of a large language model training method provided by an embodiment 1 of the present application is shown in the figure.

[0019] Figure 2 A flowchart of step S4 provided by an embodiment 1 of the present application is shown in the figure.

[0020] Figure 3 A flowchart of step S6 provided by an embodiment 1 of the present application is shown in the figure.

[0021] Figure 4 A large language model training system structure diagram described in an embodiment 2 of the present application is shown in the figure.

[0022] Figure 5 A large language model training device structure diagram described in an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0023] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined inventive purpose, the following describes in detail the specific implementation, structure, features and effects of a large language model training method, system and device according to the present application, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0025] Embodiment 1:

[0026] The specific scheme of the large language model training method provided by the present application is described in detail below in combination with the accompanying drawings.

[0027] Please refer to Figure 1 which shows a flowchart of a large language model training method according to an embodiment of the present application, specifically comprising steps S1-S6.

[0028] S1, obtaining domain text.

[0029] It should be noted that the domain text mentioned in this embodiment is the corresponding text data of a certain professional field. The certain professional field mentioned in this step can be power grid, railway track, or energy chemical industry, finance and medical industry, etc. which have professional and terminological specifications.

[0030] Among them, the acquisition of domain text needs to ensure the authority and diversity of data sources. Therefore, technical specification documents, academic research papers, industry analysis reports and other text materials in this field can be selected.

[0031] S2, preprocessing the domain text to obtain a domain text training set.

[0032] Meanwhile, in order to improve the subsequent large language model fine-tuning training effect, the preprocessing involved in this step includes data cleaning processing, word segmentation processing and vectorization processing. The objects of data cleaning involved in this step are non-text symbols (such as HTML tags, special symbols, etc.) and noise data (including garbled code, repeated paragraphs, etc.). In the word segmentation processing stage, each text data in the domain text is segmented by using the jieba word segmentation. The vectorization processing converts each word into a distributed representation of word vectors by using the Word2Vec algorithm.

[0033] S3, dividing the domain text training set into several training sample sets, the training sample sets being divided into initialization sample sets or adaptive sample sets.

[0034] It should be noted that the number of samples contained in the training sample set in the embodiment can be selected by those skilled in the art according to the actual situation to complete the division operation. At the same time, based on the division result, the training sample set is divided into an initialization sample set and an adaptive sample set in the embodiment for subsequent fine-tuning training. Specifically, in the embodiment, 20 are selected from the training sample set as the initialization sample set and are used for subsequent initialization training, and the rest are used as the adaptive sample set for subsequent adaptive training. It can be understood that the number of initialization sample sets can be selected by those skilled in the art to be an integer greater than 20. The embodiment does not make specific restrictions on this.

[0035] S4, calculating the training attention degree corresponding to each training sample set according to the semantic change and frequency of the word segmentation in each training sample set.

[0036] In the embodiment, it is considered that the domain text training set, although its source is a professional technical specification document, an academic research paper, an industry analysis report and the like, also involves a lot of regular language expressions. Therefore, in the embodiment, in order to concentrate more computing resources in the training sample set rich in more professionalism, the training sample set rich in more professionalism is given a larger rank. Therefore, in the embodiment, the training attention degree is determined from the two aspects of semantic change and word frequency.

[0037] For details, please refer to Figure 2 , which is used as an example to illustrate the calculation of the training attention degree of any training sample set.

[0038] S41, calculating the domain trend degree according to the difference between the number of occurrences of each target word segmentation in the domain text and the general text, the target word segmentation being a word segmentation corresponding to any word vector in the training sample set, and the general text being a preset training data set of the large language model.

[0039] In the embodiment, the power grid field is taken as an example for illustration: in the power grid field, high-frequency word segmentation may be power field professional terms or power equipment names, such as "power factor", "hub" and the like. Such high-frequency word segmentation is an important analysis word in the training of the large language model, so it is particularly important to evaluate the high-frequency performance of each word segmentation according to the frequency characteristics of the word segmentation in the corresponding set.

[0040] Specifically, the step includes steps S411-S415.

[0041] S411, respectively counting the number of occurrences of each target word segmentation in the general text and the domain text to obtain the general number and the domain number corresponding to each target word segmentation.

[0042] S412. Calculate the average of the general frequency and the domain frequency corresponding to each target word segmentation to obtain the average general frequency corresponding to the general text and the average domain frequency corresponding to the domain text.

[0043] S413. Calculate the domain performance degree based on the difference between the domain frequency and the average domain frequency of each target word.

[0044] Specifically, the domain performance calculation function in this step is as follows:

[0045] ;

[0046] in, Indicates the first training sample set Domain representation of each word segment; Represents the maximum-minimum normalization function; Indicates the first training sample set The number of domains for each word segmentation; This represents the mean number of times in the training sample set.

[0047] In the above calculation function, if the training sample set... The higher the frequency of a word segment compared to the average frequency of all word segments, the higher the probability that the word segment is being used. The stronger the domain representation of each word segment.

[0048] S414. The general performance is calculated based on the difference between the general frequency and the mean general frequency of each target word.

[0049] The calculation method for general monetization is exactly the same as the calculation method for the aforementioned domain performance.

[0050] S415. Calculate the domain trend degree based on the domain performance degree and the general performance degree.

[0051] Specifically, in this embodiment, the calculation function for the domain trend degree is as follows:

[0052] ;

[0053] in, Indicates the first training sample set Domain trend of each word segmentation; Indicates the first training sample set Domain trend degree of each word segment in the domain text; Indicates the first training sample set The general expressiveness of each word in general text.

[0054] In the above calculation function, in the embodiment, the segmentation considered in the field of power grid is commonly used in the field of power grid, but not commonly used in the general field, which is more likely to belong to the professional vocabulary of the power grid field. Therefore, if some segmentation in the field text is more significant in the high frequency degree of the field text training set, and the high frequency degree in the general set of general text is lower, it means that the segmentation is more likely to be a professional segmentation in the power grid field.

[0055] In the embodiment, it is considered that no matter what professional field, it belongs to the same discipline, and in the same discipline, the use and expression of the same professional vocabulary have normativity and consistency. Therefore, the use of a single high-professional segmentation in the power grid field in different sentence segments often has similarities, while the use of low-professional segmentation is more diversified. Therefore, in order to further identify the professionalism of the segmentation in the power grid field, the embodiment also includes step S42 of analyzing the context usage consistency of each segmentation in the sentence segment of the field text training set, which will be described in detail in step S42.

[0056] S42, calculating a semantic consistency evaluation value according to the meaning change of each target segmentation in the field text and the general text.

[0057] Specifically, the step includes step S421 to step S41.

[0058] S421, extracting all sentence segments containing the target segmentation to obtain at least two target sentence segments, and obtaining the position encoding of the target segmentation in each target sentence segment.

[0059] It should be noted that the position encoding in the step is calculated by the Positional Encoding position encoding algorithm, which is an algorithm for assigning a unique position vector to each element in a sequence. These position vectors can help the model understand the relative or absolute position of the element in the sequence. In the embodiment, the Positional Encoding position encoding algorithm is used to encode the position of the target segmentation in the sentence segment, thereby obtaining the position encoding of each target segmentation. The introduction of position encoding enables the model to better understand the context information of the target segmentation in the sentence segment, thereby improving the accuracy of the analysis of the context usage consistency of each segmentation in the sentence segment of the field text training set.

[0060] Further, in the present embodiment, the influence of the position of the word in the sentence on the semantics is considered. For example, in the sentence "current / overload / cause / circuit / trip", "current" appears as the subject, which directly causes the "trip" of the "circuit". In another sentence "engineer / need / measure / circuit / at / the / current", "current" appears as the object, which is the object that the "engineer" needs to measure. In these two sentences, the position of "current" is different, resulting in a significant difference in its semantics. Therefore, in the present embodiment, not only the word itself is considered, but also the specific position of each word in the sentence is further classified and analyzed to more accurately capture and understand its semantic content, as described in detail in steps S422-S424.

[0061] S422, grouping the target sentences according to the position coding, obtaining at least one sentence group, each of the sentence groups including at least two target sentences, and the position coding of the target words in each of the sentence groups being the same.

[0062] Through the above grouping method, the present embodiment can better reduce the error influence of the position coding difference by separately analyzing each sentence group.

[0063] In the present embodiment, the following problem is further considered: even if the position coding of the words in two sentences is exactly the same, the context usage of the words may still be different. For example, the sentences "spirit / in / an / overload / state" and "circuit / in / part / branch / current / in / an / overload / state", in the first sentence, "overload" is used to describe the mental state of a person, indicating a psychological pressure that is difficult to bear; in the second sentence, "overload" describes a phenomenon in the power system, which means that the current exceeds the safe range. Although the position coding of the word "overload" in these two sentences is the same, the context usage of the word is obviously different. Therefore, in the present embodiment, a word vector-based method is adopted to analyze the context difference of the words, so as to more accurately understand and process natural language.

[0064] S423, calculating the word vector mean of each target sentence in each sentence group.

[0065] S424, pairing the target sentences in each of the sentence groups two by two, obtaining a plurality of sentence pairs.

[0066] It should be noted that in the following processing process, since it involves the vector calculation that must be performed, the two-by-two pairing described in this step actually refers to a pairing that can be repeated. For example: the pairing between sentence segment A and sentence segment B, and the pairing between sentence segment B and sentence segment A, both of which are considered valid sentence pairings. In other words, sentence segment A can be paired with sentence segment B, and sentence segment B can also be paired with sentence segment A, both of which are considered two different sentence pairs.

[0067] S425, respectively calculate the cosine similarity of the word vector mean of the two target sentence segments in each sentence segment pair, and calculate the semantic consistency evaluation value based on the mean and normalization of all cosine similarities.

[0068] In this embodiment, the greater the cosine similarity of the word vector mean of the two target sentence segments in each sentence segment pair, the more similar the context of the two sentence segments. Further, the mean and normalization of all cosine similarities can reflect the semantic complexity of the segmented word, and the greater the semantic consistency evaluation value, the more uniform the use of the segmented word in different sentence pairs, and vice versa.

[0069] In this embodiment, by calculating the cosine similarity of the word vector mean of the two target sentence segments in each sentence segment pair, the similarity of the context between sentences can be quantified. A higher cosine similarity value indicates that the two sentence segments have a higher degree of semantic proximity. Subsequently, the mean of all cosine similarity values is calculated to obtain an overall similarity evaluation. The semantic consistency evaluation value obtained through normalization processing can effectively reveal the use of the segmented word in different sentence pairs. If the semantic consistency evaluation value is high, it indicates that the use of the segmented word in different sentence pairs is uniform and standardized; if the evaluation value is low, it indicates that the use of the segmented word has high complexity and diversity in different sentence pairs.

[0070] S43, calculating the attention degree corresponding to each target segmented word according to the field trend degree and the semantic consistency evaluation value of each target segmented word.

[0071] It should be noted that in this embodiment, the attention degree corresponding to the target segmented word can be obtained by simply multiplying the field trend degree and the semantic consistency evaluation value of the target segmented word.

[0072] However, in this embodiment, the standardized expression of the professional segmented word in the power grid field is also considered, not only in the field text but also in the general text. Therefore, in this embodiment, the calculation function of the attention degree is as follows:

[0073] ;

[0074] wherein, denotes the attention degree of the i-th wordpiece in the training sample set; denotes the max-min normalization function; denotes the max-min normalization function; denotes the domain trend degree of the i-th wordpiece in the training sample set; denotes the semantic consistency evaluation value of the i-th wordpiece in the training sample set; denotes the semantic consistency evaluation value of the i-th wordpiece in the training sample set; denotes the semantic consistency evaluation value of the i-th wordpiece in the training sample set; denotes the semantic consistency evaluation value of the i-th wordpiece in the training sample set; denotes the semantic consistency evaluation value of the i-th wordpiece in the training sample set;

[0075] In the above calculation function, the calculation can better reflect the normative expression of the professional word segmentation in the power grid field in the two professional fields and the daily field. If the semantic consistency evaluation value of a single wordpiece in the domain text is larger, and the corresponding semantic consistency evaluation value in the general text is smaller, and the domain trend degree of the wordpiece is larger, then the attention degree of the wordpiece in the training is larger.

[0076] S44, based on the attention degree corresponding to each target wordpiece, the mean value is calculated to obtain the training attention degree.

[0077] Through the above steps, in this embodiment, during the process of fine-tuning the pre-set large language model, the domain trend degree of each wordpiece is evaluated by the number of occurrences of the wordpiece in each training sample set, and the semantic consistency evaluation value of each wordpiece is obtained by combining the context similarity between the sentences to which the wordpiece belongs. Finally, the relative difference between each subsequent training sample set is finally confirmed by combining the domain trend degree and the semantic consistency evaluation value. In order to allocate different ranks to different training sample sets.

[0078] In this embodiment, by performing the above steps, during the process of fine-tuning the pre-set large language model, the frequency of the wordpiece in each training sample set is counted and analyzed to obtain the trend degree of each wordpiece in a specific domain, i.e. their importance and relevance in that domain. In addition, the context similarity between the sentences to which the wordpiece belongs is also considered in this embodiment, and then an evaluation value of the semantic consistency of the wordpiece is obtained. The semantic consistency evaluation value reflects the ability of the wordpiece to maintain the same or similar meaning in different contexts. Finally, the domain trend degree and the semantic consistency evaluation value are combined to determine the relative difference between each subsequent training sample set. Finally, according to the obtained relative difference, different ranks can be allocated accordingly, so as to realize a more refined and personalized training strategy.

[0079] S5, based on all the training attention degrees, the maximum and minimum normalization is performed to obtain the demand coefficient corresponding to each sample set.

[0080] S6, performing multi-round fine-tuning training on the preset large language model through the LoRA method and all the training sample sets, to obtain a target large language model, wherein the multi-round fine-tuning training first performs initialization training using an initialization sample set, and then performs adaptive training using an adaptive sample set, the rank of the initialization training is a preset initial value, and the rank of each round of the adaptive training is adjusted by a demand coefficient to obtain.

[0081] In the embodiment, the total number of rounds of fine-tuning training is determined by the number of training sample sets, which is not specifically limited in the embodiment. Meanwhile, the number of rounds of initialization training is determined by the number of initialization sample sets, which is 20 in the embodiment as described above. That is, 20 rounds of initialization training are required in the embodiment. Meanwhile, in the embodiment, the rank of the initialization training can be 2 or 3.

[0082] Meanwhile, the preset initial value can be calculated by the training attention degree calculated in the above steps for those skilled in the art. Specifically, a preset rated rank is set in advance, and the value of the preset rated rank is set to 100. The preset initial value of each round of initialization training is calculated based on the demand coefficient corresponding to each round of the initialization sample set and the preset rated rank. The preset initial value is the product of the demand coefficient and the preset rated rank. It should be noted that the preset rated rank can also be selected as other values in the embodiment, which is not specifically limited in the embodiment.

[0083] In addition, the large language model in this step can be pre-trained BERT, LLaMA2, BLOOM, etc., and BERT is taken as an example in the embodiment. In the embodiment, the LoRA fine-tuning matrix is applied to the Query matrix and the Key matrix positions of the self-attention layer in the Transformer architecture of BERT for subsequent fine-tuning training.

[0084] In addition, the large language model in this step can be a typical model with a multi-layer Transformer architecture, such as pre-trained BERT, LLaMA2, BLOOM, etc., and the widely used BERT-base version (12-layer Transformer, hidden layer dimension 768) is taken as an example in the embodiment. Specifically, in the calculation process of the self-attention mechanism of the large language model, the Query matrix is responsible for building the relevance weight of the target word and other words, and the Key matrix stores the feature expression of the sequence elements, so in the embodiment, the LoRA fine-tuning matrix is applied to the Query matrix and the Key matrix positions of the self-attention layer in the Transformer architecture of BERT in a parallel adaptation manner, and the rank is set to use a Gaussian distribution initialization strategy for parameter injection.

[0085] First, the initialization training process is completed by presetting initial values in the present embodiment. Then, the rank of adaptive training is adjusted according to the fine-tuning training that has been performed. For details, see Figure 3 , which shows that it further includes steps S61-S65.

[0086] S61, extracting from all the trained sample sets according to the first sample set to obtain a second sample set and several similar training sample sets, the difference between the training attention degrees corresponding to the similar training sample sets and the first sample set is less than a preset threshold, the first sample set is an adaptive sample set that has not been used for adaptive training, the second sample set is an adaptive sample set that has been used for adaptive training, and the adaptive training rounds of the first sample set and the second sample set are adjacent.

[0087] It should be noted that the first sample set mentioned in the present step refers to the training sample set that is about to be applied to adaptive training, that is, the adaptive sample set. At the same time, the corresponding second sample set refers to the training sample set that has been applied to fine-tuning training in the previous stage. In other words, if the 21st round of fine-tuning training is about to be carried out, the first sample set will be applied to the 21st round of fine-tuning training. The second sample set corresponds to the 20th round of fine-tuning training, which can also be called the 20th round of initialization training. In this case, the trained sample set will contain all the training sample sets used in the 1st to 19th rounds. If the 22nd round of fine-tuning training is about to be carried out, the first sample set will be applied to the 22nd round of fine-tuning training. Accordingly, the second sample set corresponds to the 21st round of fine-tuning training, which can also be called the 21st round of adaptive training. In this case, the trained sample set will contain all the training sample sets used in the 1st to 20th rounds.

[0088] Meanwhile, the difference between the training attention degrees corresponding to the similar training sample sets and the first sample set is less than a preset threshold, which means that the difference between the training attention degrees corresponding to the similar training sample sets and the first sample set is not greater than 10% of the training attention degree corresponding to the first sample set.

[0089] S62, calculating the trend value according to the change difference between the fine-tuning matrix corresponding to the second sample set and each similar training sample set.

[0090] Specifically, in the present embodiment, the trend value is calculated as follows:

[0091] S621, extracting the fine-tuning matrix corresponding to each of the similar training sample sets to obtain a fine-tuning matrix set, and calculating the first weight parameter mean based on the weight parameters in the fine-tuning matrix set.

[0092] It should be noted that the fine-tuning matrices mentioned in this step refer to the fine-tuning matrix A and fine-tuning matrix B of the LoRA method. The set of fine-tuning matrices consists of the fine-tuning matrices A and B after fine-tuning training on all similar training sample sets.

[0093] S622. Extract the fine-tuning matrix of the second sample set and calculate the mean of the weight parameters to obtain the mean of the second weight parameters.

[0094] S623. The trend value is calculated based on the mean of the first weight parameter and the mean of the second weight parameter.

[0095] Specifically, the trend value calculation function in this step is as follows:

[0096] ;

[0097] in, Indicates the first The trend value corresponding to each round of fine-tuning training; Represents the maximum-minimum normalization function; Indicates the first The mean of the second weight parameter after each round of fine-tuning training; Indicates proceeding to the first The mean of the first weight parameter calculated before each round of fine-tuning training.

[0098] In this embodiment, if the first The higher the mean value of the weight parameters in each round of fine-tuning training compared to previous rounds with similar levels of attention, the better. The higher the degree of tendency of each round towards specialized vocabulary, the better the large language model performs in the [number] round. The closer the training trends for specialized word segmentation are in each round, the more suitable the rank becomes. The higher the confidence level of the round rank, the better the analysis process described above can be.

[0099] S63. The adjustment range is calculated based on the trend value, the training attention corresponding to the second sample set and the first sample set.

[0100] Specifically, the adjustment amplitude calculation function in this step is as follows:

[0101] ;

[0102] in, Indicates the first The adjustment range during rounds of fine-tuning training; Represents the maximum-minimum normalization function; Indicates the first The training attention in each round of fine-tuning training, i.e. the training attention corresponding to the first sample set; Indicates the first The training attention corresponding to each round of fine-tuning training, and the training attention corresponding to the second sample set; Indicates the first The trend value corresponding to each round of fine-tuning training.

[0103] In the above calculation function, Reflects the first The difference in the required rank of the fine-tuning matrix between each round of fine-tuning training compared to the previous round is significant. The greater the difference in required rank between two adjacent rounds of fine-tuning training, the higher the required rank becomes. The more rounds of fine-tuning training are done, the more adjustments should be made; at the same time... This reflects that if the trend value of the previous round is lower, the confidence of the rank of the previous round is lower. Therefore, in the fine-tuning training of the y-th round, the adjustment range should be reduced compared to the previous round to reduce the consumption of unnecessary computing resources caused by the unreasonable increase of rank.

[0104] S64. Based on the adjustment range, the training attention corresponding to the first sample set is corrected to obtain the corrected training attention of the first sample set.

[0105] Specifically, the adjustment amplitude calculation function in this step is as follows:

[0106] ;

[0107] in, Indicates the first Training focus after fine-tuning training rounds. Indicates the first The training attention in each round of fine-tuning training, i.e. the training attention corresponding to the first sample set; Indicates the first The training attention corresponding to each round of fine-tuning training, and the training attention corresponding to the second sample set; Indicates the first The range of adjustments during each round of fine-tuning training.

[0108] In the above calculation function, by comparing the first... Rounds of fine-tuning training and the first The differences in training focus between different rounds of fine-tuning serve as the basis for dynamically adjusting the adjustment range, thereby achieving adaptive focus updates based on historical training states. When the... The training attention in each round is greater than or equal to that in the first round. In the round, it indicates the number of rounds. When fine-tuning training is performed on the training sample set of the turn, the large language model should be given more resources for training to improve the model training effect, so a positive number is used to promote, and a negative number is used to inhibit. This differential dynamic adjustment method can effectively balance the sample weight distribution at different training stages, prevent the direction of constantly adjusting the rank from wasting resources, and also avoid model overfitting or falling into a local optimal solution, thereby helping the model to better learn and generalize in the subsequent training process.

[0109] S65, calculating the rank of the first sample set in adaptive training based on the modified training attention and the preset rated rank of the first sample set.

[0110] In this embodiment, the value of the preset rated rank is 100 as mentioned above, and other numerical values can also be selected for the preset rated rank in this embodiment. The product of the modified training attention and the preset rated rank of the first sample set is calculated, and the product is used as the rank of the first sample set in adaptive training.

[0111] In this embodiment, domain-specific text is used as the training set, and general text used by the large language model is used as the comparison set. First, by using word segmentation technology, the frequency difference of each word in the domain text and the general text is analyzed to determine whether the word is more inclined to be used in a specific professional field. At the same time, by combining the semantic changes of the word segmentation in the domain text and the general text, the semantic consistency calculated is used to comprehensively evaluate whether each word segmentation is specific to the field. Based on these analyses, the training attention of each training sample set is further confirmed to ensure that in each round of fine-tuning training, the large model can invest more learning resources for these samples with high attention. In addition, during the LoRA fine-tuning training process, the rank of each round of fine-tuning training is dynamically adjusted in combination with the parameter changes in the fine-tuning matrix and the training attention. Compared with the method of using a fixed rank value in the traditional LoRA fine-tuning process, the present method significantly improves the accuracy of the allocation of computing resources, thereby optimizing the training effect of the model.

[0112] Embodiment 2:

[0113] As shown in Figure 4 , the present embodiment provides a large language model training system, which comprises:

[0114] An acquisition module is configured to acquire domain text.

[0115] A preprocessing module is configured to preprocess the domain text to obtain a domain text training set.

[0116] ​A dividing module is configured to divide the domain text training set into a plurality of training sample sets, and the training sample sets are divided into an initialization sample set or an adaptive sample set.

[0117] A focus calculation module is configured to calculate a training focus corresponding to each training sample set according to semantic changes and frequencies of word segmentation in each training sample set.

[0118] A demand coefficient calculation module is configured to perform maximum and minimum normalization based on all the training focuses to obtain a demand coefficient corresponding to each sample set.

[0119] A fine-tuning module is configured to perform multi-round fine-tuning training on a preset large language model by using LoRA and all the training sample sets to obtain a target large language model. In the multi-round fine-tuning training, initialization training is first performed by using the initialization sample set, and adaptive training is then performed by using the adaptive sample set. The rank of the initialization training is a preset initial value, and the rank of each round of the adaptive training is adjusted by the demand coefficient.

[0120] It should be noted that, as for the system in the above embodiment, the specific manner in which each module performs an operation has been described in detail in the embodiment of the method, and will not be described in detail here.

[0121] Embodiment 3:

[0122] Corresponding to the above method embodiment, the present embodiment also provides a large language model training device. The large language model training device described below can be mutually corresponding to the large language model training method described above.

[0123] Figure 5 is a block diagram of a large language model training device 800 according to an exemplary embodiment. As shown in Figure 5 The large language model training device 800 can include a processor 801 and a memory 802. The large language model training device 800 can also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0124] The processor 801 is configured to control overall operations of the large language model training apparatus 800 to complete all or part of the steps in the large language model training method described above. The memory 802 is configured to store various types of data to support operations of the large language model training apparatus 800. For example, the data can include instructions of any application or method operating on the large language model training apparatus 800, and application-related data, such as contact data, sent and received messages, pictures, audio, video, and the like. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 803 can include a screen and an audio component. The screen can be a touch screen, for example. The audio component is configured to output and / or input audio signals. For example, the audio component can include a microphone configured to receive external audio signals. The received audio signals can be further stored in the memory 802 or transmitted through the communication component 805. The audio component also includes at least one speaker configured to output audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, which can be a keyboard, a mouse, a button, and the like. The buttons can be virtual buttons or physical buttons. The communication component 805 is configured to perform wired or wireless communication between the large language model training apparatus 800 and other devices. The wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 805 can include a Wi-Fi module, a Bluetooth module, an NFC module.

[0125] In an example embodiment, the large language model training apparatus 800 can be implemented by one or more Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic elements for executing the above-mentioned large language model training method.

[0126] In another example embodiment, a computer readable storage medium including program instructions is also provided, which, when executed by a processor, implements the steps of the above-mentioned large language model training method. For example, the computer readable storage medium can be the above-mentioned memory 802 including program instructions, which can be executed by the processor 801 of the large language model training apparatus 800 to complete the above-mentioned large language model training method.

[0127] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0128] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments.

Claims

1. A method for training a large language model, characterized in that, The method includes: Retrieve domain text; The domain text is preprocessed to obtain a domain text training set; The domain text training set is divided into several training sample sets, which are either initialization sample sets or adaptive sample sets. The training attention level corresponding to each training sample set is calculated based on the semantic changes and frequencies of word segmentation within each training sample set. The demand coefficient for each sample set is obtained by performing max-min normalization based on all the training attention; The target large language model is obtained by performing multiple rounds of fine-tuning training on the preset large language model using the LoRA method and all the training sample sets. In the multiple rounds of fine-tuning training, the initialization training is performed first using the initialization sample set, and then the adaptive training is performed using the adaptive sample set. The rank of the initialization training is a preset initial value, and the rank of each round of adaptive training is adjusted by the demand coefficient. The training attention level for each training sample set is calculated based on the semantic changes and frequencies of word segmentation within each training sample set, including: The domain tendency is calculated based on the difference in the number of occurrences of each target word in the domain text and the general text. The target word is the word corresponding to any word vector in a training sample set, and the general text is the training dataset of the pre-set large language model. A semantic consistency evaluation value is calculated based on the meaning changes of each target word in the domain text and the general text. The attention level corresponding to each target word is calculated by multiplying its domain tendency and semantic consistency evaluation value. The training attention is calculated by averaging the attention corresponding to each target word. The rank is calculated as follows in adaptive training: Based on the first sample set extracted from all trained sample sets, a second sample set and several similar training sample sets are obtained. The difference in training attention between the similar training sample sets and the first sample set is less than a preset threshold. The first sample set is an adaptive sample set that has not been used for adaptive training, and the second sample set is an adaptive sample set that has been used for adaptive training. The adaptive training rounds of the first sample set and the second sample set are adjacent. The trend value is calculated based on the difference in change between the second sample set and the fine-tuning matrix corresponding to each similar training sample set. The adjustment range is calculated based on the trend degree value, the training attention corresponding to the second sample set and the first sample set; Based on the adjustment range, the training attention corresponding to the first sample set is corrected to obtain the corrected training attention of the first sample set. The rank of the first sample set during adaptive training is calculated based on the corrected training attention and the preset rank of the first sample set.

2. The large language model training method according to claim 1, characterized in that, The preprocessing includes data cleaning, word segmentation, and vectorization.

3. The large language model training method according to claim 1, characterized in that, The domain tendency is calculated based on the difference in the frequency of each target word in the domain text and the general text, including: The occurrence count of each target word in the general text and the domain text is counted separately to obtain the general occurrence count and the domain occurrence count for each target word; The average values ​​of the general frequency and the domain frequency corresponding to each target word are calculated separately to obtain the average general frequency corresponding to the general text and the average domain frequency corresponding to the domain text. The domain performance degree is calculated based on the difference between the domain frequency and the mean domain frequency for each target word segmentation. The general performance degree is calculated based on the difference between the general frequency and the mean general frequency of each target word segmentation. The domain trend degree is calculated based on the domain performance degree and the general performance degree.

4. The large language model training method according to claim 1, characterized in that, A semantic consistency evaluation value is calculated based on the meaning changes of each target word in the domain text and the general text, including: Extract all sentence segments containing the target word to obtain at least two target sentence segments, and obtain the positional encoding of the target word in each target sentence segment; The target sentence segments are grouped according to the positional encoding to obtain at least one sentence segment group. Each sentence segment group includes at least two target sentence segments, and the positional encoding of the target word segment is the same in each sentence segment group. Calculate the mean word vector for each target sentence segment within each sentence segment group; Pair the target segments within each of the aforementioned segment groups to obtain several segment pairs; The cosine similarity of the word vectors of the two target sentence segments in each sentence segment pair is calculated separately, and the semantic consistency evaluation value is obtained by averaging and normalizing all cosine similarities.

5. The large language model training method according to claim 1, characterized in that, The method for calculating the preset initial value is as follows: The preset initial value corresponding to each round of initial training is calculated based on the demand coefficient and preset rated rank corresponding to the initial sample set in each round.

6. The large language model training method according to claim 1, characterized in that, The trend value is calculated based on the difference in changes between the second sample set and the fine-tuning matrix corresponding to each similar training sample set, including: Extract the fine-tuning matrix corresponding to each of the similar training sample sets to obtain a set of fine-tuning matrices, and calculate the mean of the first weight parameter based on the mean of each weight parameter in the set of fine-tuning matrices. Extract the fine-tuning matrix of the second sample set and calculate the mean of the weight parameters to obtain the mean of the second weight parameters; The trend value is calculated based on the mean of the first weight parameter and the mean of the second weight parameter.

7. A large language model training system, characterized in that, include: The acquisition module is used to acquire domain text; The preprocessing module is used to preprocess the domain text to obtain a domain text training set; The partitioning module is used to divide the domain text training set into several training sample sets, wherein the training sample sets are divided into an initialization sample set or an adaptive sample set; The attention calculation module is used to calculate the training attention corresponding to each training sample set based on the semantic changes and frequencies of word segmentation within each training sample set. The demand coefficient calculation module is used to obtain the demand coefficient corresponding to each of the sample sets by performing maximum and minimum normalization based on all the training attention; The fine-tuning module is used to perform multiple rounds of fine-tuning training on the preset large language model using the LoRA method and all the training sample sets to obtain the target large language model. In the multiple rounds of fine-tuning training, the initialization training is first performed using the initialization sample set, and then the adaptive training is performed using the adaptive sample set. The rank of the initialization training is a preset initial value, and the rank of each round of adaptive training is adjusted by the demand coefficient. The training attention level for each training sample set is calculated based on the semantic changes and frequencies of word segmentation within each training sample set, including: The domain tendency is calculated based on the difference in the number of occurrences of each target word in the domain text and the general text. The target word is the word corresponding to any word vector in a training sample set, and the general text is the training dataset of the pre-set large language model. A semantic consistency evaluation value is calculated based on the meaning changes of each target word in the domain text and the general text. The attention level corresponding to each target word is calculated by multiplying its domain tendency and semantic consistency evaluation value. The training attention is calculated by averaging the attention corresponding to each target word. The rank is calculated as follows in adaptive training: Based on the first sample set extracted from all trained sample sets, a second sample set and several similar training sample sets are obtained. The difference in training attention between the similar training sample sets and the first sample set is less than a preset threshold. The first sample set is an adaptive sample set that has not been used for adaptive training, and the second sample set is an adaptive sample set that has been used for adaptive training. The adaptive training rounds of the first sample set and the second sample set are adjacent. The trend value is calculated based on the difference in change between the second sample set and the fine-tuning matrix corresponding to each similar training sample set. The adjustment range is calculated based on the trend degree value, the training attention corresponding to the second sample set and the first sample set; Based on the adjustment range, the training attention corresponding to the first sample set is corrected to obtain the corrected training attention of the first sample set. The rank of the first sample set during adaptive training is calculated based on the corrected training attention and the preset rank of the first sample set.

8. A large language model training device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the large language model training method as described in any one of claims 1 to 6.

Images