Method and system for screening high-quality instruction compliance data for ultra-long texts

Abstract

The present invention discloses a method and system for screening high-quality instruction-following data for ultra-long texts, the method comprising: calculating the perplexity score of the text under a homology model; splitting the text into multiple equal-length paragraphs, and calculating the importance score corresponding to each paragraph; extracting the word units in the text paragraphs, constructing the average attention weight using a multi-head attention mechanism, and calculating the attention weight score; calculating the context-aware attention alignment difficulty score of the text; calculating the final score of the text according to the perplexity score and attention alignment difficulty score corresponding to each given text; sorting the final scores from large to small, and selecting the texts corresponding to the scores of the top 10%-20% as the screened high-quality instruction-following data. The high-quality instruction-following data in the data set is obtained through the final scoring, and is used for the training of the language model under large-scale conditions. The trained large-scale language model achieves excellent performance in tasks under long text scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of language model training, and in particular to a method and system for screening high-quality instruction compliance data for ultra-long texts. Background Art

[0002] Instruction-following data plays an important and fundamental role in large-scale language model (LLM) training. Large-scale language models trained on such data can demonstrate good ability to follow user instructions and complete tasks given by users. Therefore, how to automatically screen out high-quality instruction-following data for large-scale language model training is a hot topic in academia and industry. At present, the methods in academia and industry only focus on instruction-following data screening methods in short text scenarios, that is, these instruction-following data generally do not exceed 4,000 words. Therefore, large-scale language models trained on instruction-following data screened by these methods cannot solve various tasks in ultra-long text scenarios, such as multi-document summarization, multi-document question and answer, etc. Summary of the invention

[0003] In view of this, the present invention proposes a method and system for screening high-quality instruction compliance data for ultra-long texts, which can effectively screen out compliance data that is beneficial for training large-scale language models in long text scenarios. The present invention provides the following technical solutions:

[0004] A method for screening high-quality instruction compliance data in an ultra-long text scenario, the method comprising:

[0005] Calculate the perplexity scores of different lengths of text in a given dataset under different homology models to measure the difficulty of model response generation due to long-distance dependencies;

[0006] Split each given text into multiple paragraphs of equal length and calculate the importance score corresponding to each paragraph;

[0007] Extract the tokens from each paragraph of each given text, use the multi-head attention mechanism to construct the average attention weight of each token, and calculate the attention weight score of each paragraph of each given text based on the attention weight of each token;

[0008] Calculate the context-aware attention alignment difficulty score for each text based on the importance score and attention weight score of each paragraph of a given text;

[0009] Calculate the final score of the text based on the perplexity score and attention alignment difficulty score corresponding to each given text;

[0010] The final scores are sorted from large to small, and the texts corresponding to the top 10%-20% scores are selected as the screened high-quality instruction compliance data.

[0011] Furthermore, the calculation formula of the perplexity score of the model is:

[0012]

[0013] Among them, c is the given text, x is the input instruction, y is the response output by the model, PPL θ (y|c, x) is the perplexity score of the given text under model θ.

[0014] Furthermore, the number of homology models is two, and the calculation formula of the perplexity score difference score under the guidance of the two homology models is:

[0015] HMP(c, x; y) = Norm(PPLθ A (y|c,x))-Norm(PPLθ B (y|c,x));

[0016] Among them, PPLθ A (y|c, x)) is the given text in the model θ A The perplexity score under B (y|c, x) is the given text in the model θ B HMP(c, x; y) is the perplexity score difference of a given text under different models.

[0017] Furthermore, the calculation formula for the importance score corresponding to each paragraph is:

[0018]

[0019] Among them, s i For the i-th paragraph of a given text S, IS θ (s i ) is the importance score of the i-th paragraph.

[0020] Furthermore, the calculation formula for the attention weight score of each paragraph of each given text is:

[0021]

[0022] Among them, Attn θ (t j |y;θ) is the paragraph s in the model θ output response y for a given text i The word t in j The average attention weight.

[0023] Further, the method for calculating the context-aware attention alignment difficulty score of each text includes:

[0024] The attention alignment difficulty score is calculated by the cosine similarity between the importance score and the attention weight score of a given text, and the calculation formula is:

[0025] CAS(c, x; y) = CosSim([IS θ (s1), ..., IS θ (s N )],[Attn θ (s1), ..., Attn θ (s N )]).

[0026] Further, the method of calculating the final score of the text according to the perplexity score and the attention alignment difficulty score corresponding to each given text includes:

[0027] The perplexity score difference score and the attention alignment difficulty score of a given text are normalized, and the importance of the two scores is balanced through hyperparameters; specifically, the calculation formula for the final score is:

[0028] Score(c,x;y)=α*Norm(HMP(c,x;y))+(1-α)*Norm(CASS(c,x;y));

[0029] Among them, α is a hyperparameter.

[0030] The present invention further discloses a screening system for high-quality instruction compliance data of ultra-long text, comprising:

[0031] The perplexity score calculation module is used to calculate the perplexity scores of texts of different lengths in a given dataset under different homology models to measure the difficulty of model response generation due to long-distance dependencies;

[0032] Importance score calculation module, used to split each given text into multiple paragraphs of equal length and calculate the importance score corresponding to each paragraph;

[0033] The attention weight score calculation module is used to extract the word units in each paragraph of each given text, construct the average attention weight of each word unit by using the multi-head attention mechanism, and calculate the attention weight score of each paragraph of each given text according to the attention weight of each word unit;

[0034] An attention alignment difficulty score calculation module, for calculating a context-aware attention alignment difficulty score for each text according to the importance score and attention weight score of each paragraph of a given text;

[0035] The final score calculation module is used to calculate the final score of the text according to the confusion score and attention alignment difficulty score corresponding to each given text; it is also used to,

[0036] The final scores are sorted from large to small, and a subset of the top 10%-20% is selected as the screened high-quality instruction compliance data.

[0037] The present invention further discloses a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0038] The present invention further discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0039] The technical solution of the present invention can measure the difficulty of generating replies due to long-distance dependencies by calculating the perplexity score of the output reply for a given text under a homology model, and at the same time calculate the importance score of a given paragraph to judge the importance of the current paragraph, and then weight them based on the perplexity score and the importance score to obtain the final score. The final score can be used to obtain high-quality instruction following data in the data set, which can be used for training language models on a large scale, so that the trained large-scale language model can achieve excellent performance in various tasks in long text scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] For the purpose of illustration and not limitation, the present invention is now described in conjunction with the embodiments of the present invention and the accompanying drawings, in which:

[0041] Figure 1 It is a flowchart of a method for screening high-quality instruction compliance data for ultra-long text in an embodiment of the present invention;

[0042] Figure 2 It is a schematic diagram of the structure of a screening system for high-quality instruction compliance data of an ultra-long text in an embodiment of the present invention;

[0043] Figure 3 Schematic diagram of the structure of an electronic product in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the implementation mode of the present application. Obviously, the described implementation mode is only a part of the implementation mode of the present application, not all the implementation modes. Based on the implementation mode in the present application, all other implementation modes obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0045] It should be noted that the terms "first", "second", etc. in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0046] In the present application, the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings. These terms are mainly used to better describe the present application and its implementation methods, and are not used to limit the indicated devices, elements or components to have a specific orientation, or to be constructed and operated in a specific orientation.

[0047] In addition, some of the above terms may be used to express other meanings in addition to indicating orientation or positional relationship. For example, the term "on" may also be used to express a certain dependency or connection relationship in some cases. For those of ordinary skill in the art, the specific meanings of these terms in this application can be understood according to the specific circumstances. In addition, the term "plurality" should mean two or more than two.

[0048] It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0049] refer to Figure 1 This embodiment discloses a method for screening high-quality instruction compliance data in an ultra-long text scenario, the method comprising:

[0050] S100: Calculate the perplexity scores of texts of different lengths in a given dataset under different homology models to measure the difficulty of model response generation due to long-distance dependencies.

[0051] Specifically, the homology models are two models with different text processing windows, but the homology models are the same in the pre-training stage and model architecture. Therefore, when the models have different capabilities to process long-distance dependencies, the introduction of the perplexity score (HMP) can well measure the difficulty of reply generation due to long-distance dependencies. There are two exemplary homology models used in this embodiment. The lengths of the processed texts of these two homology models are different. For example, the processing length of model A is 4K, and the processing length of model B is 32K. For a data set, which includes texts of varying lengths, this embodiment takes a single given text as an example, and other text processing methods are the same. The given text is recorded as a long instruction following sample (c, x; y), where c represents the input context text, x represents the input instruction, and y is the reply given by the current model θ. Based on this, the calculation formula for the perplexity score PPL of the reply given by the model is as follows:

[0052]

[0053] Then, the calculation formula for the confusion of the current given text under two homology models is:

[0054] HMP(c, x; y) = Norm(PPLθ A (y|c,x))-Norm(PPLθ B (y|c,x));

[0055] That is, the scores of the current text under the two models are normalized by applying softmax to determine the ranking of the two perplexity scores in the dataset under the guidance of the corresponding model. For example, the model θ A Using a short context window, model θ B Using a long context window, i.e., LLaMA-2-7B-base-4k is the model θ A , while LLaMA-2-7B-base-32k is the model θ B .

[0056] S200: Split each given text into multiple paragraphs of equal length, and calculate the importance score corresponding to each paragraph.

[0057] Due to long-range dependencies, it is difficult for a large-scale language model (LLM) to utilize key information hidden in an extremely long context. For example, the attention of the LLM may be focused on irrelevant content. To solve this problem, this embodiment introduces a context-aware attention alignment difficulty score (CAS) to evaluate whether the attention of the LLM is appropriately focused on important paragraphs in the long input text context. Taking the given text in step S100 as an example, the input context text c is split into N paragraphs of equal length: Each paragraph has a length of L. For a given paragraph s i , first calculate its importance score IS θ (s i ), the specific calculation formula is:

[0058]

[0059] Among them, s i For the i-th paragraph of a given text S, IS θ (s i ) is the importance score of the i-th paragraph. By retaining the input context of a given paragraph s i To calculate the perplexity score of generating the response y, to indicate that based on the paragraph s i The difficulty of generating a response y. Apply softmax normalization to the score of each paragraph of the given text to determine the difficulty of generating a response s i The ranking among all paragraphs in the text. For the ranking results, the higher the score, the greater the difficulty of LLM and the lower its importance.

[0060] S300: Extract the word units in each paragraph of each given text, use the multi-head attention mechanism to construct the average attention weight of each word unit, and calculate the attention weight score of each paragraph of each given text according to the attention weight of each word unit.

[0061] Specifically, after calculating the importance scores of different paragraphs of the current specified text, the attention weights in the multi-head attention mechanism are used to measure how LLM uses these paragraphs, that is, extract paragraph s i Multiple words in [t1, ..., t L ], and use its attention weight as a score to perform softmax normalization on multiple words to form an attention weight score, and further determine the score ranking of the word in the paragraph. The calculation formula of the attention weight score is:

[0062]

[0063] Among them, Attn θ (t j|y;θ) is the paragraph s in the model θ output response y for a given text i The word t in j The average attention weight.

[0064] In this way, the attention weight between the reply and the paragraph can be calculated to indicate whether LLM pays attention to and utilizes the overly long input text according to the self-attention mechanism in the process of generating the reply, that is, giving higher weight to important text.

[0065] S400: Calculate a context-aware attention alignment difficulty score for each text according to the importance score and attention weight score of each paragraph of a given text.

[0066] After obtaining the importance score and attention weight score of each paragraph, in order to further measure the difficulty of understanding the long input context due to long-distance dependencies, the cosine similarity between the importance score and the attention weight score is constructed to calculate the attention alignment difficulty score. The calculation formula is as follows:

[0067] CAS(c, y; y) = CosSim([IS θ (s1), ..., IS θ (s N )],[Attn θ (s1), ..., Attn θ (s N )]).

[0068] S500: Calculate the final score of the text based on the perplexity score and attention alignment difficulty score corresponding to each given text.

[0069] By weighting the perplexity score and attention alignment difficulty score corresponding to a given text, we can get the final score that takes both into account. The calculation formula is:

[0070] Score(c,x;y)=α*Norm(HMP(c,x;y))+(1-α)*Norm(CAS(c,x;y));

[0071] Through the above formula, the perplexity score and attention alignment difficulty score of a given text are softmax normalized, where α is a hyperparameter used to balance the importance of the above two scores.

[0072] S600: Sort the final scores from large to small, and select a subset of the top 10%-20% of the scores as the selected high-quality instruction compliance data.

[0073] In this embodiment, for example, a subset of top 10% is selected as high-quality instruction compliance data, and the data is used as input data for LLM training.

[0074] It should be understood that this embodiment only describes the implementation of the scheme based on a specified text among multiple input texts, and the processing methods of other texts in the data set and texts in other data sets are the same as this embodiment.

[0075] The compliance data screened out by the method of this embodiment can be used for training large-scale language models, so that the trained large-scale language models can achieve excellent performance in various tasks in long text scenarios.

[0076] refer to Figure 2 This embodiment further discloses a screening system for high-quality instruction compliance data of ultra-long texts, which is used to implement the above method. The system includes:

[0077] The perplexity score calculation module is used to calculate the perplexity scores of texts of different lengths in a given dataset under different homology models to measure the difficulty of model response generation due to long-distance dependencies; the calculation formula of the perplexity score of the model is:

[0078]

[0079] Among them, c is the given text, x is the input instruction, y is the response output by the model, PPL θ (y|c, x) is the perplexity score of a given text under model θ. In this embodiment, the number of homology models is two, and the calculation formula of the perplexity score difference score under the guidance of the two homology models is:

[0080] HMP(c, x; y) = Norm(PPLθ A (y|c,x))-Norm(PPLθ B (y|c,x));

[0081] Among them, PPLθ A (y|c, x)) is the given text in the model θ A The perplexity score under B (y|c, x) is the given text in the model θ B HMP(c, x; y) is the perplexity score difference of a given text under different models.

[0082] The importance score calculation module is used to split each given text into multiple paragraphs of equal length and calculate the importance score corresponding to each paragraph; the calculation formula for the importance score corresponding to each paragraph is:

[0083]

[0084] Among them, s i For the i-th paragraph of a given text S, IS θ (s i ) is the importance score of the i-th paragraph.

[0085] The attention weight score calculation module is used to extract the word units in each paragraph of each given text, construct the average attention weight of each word unit by using the multi-head attention mechanism, and calculate the attention weight score of each paragraph of each given text according to the attention weight of each word unit; wherein, the calculation formula of the attention weight score of each paragraph of each given text is:

[0086]

[0087] Among them, Attn θ (t j |y;θ) is the paragraph s in the model θ output response y for a given text i The word t in j The average attention weight.

[0088] The attention alignment difficulty score calculation module is used to calculate the context-aware attention alignment difficulty score of each text according to the importance score and attention weight score of each paragraph of the given text; wherein the calculation method of calculating the context-aware attention alignment difficulty score of each text includes:

[0089] The attention alignment difficulty score is calculated by the cosine similarity between the importance score and the attention weight score of a given text, and the calculation formula is:

[0090] CAS(c, x; y) = CosSim([IS θ (s1), ..., IS θ (s N )],[Attn θ (s1), ..., Attn θ (s N )]).

[0091] The final score calculation module is used to calculate the final score of the text according to the confusion score and attention alignment difficulty score corresponding to each given text; it is also used to,

[0092] The final scores are sorted from large to small, and the texts corresponding to the top 10%-20% scores are selected as the selected high-quality instruction following data. The method of calculating the final score of the text according to the perplexity score and attention alignment difficulty score corresponding to each given text includes: normalizing the perplexity score difference score and attention alignment difficulty score of the given text, and balancing the importance of the two scores through hyperparameters; specifically, the calculation formula of the final score is:

[0093] Score(c,x;y)=α*Norm(HMP(c,x;y))+(1-α)*Norm(CAS(c,x;y));

[0094] Among them, α is a hyperparameter.

[0095] Figure 3 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 3 As shown, the electronic device 50 includes: a processor 501 (processor), a memory 502 (memory) and a bus 503;

[0096] The processor 501 and the memory 502 communicate with each other via the bus 503 ; the processor 501 is used to call program instructions in the memory 502 to execute the methods provided by the above-mentioned method implementation methods.

[0097] This embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable a computer to execute the methods provided by the above-mentioned method embodiments.

[0098] A person skilled in the art can understand that all or part of the steps for implementing the above-mentioned method implementation method can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method implementation method; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various storage media that can store program codes.

[0099] The device implementation described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present implementation scheme. Those of ordinary skill in the art may understand and implement it without creative effort.

[0100] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each implementation method or some parts of the implementation method.

[0101] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for screening high-quality instruction compliance data for ultra-long texts, characterized in that: The method comprises: Calculate the perplexity scores of texts of different lengths in a given dataset under different homology models. The number is used to measure the difficulty of generating model responses due to long-distance dependencies; Split each given text into multiple paragraphs of equal length and calculate the weight corresponding to each paragraph. Importance score: Extract the word units in each paragraph of each given text and use the multi-head attention mechanism to construct each The average attention weight of the word units, and the attention weight of each word unit is calculated for each given The attention weight score for each paragraph of the text; Calculate each paragraph based on the importance score and attention weight score of each paragraph in the given text Context-aware attention alignment difficulty score for text; Calculate the text based on the confusion score and attention alignment difficulty score corresponding to each given text Ben’s final score; Sort the final scores from largest to smallest and select the top 10%-20% scores. The corresponding texts are used as the filtered high-quality instruction-following data.

2. The screening of high-quality instruction compliance data for ultra-long text as claimed in claim 1 The method is characterized in that The perplexity score of the model is calculated as: Among them, c is the given text, x is the input instruction, y is the response output by the model, and PPLθ(y|c, x) is the confusion score of the given text under model θ.

3. The screening of high-quality instruction compliance data for super-long text as claimed in claim 2 The method is characterized in that The number of homology models is two. Under the guidance of the two homology models The calculation formula for the perplexity score difference score is: HMP(c,x;y)=Norm(PPLθ A (y|c,x))-Norm(PPLθ B (y|c,x)); Among them, PPLθ A (y|c, x)) is the given text in the model θ A The perplexity score under B (y|c, x) is the given text in the model θ B HMP(c, x; y) is the perplexity score difference of a given text under different models.

4. The method for screening high-quality instruction compliance data for ultra-long text according to claim 3, characterized in that: The calculation formula for the importance score corresponding to each paragraph is: Among them, s i For the i-th paragraph of a given text S, IS θ (s i ) is the importance score of the i-th paragraph.

5. The method for screening high-quality instruction compliance data for super-long text according to claim 4, characterized in that: The calculation formula for the attention weight score of each paragraph of each given text is: Among them, Attn θ (t j |y;θ) is the paragraph s in the model θ output response y for a given text i The word t in j The average attention weight.

6. The method for screening high-quality instruction compliance data for super-long text according to claim 5, characterized in that: The method for calculating the context-aware attention alignment difficulty score for each text includes: The attention alignment difficulty score is calculated by the cosine similarity between the importance score and the attention weight score of a given text, and the calculation formula is:

7. The method for screening high-quality instruction compliance data for super-long text according to claim 6, characterized in that: The method for calculating the final score of a text according to the perplexity score and the attention alignment difficulty score corresponding to each given text includes: The perplexity score difference score and the attention alignment difficulty score of a given text are normalized, and the importance of the two scores is balanced through hyperparameters; specifically, the calculation formula for the final score is: Score(c,x;y)=α*Norm(HMP(c,x;y))+(1-α)*Norm(CAS(c,x;y)); Among them, α is a hyperparameter.

8. A screening system for high-quality instruction compliance data of ultra-long texts, characterized in that: include: The perplexity score calculation module is used to calculate the perplexity scores of texts of different lengths in a given dataset under different homology models to measure the difficulty of model response generation due to long-distance dependencies; Importance score calculation module, used to split each given text into multiple paragraphs of equal length and calculate the importance score corresponding to each paragraph; The attention weight score calculation module is used to extract the word units in each paragraph of each given text, construct the average attention weight of each word unit by using the multi-head attention mechanism, and calculate the attention weight score of each paragraph of each given text according to the attention weight of each word unit; An attention alignment difficulty score calculation module, for calculating a context-aware attention alignment difficulty score for each text according to the importance score and attention weight score of each paragraph of a given text; The final score calculation module is used to calculate the final score of the text according to the confusion score and attention alignment difficulty score corresponding to each given text; it is also used to, The final scores are sorted from large to small, and the texts corresponding to the top 10%-20% scores are selected as the screened high-quality instruction compliance data.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 7 is implemented.

Images