Training data construction method and device

Abstract

The invention provides a training data construction method and device, and relates to the technical field of data processing, and the method comprises the steps: executing iterative semantic segmentation on a long text document to be processed, generating text blocks with complete semantics, and generating training data pairs containing questions, context text blocks and standard answers based on a predefined task type and the text blocks; reasoning the training data pairs by using a large language model to obtain prediction accuracy rates of different task types, and dynamically adjusting sampling weights of the training data pairs corresponding to the task types based on the prediction accuracy rates to generate a difficulty-balanced training data set; embedding a task guide identifier corresponding to the task type in the data of the training data set to obtain final training data for model training; wherein the task guide identifier is used for enabling the model to identify different task types and generate differentiated reward signals.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for constructing training data. Background Technology

[0002] With the rapid development of large language models in the field of natural language processing, improving the model's ability to process long texts has become a research hotspot. In the reinforcement learning training of large language models, the model is a key component guiding policy optimization. Existing technologies have the following main shortcomings when constructing model training data.

[0003] Existing methods for constructing training data heavily rely on manual annotation and selection, which is inefficient and costly. Furthermore, existing methods lack effective control over task difficulty during data generation, easily leading to training data that is either too simple or too difficult, thus impacting the model's learning efficiency.

[0004] Therefore, how to effectively construct high-quality training data for models has become an urgent problem to be solved in the industry. Summary of the Invention

[0005] This invention provides a training data construction method to solve the problem of how to effectively construct high-quality training data for models in the prior art.

[0006] This invention provides a method for constructing training data, comprising: Iterative semantic segmentation is performed on the long text document to be processed to generate semantically complete text blocks, and training data pairs containing questions, contextual text blocks and standard answers are generated based on the predefined task type and the text blocks. The training data pairs are inferred using a large language model to obtain the prediction accuracy for different task types. Based on the prediction accuracy, the sampling weights of the training data pairs corresponding to each task type are dynamically adjusted to generate a training dataset with balanced difficulty. The task guidance identifiers corresponding to the task types are embedded in the training dataset to obtain the final training data for model training; wherein, the task guidance identifiers are used to enable the model to identify different task types and generate differentiated reward signals.

[0007] According to a training data construction method provided by the present invention, the method for generating the training data pairs includes: For any task type, based on the feature template corresponding to the task type, the large language model is invoked to generate a question that matches the task type. Based on the task type, select the corresponding answer generation strategy and generate a standard answer corresponding to the question based on the text block; The generated questions, corresponding text blocks as context, and standard answers are combined to form training data pairs.

[0008] According to a training data construction method provided by the present invention, the answer generation strategy includes: When the task type is a location retrieval task, the target content fragment to be located is obtained in advance; Perform syntactic structure analysis on the text block to identify sentence boundaries and paragraph structure, and determine candidate insertion positions that do not disrupt the semantic coherence of the original text; The target content fragment is randomly inserted into a selected candidate insertion position to form a new text block containing the pre-embedded content; The insertion position information of the target content fragment is recorded as the standard answer for the location retrieval task.

[0009] According to a training data construction method provided by the present invention, the answer generation strategy includes: When the task type is a factual question-and-answer task, multiple large language models are invoked to generate candidate answers for the same question in the text block. Semantic similarity calculation and cluster analysis are performed on all candidate answers. The frequency of each answer cluster is counted. The standard answer is determined based on the cluster of answers with the highest frequency that exceeds the preset voting threshold.

[0010] According to a training data construction method provided by the present invention, after the step of generating training data pairs containing questions, context text blocks, and standard answers based on predefined task types and the text blocks, the method further includes: Based on the character length of the context text block in the training data pair, the training data pair is assigned to multiple preset length intervals, and the length intervals are set in an exponentially increasing manner. The length intervals are sorted in ascending order to form a progressive training data sequence. Training and difficulty balancing are then carried out in stages according to the progressive training data sequence.

[0011] According to a training data construction method provided by the present invention, the method involves using a large language model to infer the training data pairs, obtaining prediction accuracy for different task types, and dynamically adjusting the sampling weights of the training data pairs corresponding to each task type based on the prediction accuracy to generate a difficulty-balanced training dataset, comprising: The large language model is used to perform inference and prediction on the training data for each task type, and the prediction accuracy for each task type is calculated. For each task type, when the prediction accuracy of the task type is higher than the target accuracy, the sampling weight of the task type is reduced; when the prediction accuracy of the task type is lower than the target accuracy, the sampling weight of the task type is increased. Based on the adjusted sampling weights, the training data pairs for each task type are resampled to generate a training dataset with balanced difficulty.

[0012] According to a training data construction method provided by the present invention, the method for embedding the task guidance identifier includes: Obtain a unique identifier for each task type, the identifier containing semantic description information of the corresponding task type; A formatted task guidance identifier is inserted at the beginning of the text sequence of the training data. This task guidance identifier serves as an explicit feature in subsequent model training, guiding the model to activate the corresponding reward calculation branch for different task types.

[0013] According to a training data construction method provided by the present invention, the step of performing iterative semantic segmentation on the long text document to be processed to generate semantically complete text blocks includes: Automated segmentation is performed based on the structured tags of the long text document to obtain an initial set of text blocks; The semantic integrity of each text block in the initial text block set is scored by calling the large language model, and the semantic integrity score of each text block is obtained. The segmentation boundaries are adjusted based on the score, and the automated segmentation and scoring steps are repeated until the semantic integrity score reaches the preset convergence standard, generating a semantically complete text block.

[0014] The present invention also provides a training data construction apparatus, comprising: The generation module is used to perform iterative semantic segmentation on the long text document to be processed, generate semantically complete text blocks, and generate training data pairs containing questions, contextual text blocks, and standard answers based on the predefined task type and the text blocks; The balancing module is used to infer the training data pairs using a large language model, obtain the prediction accuracy for different task types, and dynamically adjust the sampling weights of the training data pairs corresponding to each task type based on the prediction accuracy to generate a training dataset with balanced difficulty. A construction module is used to embed task guidance identifiers corresponding to task types into the data of the training dataset to obtain the final training data for model training; wherein, the task guidance identifiers are used to enable the model to identify different task types and generate differentiated reward signals.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the training data construction method as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training data construction method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the training data construction method as described above.

[0018] The training data construction method and apparatus provided by this invention generate semantically complete text blocks by performing iterative semantic segmentation on long text documents, and automatically generate training data pairs containing questions, context, and standard answers based on predefined task types. This achieves efficient and automated construction of training data, significantly reducing manual costs. A large language model is used to infer prediction accuracy from the training data, and the sampling weights of data for each task type are dynamically adjusted accordingly. This ensures that the generated training dataset achieves an optimal balance in difficulty, allowing the model to obtain sufficient gradient signals across various tasks, thereby maximizing the training efficiency of reinforcement learning. By embedding task guidance identifiers corresponding to task types into the training data, the model can accurately identify different task types and generate differentiated reward signals. This effectively eliminates crosstalk between tasks in multi-task scenarios, significantly improving the accuracy and stability of reward signals, and providing more accurate and reliable optimization guidance for subsequent reinforcement learning training. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the training data construction method provided by the present invention; Figure 2 A schematic diagram of the objective problem acquisition process provided by this invention; Figure 3 A schematic diagram illustrating the process of obtaining the answer is provided for this invention; Figure 4 A schematic diagram of the model training process for preventing task crosstalk provided by the present invention; Figure 5A flowchart illustrating the process of constructing objective data from long texts provided by this invention; Figure 6 A flowchart illustrating the construction of objective long text data based on length awareness and difficulty balance provided by this invention. Figure 7 A schematic diagram of the training data construction device provided by the present invention; Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] Figure 1 This is a flowchart illustrating the training data construction method provided by the present invention, as shown below. Figure 1 As shown, the method includes the following: Step 110: Perform iterative semantic segmentation on the long text document to be processed to generate semantically complete text blocks, and generate training data pairs containing questions, contextual text blocks and standard answers based on the predefined task type and the text blocks; Specifically, this method first needs to process long text documents. Long text documents here refer to texts whose character count or number of terms exceeds the processing window length of a typical large language model, such as, but not limited to, academic papers, legal documents, listed company annual reports, and book chapters. Due to their length, direct processing is difficult and inefficient.

[0023] Therefore, this long text document needs to be segmented. To ensure the semantic independence and integrity of each segmented text block, and to avoid damaging the logical structure and information association of the original text due to improper segmentation, this embodiment adopts an iterative semantic segmentation method.

[0024] In this application, iterative semantic segmentation refers to segmenting a long text document through multiple iterations. Each iteration evaluates the semantic completeness of the segmentation results and adjusts the segmentation strategy based on the evaluation results until a semantically complete text block is obtained. A semantically complete text block refers to a text segment that contains complete semantic information, is logically coherent, and can be independently understood.

[0025] After obtaining semantically complete text blocks, training data can be generated based on predefined task types.

[0026] Task types are defined to train large language models to perform specific tasks in long text processing. For example, they can be location retrieval tasks, which involve accurately locating specific information in long texts; factual question answering tasks, which involve answering factual questions based on text content; summary generation tasks; and reasoning question answering tasks.

[0027] For each task type and each text block, a training data pair is generated. This training data pair is a structured data unit, and its standard form contains three core elements: a question generated based on the task type and text block content; the corresponding text block itself as context; and a standard answer corresponding to the question.

[0028] For example, given a text block about a company's annual report and a "factual question and answer" task type, the question might be: "How much did the company invest in R&D last year?" The context would be the text block of the annual report, and the standard answer would be the specific amount extracted from the text block.

[0029] During the generation process, different generation strategies can be adopted based on the characteristics of the task type. For example, for factual question-answering tasks, key factual information in text blocks is identified, and questions targeting these facts are generated; for reasoning and analysis tasks, questions requiring multiple steps of reasoning to answer are constructed. Question generation can be achieved through pre-set templates, rules, or by invoking a language model. The generation of standard answers must ensure consistency and accuracy between the answer and the question and context.

[0030] Step 120: Use a large language model to infer the training data pairs, obtain the prediction accuracy for different task types, and dynamically adjust the sampling weights of the training data pairs corresponding to each task type based on the prediction accuracy to generate a training dataset with balanced difficulty. In this application, the large language model refers to the language model to be trained or optimized, which can be any pre-trained model based on the Transformer architecture. Specifically, it refers to the target to be optimized by the model trained with the training data in this application.

[0031] This step assesses the difficulty level of different task types by having a large language model perform inference predictions on training data pairs.

[0032] Specifically, the question and contextual text block from each training data pair are input into a large language model to obtain the model's predicted answer.

[0033] The prediction accuracy for each task type is calculated by comparing the predicted answers with the standard answers. The prediction accuracy reflects the ability of the large language model to handle this type of task in the current state.

[0034] Based on the obtained prediction accuracy, the sampling weights of training data pairs for each task type are dynamically adjusted.

[0035] The sampling weights determine the proportion of each task type in the final training dataset. For task types where the model performs well, the sampling weights are reduced; for task types where the model performs poorly, the sampling weights are increased.

[0036] The dynamic adjustment mechanism ensures the balance of the training dataset in terms of difficulty distribution, avoiding the problem of overfitting the model on some simple tasks or underfitting it on some difficult tasks.

[0037] Based on the adjusted sampling weights, data is proportionally extracted from the training data pools for each task type to form the final training dataset. This dataset provides a reasonable distribution of tasks of varying difficulty levels, offering better learning gradients for model training.

[0038] Step 130: Embed task guidance identifiers corresponding to task types into the data of the training dataset to obtain the final training data for model training; wherein, the task guidance identifiers are used to enable the model to identify different task types and generate differentiated reward signals.

[0039] Task guidance identifiers are special tags or labels used to explicitly indicate the task type to which the data belongs in the training data. These identifiers can be text labels (such as [FACTQA], [REASONING]) or specific token sequences or encoded vectors.

[0040] Embedded task-guided identifiers are first assigned a unique identifier for each task type, and then these identifiers are inserted at the appropriate position in the training data, typically at the beginning of a text sequence.

[0041] The embedding method needs to ensure that the identifier does not interfere with the semantics of the original text, while being effectively recognized by the model.

[0042] Task-guided identifiers play a crucial role in model training. The model is used within a reinforcement learning framework to evaluate the quality of its output, and it needs to generate differentiated reward signals for different task types.

[0043] The model can activate corresponding evaluation strategies by recognizing task guidance markers, and adopt different scoring standards and weight configurations for different task types.

[0044] For example, in factual question-answering tasks, the model focuses more on the accuracy of the answer; in creative writing tasks, it emphasizes the originality and fluency of the content. This differentiated reward signal can more accurately guide the model's optimization direction and improve the efficiency of reinforcement learning.

[0045] For example, processed data might be presented as: "<Long Text Question Answering> Question: XXX Context: YYY". This identifier will serve as an explicit input feature for the model, guiding it to internally activate evaluation logic or computational branches appropriate to the task type, thereby generating differentiated reward signals for different tasks and avoiding interference between tasks. The data processed in this way becomes the final training data used for model training.

[0046] In an optional embodiment, the model can be a reward model. It should be noted that the long-text-based training dataset constructed in this invention contains questions, contextual text blocks, and standard answers, constituting objective and verifiable evaluation data. This dataset can be used to train the reward model, enabling it to learn how to evaluate the quality of the model's output, or it can be directly applied to reinforcement learning-based policy optimization training processes, such as GRPO (Group Relative Policy Optimization) training.

[0047] In the GRPO training scenario, the training dataset used for training the reward model can be directly reused without the need for additional data construction. The embedding of task guidance identifiers in this application primarily serves the training and inference phases of the reward model.

[0048] When using the trained reward model to evaluate the reward of the policy model output during GRPO training, a task guidance identifier is added to the input of the reward model, while the input of the policy model itself does not need to be added.

[0049] Through this design, the reward model can perceive the type of the current evaluation task based on task-guided identifiers, thereby activating the corresponding evaluation strategy and generating more accurate reward signals, thus improving the optimization effect of the policy model during reinforcement learning. This combination of data reuse mechanism and identifier differentiation strategy ensures the efficiency of training data construction.

[0050] In this application, iterative semantic segmentation of long text documents generates semantically complete text blocks, and training data pairs containing questions, context, and standard answers are automatically generated based on predefined task types. This achieves efficient and automated construction of training data, significantly reducing manual costs. A large language model is used to infer prediction accuracy from the training data, and the sampling weights of each task type are dynamically adjusted accordingly. This ensures that the generated training dataset achieves an optimal balance in difficulty, allowing the model to obtain sufficient gradient signals across various tasks, thereby maximizing the training efficiency of reinforcement learning. By embedding task guidance identifiers corresponding to task types into the training data, the model can accurately identify different task types and generate differentiated reward signals. This effectively eliminates crosstalk between tasks in multi-task scenarios, significantly improving the accuracy and stability of reward signals and providing more accurate and reliable optimization guidance for subsequent reinforcement learning training.

[0051] Optionally, the method for generating the training data pairs includes: For any task type, based on the feature template corresponding to the task type, the large language model is invoked to generate a question that matches the task type. Based on the answer generation strategy corresponding to the task type, a standard answer corresponding to the question is generated based on the text block; The generated questions, corresponding text blocks as context, and standard answers are combined to form training data pairs.

[0052] In this application, feature templates are pre-designed problem generation frameworks, with each task type having its own set of feature templates. Feature templates define the basic structure, key elements, and language style of the problem.

[0053] For example, for a "location retrieval task", the feature template could be "Please find the description of '{entity name}' in the following text."; for a "factual question-and-answer task", the template could be "Based on the text, what is the definition of '{concept}'?".

[0054] When generating a question, first select a task type and then select a feature template from the question template library corresponding to that task type.

[0055] Then, keywords, entities, and other information can be extracted from the corresponding text blocks and filled into the placeholders in the template to form a specific Prompt.

[0056] Finally, based on this prompt, the large language model is invoked. The model generates a natural, fluent question that matches the task type, based on the prompt and its understanding of the text block content.

[0057] Figure 2This invention provides a schematic diagram of the objective problem acquisition process, such as... Figure 2 As shown, corresponding question sets are generated for different task types, such as finding a needle in a haystack or objective question answering.

[0058] In this application, the answer generation strategy is an answer construction method designed for different task types. Each task type has its specific answer format and generation requirements. The choice of answer generation strategy directly affects the quality of training data and the effectiveness of model training.

[0059] For extractive question answering tasks, the answer generation strategy is to directly locate the text segment containing the answer from the text block, which is usually achieved through methods such as string matching and semantic similarity calculation. For generative question answering tasks, the answer generation strategy may need to synthesize, infer, or paraphrase the text information, in which case a language model will be invoked to generate new answer text based on the content of the text block.

[0060] For classification tasks, the answer generation strategy involves analyzing text blocks and assigning corresponding category labels based on a predefined category system. The process of generating standard answers must ensure both accuracy and completeness.

[0061] This application will verify the generated answers, checking whether they match the question, have sufficient textual support, and meet the requirements of the task type.

[0062] After generating the question and the standard answer separately, these three elements—the generated question, the original text block serving as the background for the question, and the generated standard answer—are combined to form a complete, structured training data pair. This data pair serves as the basic unit for subsequent difficulty balancing and model training.

[0063] During the combination process, necessary preprocessing can be performed on each element, such as text cleaning, format standardization, and special character handling.

[0064] At the same time, necessary tags and delimiters can be added to ensure that the data can be correctly parsed in subsequent processing. The combined training data pairs will undergo quality checks to verify the integrity and consistency of the data, ensuring that there is no missing or erroneous information.

[0065] In this application, the above-mentioned refined training data generation method realizes the automatic construction of high-quality, task-specific training data, providing a reliable data foundation for subsequent model training.

[0066] Optionally, the answer generation strategy includes: When the task type is a location retrieval task, the target content fragment to be located is obtained in advance; Perform syntactic structure analysis on the text block to identify sentence boundaries and paragraph structure, and determine candidate insertion positions that do not disrupt the semantic coherence of the original text; The target content fragment is randomly inserted into a selected candidate insertion position to form a new text block containing the pre-embedded content; The insertion position information of the target content fragment is recorded as the standard answer for the location retrieval task.

[0067] In this application, location retrieval tasks are a special type of task whose goal is to train a model to accurately locate specific information in long texts; for example, finding specific information in a long document is commonly known as the "needle in a haystack" task.

[0068] The target content fragment to be located refers to the specific information that needs to be found in the text, which can be a word, phrase, sentence, or paragraph. These target content fragments can be selected from a preset content library or dynamically generated based on the theme of the text block.

[0069] For example, this fragment could be a specific piece of text, such as a fictional password "This is a mysterious password: 6677ptac," a specific fact, or a unique phrase. This fragment can be generated by an automated program or selected from an external corpus.

[0070] To seamlessly integrate target content fragments into the original text, rather than simply piecing them together, syntactic structure analysis of the contextual text blocks is necessary. This analysis can utilize natural language processing tools to identify sentence boundaries, such as periods, question marks, exclamation marks, and paragraph structures.

[0071] This analysis can identify a range of candidate insertion points, typically between sentences or paragraphs. Inserting new content at these points will not disrupt the structure of existing sentences, thus minimizing the disruption of the semantic coherence of the original text.

[0072] The insertion process employs a random selection strategy, randomly choosing one or more insertion positions from a defined pool of candidate locations. This randomness ensures the diversity of training data and prevents the model from learning fixed positional patterns. During insertion, necessary adjustments are made to the target content fragment, such as adding appropriate conjunctions and adjusting voice and tense to ensure natural integration with the context.

[0073] The resulting new text block contains the original content and the inserted target content fragment. A fluency check is performed on the new text block to ensure that the insertion operation does not cause grammatical errors or semantic confusion. If necessary, minor adjustments are made to the text near the insertion location, such as adjusting punctuation or adding transitional phrases, to improve the naturalness of the text.

[0074] The insertion location information includes data from multiple dimensions: character-level start and end positions, word-level indexes, sentence numbers, paragraph numbers, etc. This location information constitutes the standard answer for the location retrieval task and is used to train the model to accurately locate the target content.

[0075] When recording location information, multiple representation methods are used to adapt to different evaluation needs. For example, it can be recorded as character offsets, relative positions, contextual anchors, etc. This multi-dimensional location representation improves the robustness of the answer, and can still accurately evaluate the model's localization ability even if the text undergoes slight changes in subsequent processing.

[0076] This embodiment provides an efficient and reliable data construction scheme for location retrieval tasks that are difficult to automatically generate objective answers through implantation and positioning. It ensures the absolute correctness of the standard answer, avoids ambiguity, and greatly improves the quality and usability of training data for this type of task, thus helping to train models to accurately locate long text information.

[0077] Optionally, the answer generation strategy includes: When the task type is a factual question-and-answer task, multiple large language models are invoked to generate candidate answers for the same question in the text block. Semantic similarity calculation and cluster analysis are performed on all candidate answers. The frequency of each answer cluster is counted. The standard answer is determined based on the cluster of answers with the highest frequency that exceeds the preset voting threshold.

[0078] In this application, factual question-answering tasks require answers to accurately reflect the factual information in the text; therefore, the accuracy of the generated answers is crucial. This embodiment employs a multi-model ensemble strategy to improve the reliability of the answers.

[0079] During the call, each model receives the same input: the question text and a block of context text. However, due to differences in training data, architecture, and parameters, they generate different candidate answers. This diversity provides a rich reference for subsequent answer validation and selection.

[0080] Semantic similarity calculation is the process of evaluating the semantic equivalence between different answers. The training server uses a pre-trained semantic encoding model to convert each candidate answer into a vector representation, and then calculates the cosine similarity between the vectors. Answers with high similarity are considered semantically equivalent, even if they appear different on the surface.

[0081] Cluster analysis is performed based on the calculated similarity matrix. Hierarchical clustering or K-means clustering algorithms are used to group semantically similar answers into the same cluster. The granularity of clustering can be adjusted using a similarity threshold to ensure that answers within the same cluster are highly semantically consistent.

[0082] The frequency of each answer cluster is calculated, i.e., the number of answers contained in each cluster. The frequency reflects the degree of agreement among multiple models for that answer.

[0083] A preset voting threshold is used to filter credible answers. Only when the frequency of a certain answer cluster exceeds the voting threshold is it considered a reliable standard answer. If no answer cluster reaches the threshold, the question will be readjusted or more models will be requested to participate in the voting until a credible answer is obtained.

[0084] From the selected answer clusters, the most representative answer is chosen as the final standard answer. Representativeness can be determined in several ways: selecting the answer closest to the cluster center, selecting the answer with the clearest and most complete expression, or combining multiple answers to generate a new standard answer.

[0085] In this embodiment, by employing a multi-model voting and clustering analysis mechanism, this method effectively aggregates knowledge from multiple models, cross-validates the accuracy of factual information, and generates a more confident and reliable standard answer. This significantly reduces the risk of training data contamination due to errors in a single model, and substantially improves the overall quality of training data for factual question answering tasks.

[0086] Figure 3 This invention provides a schematic diagram for obtaining the answer, such as... Figure 3 As shown, it includes: Firstly, for tasks where answers can be directly extracted from text, such as word frequency statistics, an automated program can be used to search and calculate within the text block to directly derive the standard answer.

[0087] Secondly, for tasks that require pre-embedded information points, such as finding a needle in a haystack, the answer is first generated by an automated program, and then this answer is embedded into the original text block without loss, forming a new text block containing the "pre-embedded answer".

[0088] Third, for open-ended objective question-answering tasks, i.e., factual question-answering tasks, a "Majority-N-early-stop" voting mechanism is adopted. The same question is submitted to multiple different large language models (LLM1, LLM2, LLM3...), and their answers are collected. When the answers from the majority of models tend to be consistent and remain stable, the voting terminates early, and the answer accepted by the majority is taken as the final standard answer.

[0089] Optionally, after the step of generating training data pairs containing questions, contextual text blocks, and standard answers based on a predefined task type and the text blocks, the method further includes: Based on the character length of the context text block in the training data pair, the training data pair is assigned to multiple preset length intervals, and the length intervals are set in an exponentially increasing manner. The length intervals are sorted in ascending order to form a progressive training data sequence. Training and difficulty balancing are then carried out in stages according to the progressive training data sequence.

[0090] In this application, the character length of the context text block is an important indicator of text complexity. Specifically, the number of characters in the context text block of each training data pair can be counted, including letters, numbers, punctuation marks, and spaces. Based on the statistical results, the training data pairs are assigned to a preset length range.

[0091] More specifically, the length intervals are set in an exponentially increasing manner. This setting conforms to the distribution pattern of natural text length and is also conducive to the gradual learning of the model. For example, the length intervals can be set as: [0, 500), [500, 1000), [1000, 2000), [2000, 4000), [4000, 8000), [8000, 16000), etc. The upper bound of each interval is approximately twice the lower bound, forming an exponentially growing sequence. This setting ensures that each interval has sufficient data samples while covering the complete range from short to long texts.

[0092] During the allocation process, statistical information such as the number of data points, task type distribution, and average difficulty for each length interval is recorded. This information is used for subsequent training strategy development and performance evaluation.

[0093] Progressive training data sequences are data structures that organize data of different length ranges in ascending order of text length. This allows the model to first learn simple short text tasks and gradually transition to complex long text tasks.

[0094] The phased training process is as follows: In the first phase, the model is initially trained using the shortest length interval data to enable it to master basic language understanding and generation capabilities; in the second phase, the next shortest length interval data is added to gradually improve the model's ability to process longer texts; in subsequent phases, longer text data is added sequentially until the model can effectively process the longest text.

[0095] At each training phase, a difficulty balancing operation is performed. Specifically, the model's performance across different task types within the current length range is monitored, and the proportion of data for different task types is dynamically adjusted. This phased difficulty balancing ensures that the model achieves balanced learning results at each length level, avoiding training instability caused by excessive differences in text length.

[0096] The advantages of progressive training are: it reduces the learning difficulty in the early stages of training and improves the convergence speed; it allows the model to gradually build the ability to process long texts, avoiding the difficulties of directly dealing with long texts; and the successful learning at each stage provides a good initialization for the next stage, improving the final training effect.

[0097] For example, the dataset can be divided into intervals of [less2k], [2k-4k], [4k-8k], [8k-16k], and [16k-32k]. During dataset construction, the process proceeds in stages, from shortest to longest interval. The first stage involves the [less2k] interval. At this stage, all documents to be processed undergo length checks (measured in real-time by the model's word segmenter, using the number of tokens as the metric): if the document length is between 1 and 2048 tokens, the question and answer retrieval process is initiated; if the document length is less than 1 token (i.e., an empty document or one containing only meaningless symbols), it is directly filtered; if the document length is between 2049 and 4096 tokens, it is marked as "long text to be processed" and moved to the [2k-4k] interval's processing queue, while simultaneously recording the document's unique identifier and the current length check result. In the second stage, [2k-4k], operations are only performed on documents in this interval's processing queue. For documents with a length between 2k and 4k, the question retrieval and answer retrieval process is executed. If there are documents with a length of less than 2049 tokens in the queue, they are skipped directly after being confirmed by secondary word segmentation verification. If the document length is between 4097 and 8192 tokens, it is transferred to the waiting queue in the range of 4k to 8k and the document status label is updated to "cross-stage transfer".

[0098] Subsequent stages proceed according to this logic, processing context up to 32k, and temporarily ignoring context exceeding 32k, ensuring that question-answer pair construction can be completed within the most suitable range for documents of different lengths. This allows the model to gradually encounter and deeply learn the processing logic of long texts during the training process, steadily improving its long text processing capabilities.

[0099] In this application, by introducing a progressive training sequence based on text length, the model can first master basic capabilities on shorter texts and then gradually expand to longer texts, smoothing the learning curve, reducing the difficulty in the early stages of training, and enabling the model to learn and master the ability to process ultra-long texts more effectively.

[0100] Optionally, the step of using a large language model to infer the training data pairs, obtaining the prediction accuracy for different task types, and dynamically adjusting the sampling weights of the training data pairs corresponding to each task type based on the prediction accuracy to generate a difficulty-balanced training dataset includes: The large language model is used to perform inference and prediction on the training data for each task type, and the prediction accuracy for each task type is calculated. For each task type, when the prediction accuracy of the task type is higher than the target accuracy, the sampling weight of the task type is reduced; when the prediction accuracy of the task type is lower than the target accuracy, the sampling weight of the task type is increased. Based on the adjusted sampling weights, the training data pairs for each task type are resampled to generate a training dataset with balanced difficulty.

[0101] In this application, training data pairs for each task type are input into a large language model. The input format typically includes task instructions, questions, and contextual text blocks, and the large language model needs to generate answers based on this information.

[0102] The method for calculating prediction accuracy varies depending on the task type. For exact matching tasks, accuracy is calculated by comparing the degree of perfect match between the predicted answer and the standard answer; for fuzzy matching tasks, accuracy may be calculated using evaluation metrics such as ROUGE and BLEU; for classification tasks, accuracy is the proportion of samples that are correctly classified.

[0103] In this application, a target accuracy can be set, which represents the level of difficulty that is most suitable for the model.

[0104] According to reinforcement learning theory, the learning gradient is maximized and training efficiency is highest when the model's success rate is around 50%. Therefore, a preferred implementation is to set the target accuracy to 0.5.

[0105] If the prediction accuracy ε for a certain task type is significantly higher than the target accuracy of 0.5 (e.g., ε=0.9), it indicates that the task is too simple, and the sampling weight of all data pairs under that task type will be reduced.

[0106] If the prediction accuracy ε for a certain task type is significantly lower than the target accuracy of 0.5 (e.g., ε=0.2), indicating that the task is too difficult, the sampling weight of all data pairs under that task type will be increased. The adjustment of the weight can be proportional to the degree to which the accuracy deviates from the target accuracy.

[0107] Weight adjustments also consider smoothing factors and boundary conditions. Smoothing factors prevent excessively drastic weight changes, maintaining training stability. Boundary conditions ensure that each task type maintains minimal sampling weights, preventing some tasks from being completely ignored.

[0108] After calculating new sampling weights for each task type, these weights are applied to the entire dataset. When building the next training batch, resampling is performed based on these weights.

[0109] Data from task types with higher weights have a greater probability of being selected, while data from task types with lower weights have a lower probability of being selected. Through this weighted sampling method, the difficulty distribution of various tasks in the final generated training dataset will tend to be closer to the level represented by the target accuracy.

[0110] In this application, an adaptive difficulty balancing mechanism is achieved by setting a target accuracy and dynamically adjusting the sampling weights accordingly. This ensures that the model focuses on learning the most valuable content at each stage of training, avoiding wasting computational resources on overly easy or overly difficult data, thereby maximizing the training efficiency and effectiveness of reinforcement learning.

[0111] In a more specific embodiment, constructing a long text objective dataset should be of moderate difficulty for the model to be optimized. Too much difficulty results in sparse rewards and low training efficiency, while too little difficulty renders the data meaningless for the optimization objective. For example, consider two long text word frequency statistics tasks: Problem 1, "Count the frequency of words in the above corpus and return the top-ranked word," is relatively easy, while Problem 2, "Count the frequency of words in the above corpus and return the top ten words," is much more difficult. Considering the training efficiency and optimization objective of reinforcement learning, the optimal difficulty ε for a given task is 0.5; it should not be too difficult to avoid excessively sparse rewards. To obtain the optimal ε value for task difficulty, the following formula is derived: The reinforcement learning optimization objective of large language models is based on the initialization strategy. The strategy is continuously adjusted through interaction with the environment to maximize rewards, which can be expressed by the following formula. ; The optimization objective also includes some entropy constraints, penalizing excessive divergence between input and output to ensure, to some extent, the randomness of the policy and the stability of policy updates. Therefore, the corresponding optimal state-value function is: Based on the initialization strategy, the closed-form solution of the corresponding optimal strategy can be obtained through the Bellman equation. Taking the difference between the logarithms of the initial policy and the optimal policy, and then taking the expectation, yields the expectation expression. That is, the KL divergence between the initialization policy and the optimal policy is .

[0112] Depending on the difficulty level, the following situations apply: If the difficulty coefficient of a task is ε=0, then ,but , ; If the difficulty coefficient of a task is ε=1, then ,but , .

[0113] In both cases, the KL divergence is 0, indicating that there is no room for optimization in this RL training and it has no optimization significance for RL training.

[0114] Regarding the difficulty level of a certain task ,but If it follows a Bernoulli bivariate distribution, then the expectation here can be estimated as follows: ,but ,then Using Taylor expansion, we obtain the exponent part. , bring in get ,when When the KL divergence between the initial and optimal policies is maximized, the optimization significance of RL training is maximized, thus yielding the optimal task difficulty value. .

[0115] Optionally, the method for embedding the task guidance identifier includes: Obtain a unique identifier for each task type, the identifier containing semantic description information of the corresponding task type; A formatted task guidance identifier is inserted at the beginning of the text sequence of the training data. This task guidance identifier serves as an explicit feature in subsequent model training, guiding the model to activate the corresponding reward calculation branch for different task types.

[0116] In this application, a unique identifier is a distinctive label assigned to each task type to clearly distinguish different tasks in data and model processing. The design of the identifier needs to consider readability, uniqueness, and semantic information content.

[0117] For example, the identifier for a fact-based question-answering task could be [FACT_QA] or<factual_question_answering> The identifier for the reasoning task can be [REASONING] or...<logical_inference> .

[0118] The semantic descriptive information contained in identifiers helps models understand the nature of the task. This semantic information can be encoded in several ways: using descriptive English words or phrases; containing key features of the task (such as extractive, generative); or reflecting the difficulty level or domain characteristics of the task.

[0119] Figure 4 This is a schematic diagram of the model training process for preventing task crosstalk provided by the present invention, as shown below. Figure 4 As shown, the training methods of the two models (Reward Model, RM) are compared, aiming to solve the crosstalk problem in multi-task training.

[0120] Traditional process: The document, standard answer, and model's predicted answer are concatenated and directly input into the model, which then outputs a reward value as a supervision signal. The drawback of this method is that the model cannot distinguish between different task types.

[0121] This solution works as follows: Before inputting the data, a clear "<label>" (e.g., <long text question answering> or <summary generation>) is added. This label explicitly informs the model of the type of task being processed. In this way, the model can learn the reward criteria for a specific task, thereby effectively preventing interference between different tasks and generating more accurate and stable reward signals.

[0122] Semantically rich identifiers are not only easier for humans to understand, but also help models learn the relationships and differences between tasks. In this application, a task type identifier mapping table can be maintained, recording the correspondence between each task type and its identifier. This mapping table remains consistent throughout the training process, ensuring the coherence of data processing.

[0123] Formatted task-guided tokens are the result of organizing raw identifiers according to a specific format. The formatting process includes: adding special boundary markers (such as square brackets and angle brackets) to distinguish the tokens from the text; ensuring that the tokens are processed correctly during tokenization; and adding necessary delimiters or spaces to maintain the readability of the text.

[0124] In an alternative embodiment, the insertion position can be selected at the beginning of the text sequence because the identifier of the beginning position is most easily noticed and processed by the model; it will not interfere with the semantic structure of the original text; and it facilitates unified processing during the data preprocessing stage.

[0125] When inserting, you can add an identifier before the original text, then add a separator (such as a colon or a newline), and then follow the original text content.

[0126] The mechanism by which task-guided identifiers act as explicit features is that, in the input layer of the model, the identifiers are encoded as specific vector representations; these vectors activate the corresponding neural network branches or attention modules in the model; different branches use different parameters and computational logic to generate task-specific reward signals.

[0127] For example, when the model recognizes the [FACT_QA] flag, it activates the factual accuracy assessment branch, which focuses on evaluating the consistency between the answer and the facts; when it recognizes the [CREATIVE] flag, it activates the creativity assessment branch, which focuses more on the novelty and diversity of the content.

[0128] Through this explicit task guidance mechanism, the model can provide accurate evaluation and feedback for different task types, avoiding the bias problems caused by using a uniform evaluation standard. This differentiated reward signal can better guide the reinforcement learning process, enabling the model to optimize across various tasks.

[0129] In this application, by explicitly embedding task guidance identifiers with semantic information into the data, this method provides the model with a clear and unambiguous task-discriminating signal. This effectively solves the problem of confusion and crosstalk caused by different task objectives in multi-task mixed training, enabling the model to learn a more accurate and stable reward function for each task, thereby significantly improving the accuracy of the reward signal and the overall performance of subsequent reinforcement learning.

[0130] Optionally, the long text document to be processed undergoes iterative semantic segmentation to generate semantically complete text blocks, including: Automated segmentation is performed based on the structured tags of the long text document to obtain an initial set of text blocks; The semantic integrity of each text block in the initial text block set is scored by calling the large language model, and the semantic integrity score of each text block is obtained. The segmentation boundaries are adjusted based on the score, and the automated segmentation and scoring steps are repeated until the semantic integrity score reaches the preset convergence standard, generating a semantically complete text block.

[0131] In this application, the training server needs to acquire and preprocess long text documents before performing automated segmentation. To ensure the professionalism and scenario adaptability of the training data, document resources are selectively acquired from three key areas: finance, law, and literature and history.

[0132] Documents in the financial field include listed company annual reports, industry research reports, interpretations of financial regulatory policies, and product prospectuses. These documents are characterized by strict formatting standards and a high density of professional terminology. The legal field encompasses current laws and regulations, judicial interpretations, compilations of court precedents, and lawyers' arguments, characterized by rigorous logic and clear hierarchical structure. The literature and history field includes ancient books, academic papers, and historical archives, often containing complex citation relationships and annotation systems. These sources include both publicly published materials and professionally circulated materials within institutions, ensuring the authenticity and representativeness of the training data.

[0133] For the acquired PDF files, the training server uses professional parsing tools for text extraction. Considering the complexity of different PDF document layouts—such as those containing tables, mathematical formulas, embedded images, and multi-column layouts—tools such as PyPDF2 and pdfplumber are used in combination: PyPDF2 is used to efficiently extract plain text content and is suitable for processing simple text-based documents; pdfplumber is specifically designed for documents containing complex tables, accurately identifying text within tables and restoring their layout logic, ensuring that the extracted text maintains its original semantic integrity and structural coherence.

[0134] After text extraction, a dual filtering process is performed. First, a word filtering stage is conducted, constructing a dedicated stop word list based on the characteristics of each domain: in the financial domain, meaningless function words and uncertain expressions such as "it is said" and "it is rumored" are removed, while key business terms such as "growth" and "profit" are retained; in the legal domain, colloquial particles such as "ah" and "ne" are filtered out, but legal terms such as "should" and "must" are retained; in the literature and history domain, high-frequency modern function words and meaningless collation symbols are removed, while classical Chinese-specific modal particles and classical Chinese function words are retained. Second, a document filtering stage is conducted, relying on a pre-set blacklist mechanism to automatically identify and exclude low-quality documents containing a large amount of advertising pop-up text, OCR scan garbled characters, and appendices unrelated to the domain's theme, retaining only the core, valid text.

[0135] After preprocessing, the first round of coarse segmentation begins. The training server predefines a set of delimiters with different priorities, forming a hierarchical segmentation strategy. The highest priority consists of domain-specific structural symbols: chapter titles, core conclusions, risk warnings, etc., in financial reports; the numbering system of Article 1, Paragraph 2, Item 3, etc., in legal texts; and traditional formatting markers such as Volume 1, Part 3, Part 1, etc., in historical documents. The next highest priority consists of general typesetting delimiters, including special bullet points, consecutive line breaks (\n\n), page breaks, etc. The basic priority consists of natural delimiters within paragraphs, such as areas with dense periods at the end of sentences, and parallel structures formed by consecutive semicolons.

[0136] Based on this priority system, a hierarchical segmentation operation is performed. First, using the highest priority delimiter as the baseline, the text is divided into several chapter units with clearly defined thematic boundaries, ensuring that each chapter unit revolves around a complete theme. Then, each chapter unit is further subdivided using secondary priority delimiters, generating paragraph-level text blocks. Finally, if necessary, basic priority delimiters are used to appropriately split excessively long paragraphs. This hierarchical strategy ensures the rationality of the segmentation, maintaining the document's hierarchical structure while avoiding semantic fragmentation caused by excessive subdivision. Through this process, an initial set of text blocks covering the entire text is generated.

[0137] The training server inputs the initial set of text blocks obtained from the first round of coarse segmentation into the large language model, and uses carefully designed prompting engineering to guide the model to complete the dual judgment task.

[0138] First, the constructed prompt words require the large language model to deeply analyze the logical connections between sentences within each text block, check whether there are clear logical relationships such as cause and effect, progression, and transition between adjacent sentences; verify whether the pronoun references are clear and will not cause ambiguity due to the lack of preceding text; and evaluate whether the paragraph theme is consistent and whether there are abrupt topic changes.

[0139] Through these analyses, the model can identify semantic breaks caused by mis-triggered delimiters, i.e., text blocks that are segmented in inappropriate places, resulting in the forced separation of originally coherent semantics.

[0140] Secondly, the title hierarchy check is performed. For text blocks containing titles, the large language model verifies the matching degree between the title description and its subordinate content.

[0141] The model needs to determine: whether the title accurately summarizes the following content; whether the following content is all related to the title's theme; and whether there are any misalignments between the title and the content. Through this verification, the model can identify misaligned title blocks, i.e., abnormal text blocks where the title does not match the actual content.

[0142] Based on the results of the dual-judgment approach, the large language model generates a semantic integrity score of 0-100 for each text block. The score comprehensively considers semantic coherence and structural rationality. Simultaneously, the model outputs a detailed problem diagnosis report, clearly indicating the specific problem type, severity, and suggested adjustment directions for each text block.

[0143] Based on the scores and adjustment suggestions output by the large language model, the automated program on the training server initiates the first round of optimization. The optimization operations include three types: For adjacent text blocks marked as semantic breaks, a merging process is performed, recombining the originally incorrectly segmented content. For example, when a complete argument is split into two parts, and the latter part lacks necessary context, the system will merge the two parts into a single text block.

[0144] For text blocks that are too complex and contain multiple independent topics, they should be split at appropriate locations. The selection of split points is based on the topic transition locations identified by the model, ensuring that each sub-block has independent semantic integrity.

[0145] For misaligned title blocks, adjust the correspondence between the title and the content, moving the misaligned title to the correct position, or adjusting the content under the corresponding title.

[0146] After optimization, the automated program counts the number of changes in the segmentation results of this round, including the total number of merge, split, and move operations.

[0147] If the number of changes exceeds the current threshold, the next iteration is triggered. The optimized set of text blocks is then input into the large language model again. This time, the prompts guide the model to perform more detailed semantic analysis: not only focusing on sentence-level coherence, but also analyzing the logical relationships between paragraphs; not only checking explicit title matching, but also identifying implicit topic boundaries. Based on the results of this new round of analysis, the automated program performs a second optimization, which may discover and correct problems missed in the first round of optimization.

[0148] The final semantically complete text blocks have the following quality assurances: each text block has passed the dual verification of semantic coherence and structural rationality; the boundaries of the text blocks are located at natural semantic breakpoints and structural boundaries; the professionalism and domain characteristics of the original document are maintained; the text blocks are of moderate length and reasonably distributed, neither too fragmented nor too long.

[0149] Through the semantic segmentation method based on domain characteristics, multi-round iteration, and intelligent optimization, this embodiment achieves high-quality segmentation of long texts in professional fields such as finance, law, and literature and history. The generated semantically complete text blocks provide a solid foundation for the construction of subsequent training data, ensuring the professionalism, accuracy, and usability of the final training data, and significantly improving the training effect of large language models for specific domains.

[0150] In one alternative embodiment, Figure 5 The flowchart for constructing objective data from long texts provided by this invention is as follows: Figure 5 As shown, a long text document is first obtained as input. By combining the prompting engineering and automation program of the large language model, the long text is segmented into a set of structured paragraphs and chapters using an iterative method. For these segmented text blocks, the large language model automatically generates corresponding question sets and answers according to the preset task type, such as objective question answering and information extraction, and prompt templates. Based on the generated question and answer pairs, the system constructs an instruction dataset and calls the model to be optimized to perform difficulty assessment and filtering of the data. Finally, a high-quality optimized dataset with balanced difficulty is output for subsequent reinforcement learning training.

[0151] Figure 6 A detailed flowchart for constructing objective long text data based on length awareness and difficulty balance, as provided in this invention, is shown below. Figure 6As shown, firstly, the document is broken down into multiple text blocks using a text hierarchy segmentation module. Next, in the task classification stage, various task types are generated at both the paragraph and document levels for these text blocks, such as needle-in-a-haystack (locating specific information in a long text), objective question answering, and word frequency statistics. The answer generator then extracts or generates standard answers.

[0152] Finally, all generated question-answer pairs will enter the difficulty screening stage, which will use a large language model for evaluation and filtering to ensure that the quality and difficulty of the data are appropriate, ultimately producing a high-quality long text objective dataset.

[0153] The training data construction apparatus provided by the present invention is described below. The training data construction apparatus described below and the training data construction method described above can be referred to in correspondence.

[0154] Figure 7 A schematic diagram of the training data construction device provided by the present invention is shown below. Figure 7 As shown, it includes: The generation module 710 is used to perform iterative semantic segmentation on the long text document to be processed, generate semantically complete text blocks, and generate training data pairs containing questions, contextual text blocks and standard answers based on the predefined task type and the text blocks; The balancing module 720 is used to infer the training data pairs using a large language model, obtain the prediction accuracy of different task types, and dynamically adjust the sampling weight of the training data pairs corresponding to each task type based on the prediction accuracy to generate a training dataset with balanced difficulty. The construction module 730 is used to embed task guidance identifiers corresponding to task types into the data of the training dataset to obtain the final training data for model training; wherein, the task guidance identifiers are used to enable the model to identify different task types and generate differentiated reward signals.

[0155] In this application, iterative semantic segmentation of long text documents generates semantically complete text blocks, and training data pairs containing questions, context, and standard answers are automatically generated based on predefined task types. This achieves efficient and automated construction of training data, significantly reducing manual costs. A large language model is used to infer prediction accuracy from the training data, and the sampling weights of each task type are dynamically adjusted accordingly. This ensures that the generated training dataset achieves an optimal balance in difficulty, allowing the model to obtain sufficient gradient signals across various tasks, thereby maximizing the training efficiency of reinforcement learning. By embedding task guidance identifiers corresponding to task types into the training data, the model can accurately identify different task types and generate differentiated reward signals. This effectively eliminates crosstalk between tasks in multi-task scenarios, significantly improving the accuracy and stability of reward signals and providing more accurate and reliable optimization guidance for subsequent reinforcement learning training.

[0156] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a training data construction method, which includes: performing iterative semantic segmentation on the long text document to be processed, generating semantically complete text blocks, and generating training data pairs containing questions, contextual text blocks, and standard answers based on a predefined task type and the text blocks. The training data pairs are inferred using a large language model to obtain the prediction accuracy for different task types. Based on the prediction accuracy, the sampling weights of the training data pairs corresponding to each task type are dynamically adjusted to generate a training dataset with balanced difficulty. The task guidance identifiers corresponding to the task types are embedded in the training dataset to obtain the final training data for model training; wherein, the task guidance identifiers are used to enable the model to identify different task types and generate differentiated reward signals.

[0157] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0158] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the training data construction method provided by the above methods, the method including: performing iterative semantic segmentation on the long text document to be processed, generating semantically complete text blocks, and generating training data pairs containing questions, context text blocks and standard answers based on a predefined task type and the text blocks; The training data pairs are inferred using a large language model to obtain the prediction accuracy for different task types. Based on the prediction accuracy, the sampling weights of the training data pairs corresponding to each task type are dynamically adjusted to generate a training dataset with balanced difficulty. The task guidance identifiers corresponding to the task types are embedded in the training dataset to obtain the final training data for model training; wherein, the task guidance identifiers are used to enable the model to identify different task types and generate differentiated reward signals.

[0159] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training data construction method provided by the above methods, the method comprising: performing iterative semantic segmentation on a long text document to be processed, generating semantically complete text blocks, and generating training data pairs containing questions, context text blocks and standard answers based on a predefined task type and the text blocks; The training data pairs are inferred using a large language model to obtain the prediction accuracy for different task types. Based on the prediction accuracy, the sampling weights of the training data pairs corresponding to each task type are dynamically adjusted to generate a training dataset with balanced difficulty. The task guidance identifiers corresponding to the task types are embedded in the training dataset to obtain the final training data for model training; wherein, the task guidance identifiers are used to enable the model to identify different task types and generate differentiated reward signals.

[0160] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing training data, characterized in that, include: Iterative semantic segmentation is performed on the long text document to be processed to generate semantically complete text blocks, and training data pairs containing questions, contextual text blocks and standard answers are generated based on the predefined task type and the text blocks. The training data pairs are inferred using a large language model to obtain the prediction accuracy for different task types. Based on the prediction accuracy, the sampling weights of the training data pairs corresponding to each task type are dynamically adjusted to generate a training dataset with balanced difficulty. The task guidance identifiers corresponding to the task types are embedded in the training dataset to obtain the final training data for model training; wherein, the task guidance identifiers are used to enable the model to identify different task types and generate differentiated reward signals.

2. The training data construction method according to claim 1, characterized in that, The method for generating the training data pairs includes: For any task type, based on the feature template corresponding to the task type, the large language model is invoked to generate a question that matches the task type. Based on the answer generation strategy corresponding to the task type, a standard answer corresponding to the question is generated based on the text block; The generated questions, corresponding text blocks as context, and standard answers are combined to form training data pairs.

3. The training data construction method according to claim 2, characterized in that, The answer generation strategy includes: When the task type is a location retrieval task, the target content fragment to be located is obtained in advance; Perform syntactic structure analysis on the text block to identify sentence boundaries and paragraph structure, and determine candidate insertion positions that do not disrupt the semantic coherence of the original text; The target content fragment is randomly inserted into a selected candidate insertion position to form a new text block containing the pre-embedded content; The insertion position information of the target content fragment is recorded as the standard answer for the location retrieval task.

4. The training data construction method according to claim 2, characterized in that, The answer generation strategy includes: When the task type is a factual question-and-answer task, multiple large language models are invoked to generate candidate answers for the same question in the text block. Semantic similarity calculation and cluster analysis are performed on all candidate answers. The frequency of each answer cluster is counted. The standard answer is determined based on the cluster of answers with the highest frequency that exceeds the preset voting threshold.

5. The training data construction method according to claim 1, characterized in that, After the step of generating training data pairs containing questions, contextual text blocks, and standard answers based on the predefined task type and the text blocks, the method further includes: Based on the character length of the context text block in the training data pair, the training data pair is assigned to multiple preset length intervals, and the length intervals are set in an exponentially increasing manner. The length intervals are sorted in ascending order to form a progressive training data sequence. Training and difficulty balancing are then carried out in stages according to the progressive training data sequence.

6. The training data construction method according to claim 1, characterized in that, The process involves using a large language model to infer the training data pairs, obtaining prediction accuracy for different task types, and dynamically adjusting the sampling weights of the training data pairs corresponding to each task type based on the prediction accuracy to generate a difficulty-balanced training dataset, including: The large language model is used to perform inference and prediction on the training data for each task type, and the prediction accuracy for each task type is calculated. For each task type, when the prediction accuracy of the task type is higher than the target accuracy, the sampling weight of the task type is reduced; when the prediction accuracy of the task type is lower than the target accuracy, the sampling weight of the task type is increased. Based on the adjusted sampling weights, the training data pairs for each task type are resampled to generate a training dataset with balanced difficulty.

7. The training data construction method according to claim 1, characterized in that, The method for embedding the task guidance identifier includes: Obtain a unique identifier for each task type, the identifier containing semantic description information of the corresponding task type; A formatted task guidance identifier is inserted at the beginning of the text sequence of the training data. This task guidance identifier serves as an explicit feature in subsequent model training, guiding the model to activate the corresponding reward calculation branch for different task types.

8. The training data construction method according to claim 1, characterized in that, The long text document to be processed undergoes iterative semantic segmentation to generate semantically complete text blocks, including: Automated segmentation is performed based on the structured tags of the long text document to obtain an initial set of text blocks; The semantic integrity of each text block in the initial text block set is scored by calling the large language model, and the semantic integrity score of each text block is obtained. The segmentation boundaries are adjusted based on the score, and the automated segmentation and scoring steps are repeated until the semantic integrity score reaches the preset convergence standard, generating a semantically complete text block.

9. A training data construction apparatus, characterized in that, include: The generation module is used to perform iterative semantic segmentation on the long text document to be processed, generate semantically complete text blocks, and generate training data pairs containing questions, contextual text blocks, and standard answers based on the predefined task type and the text blocks; The balancing module is used to infer the training data pairs using a large language model, obtain the prediction accuracy for different task types, and dynamically adjust the sampling weights of the training data pairs corresponding to each task type based on the prediction accuracy to generate a training dataset with balanced difficulty. A construction module is used to embed task guidance identifiers corresponding to task types into the data of the training dataset to obtain the final training data for model training; wherein, the task guidance identifiers are used to enable the model to identify different task types and generate differentiated reward signals.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the training data construction method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the training data construction method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the training data construction method as described in any one of claims 1 to 8.

Images