Text data generation method, electronic equipment, readable storage medium and program product

Abstract

The invention discloses a text data generation method, electronic equipment, a readable storage medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: inputting a source document and a global semantic guidance cue word into an evaluation data generation model, generating semantic constraint information summarizing a source document theme and a logic structure under the prompt of the semantic guidance cue word, and performing block processing on the source document based on the semantic constraint information, generating a prompt word according to semantic constraint information and evaluation data generated by each text block, identifying semantic correlation between the initial evaluation data and the source document when each initial evaluation data is generated, and performing semantic duplicate removal processing; and executing a text generation task by using the text data generation model passing the evaluation data set test, and generating text data matched with the text generation task. According to the method, the problems of high manual dependency, incomplete semantic coverage and deviation of the generated content from the original text in related technologies can be solved, and the accuracy of the generated text data is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a text data generation method, electronic device, readable storage medium, and program product. Background Technology

[0002] Automated text generation from source documents is a type of natural language processing task, and the accuracy of text generation is related to the effectiveness of the dataset used during training. Currently, related technologies utilize large language models to automatically generate evaluation datasets to assess the performance of text generation models used for this task. However, this approach suffers from high reliance on human intervention, incomplete semantic coverage, and deviations from the original text in the generated content. Consequently, the performance of the final text generation model fails to meet user needs, resulting in low accuracy in text generation tasks.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] This application provides a text data generation method, electronic device, computer-readable storage medium, and computer program product, which solves the problems of high dependence on manual labor, incomplete semantic coverage, and deviation of generated content from the original text, and can significantly improve the semantic consistency, content relevance, and structural rationality of the final generated text data.

[0005] To solve the above-mentioned technical problems, this application provides the following technical solution: This application provides a method for generating text data, including: The model is generated by inputting the source document and global semantic guidance prompts into the evaluation data. The evaluation data generation model, guided by the semantic guidance prompts, generates semantic constraint information summarizing the theme and logical structure of the source document. Based on the semantic constraint information, the source document is segmented into blocks, and evaluation data generation prompts are generated according to the semantic constraint information and each text block. An initial evaluation dataset is generated based on the evaluation data generation prompts. For each initial evaluation dataset generated, the semantic relevance between the initial evaluation dataset and the source document is identified, and semantic deduplication is performed on the initial evaluation datasets that satisfy the semantic relevance to obtain the evaluation dataset. Using the text data generation model tested on the evaluation dataset, a text generation task is performed to generate text data that matches the text generation task.

[0006] This application also provides an electronic device, including a memory and a processor, wherein the processor is used to implement the steps of the above-described text data generation method when executing a computer program stored in the memory.

[0007] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described text data generation method.

[0008] Finally, this application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described text data generation method.

[0009] The advantage of the technical solution provided in this application lies in that, by inputting the source document and global semantic guidance prompts into the evaluation data generation model, it first generates semantic constraint information summarizing the document's theme and logical structure. This allows for a holistic understanding and extraction of the core content and structural relationships of the source document, thus providing global semantic guidance for the subsequent generation process and effectively preventing the generated content from deviating from the original text's main idea. Furthermore, based on the semantic constraint information, the source document is segmented into blocks, ensuring that each text block achieves comprehensive content coverage while maintaining semantic coherence. Combining the semantic constraint information and each text block, targeted evaluation data and prompts are generated, ensuring that the generated initial evaluation dataset can evenly cover all parts of the document. The content is semantically relevant; then, semantic relevance is identified for each generated initial evaluation data point to ensure that each evaluation data point originates from or is closely related to the original text content, thereby significantly reducing the proportion of irrelevant or erroneous data caused by model illusion. At the same time, semantic deduplication is performed on the evaluation data that meets the semantic relevance requirements to automatically identify and remove duplicate content items, thereby improving the diversity and information density of the final evaluation dataset. Finally, the text data generation model, which has been fully tested and optimized through this evaluation dataset, is used to perform various text generation tasks, improving the performance of the text generation model, and thus significantly improving the semantic consistency, content relevance, and structural rationality of the final generated text data.

[0010] Furthermore, this application also provides corresponding electronic devices, computer-readable storage media, and computer program products for implementing the text data generation method, further making the method more practical. The electronic devices, computer-readable storage media, and computer program products have corresponding advantages.

[0011] The technical features mentioned above, those to be mentioned below, and those shown individually in the accompanying drawings can be arbitrarily combined, as long as the combined technical features are not contradictory. All feasible combinations of features are the technical content explicitly described in this application. Any one of the multiple sub-features contained in the same statement can be applied independently, without necessarily being applied together with other sub-features.

[0012] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating a text data generation method provided in this application; Figure 2 A flowchart illustrating another text data generation method provided in this application; Figure 3 A structural framework diagram of an exemplary embodiment of the text data generation apparatus provided in this application; Figure 4 This is a structural diagram of an exemplary embodiment of the electronic device provided in this application. Detailed Implementation

[0015] To enable those skilled in the art to better understand the technical solutions of this application, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. The terms "first," "second," "third," "fourth," etc., used in the specification and the aforementioned drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. The term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments.

[0016] In the field of Natural Language Processing (NLP), automatically generating text data, such as question-answer pairs and quizzes, from user-input documents is a widely used NLP task in daily work and life. Text data generation tasks are typically performed using pre-trained models. For clarity, this model is defined as a text data generation model. The accuracy of text data generation is related to the effectiveness of the dataset used during the training of the text data generation model. The dataset used in the training of the text data generation model includes a training set, a validation set, and a test set. The training set is used to directly train the model. The validation set is used to monitor model performance during training, adjust hyperparameters (such as the learning rate), and stop training early to prevent overfitting. The test set is the final evaluation dataset, used after all model design and training are completed to provide the final performance evaluation results, representing the model's true level.

[0017] Currently, related technologies utilize large language models to automatically generate evaluation datasets. For example, one related technology employs a block-based processing method with fixed rules, segmenting and extracting structured data from documents using predefined patterns, such as extracting data elements from standard documents. The goal of this method is to extract information from known structures, rather than generating new and diverse evaluation questions. After block segmentation, it only determines whether the data is the target data, discarding mismatched blocks directly, resulting in low document content utilization and failing to meet the requirement of broad document content coverage in evaluation sets. Its block segmentation and extraction process lacks understanding and application of the overall document theme and logical structure. Each text block is processed in isolation, lacking an integration mechanism for the overall document logic and theme. This leads to generated content focusing only on local information while ignoring the global context, resulting in final generated data that deviates from the core idea or becomes semantically one-sided due to being detached from the full text context. Another related technology focuses on building a retrieval-enhanced knowledge base, achieving efficient retrieval and supporting external query responses through block segmentation and graph construction. Its segmentation and indexing optimizations aim to efficiently retrieve answers to random external queries, rather than to proactively and systematically generate a comprehensive evaluation set covering the document's internal knowledge. This passive response model cannot perform a traversal scan of the document content, easily overlooking key semantic units. Furthermore, the generation process is driven by random queries, lacking a systematic exploration of the document's internal structure, resulting in uneven distribution of evaluation questions and poor semantic consistency.

[0018] It is evident that when automatically generating evaluation datasets, related technologies suffer from contradictions such as local extraction versus global coverage, passive retrieval versus active generation, and mismatch between fixed segmentation and semantic coherence. These contradictions lead to problems such as incomplete coverage, semantic bias, and high repetition rates in the generated datasets, requiring manual intervention for correction and reducing the reliability and efficiency of the automated process.

[0019] In light of this, this application overcomes local generation bias by establishing global semantic constraints, addressing the isolated, segmented processing of documents in related technologies. By first generating a guiding global semantic framework, it ensures that subsequent evaluation questions generated for each text block are constrained by the overall theme and logical structure of the document, fundamentally resolving the issue of generated content deviating from the core ideas of the original text. Furthermore, through an active, traversal document processing workflow, it achieves balanced content mining, resolving the coverage blind spots caused by passive retrieval. By combining a semantically understanding-based intelligent segmentation method, it ensures a systematic and comprehensive scanning and transformation of document content, efficiently generating diverse evaluation questions covering the entire document. Further, it enhances dataset usability through built-in automated quality control: integrating automated verification, deduplication, and filtering steps into the generation process, it instantly identifies and removes unacceptable or duplicate questions, significantly reducing noise and redundancy in the generated results. This greatly reduces the workload of subsequent manual screening and correction, improving the usability of the final evaluation dataset.

[0020] The various non-limiting embodiments of this application are described in detail below with reference to the accompanying drawings and specific embodiments. First, please refer to... Figure 1 According to the text data generation method provided in this application, it can be implemented as a computer program product, installed and run on various node devices involved in natural language tasks, such as user terminal devices and servers, to generate performance evaluation datasets for the text data generation model used in the text data generation task. In some embodiments of the method, the method includes the following steps: S101: Input the source document and global semantic guidance prompts into the evaluation data to generate the model.

[0021] The source document refers to the original textual material containing the knowledge to be mined, such as technical standards, papers, reports, and manuals. It supports mixed formats of text, code, and tables, and can also be extended to multimodal scenarios with images and text. Global semantic guidance prompts are pre-designed text instructions that guide the large language model to understand the document as a whole. For example: "Please summarize the core theme, chapter logic, and key entity relationships of this document." These global semantic guidance prompts guide the evaluation data generation model to mine the core information of the source document, enabling the model to focus on the document's theme, logical structure, and key entity relationships. The evaluation data generation model is any large language model (such as GPT-4 or ChatGLM) with natural language understanding and generation capabilities. It is configured to perform subsequent tasks such as semantic constraint information generation, text chunking, evaluation data generation, and relevance verification. Semantic constraint information is a set of core information summarizing the theme and logical structure of the source document, providing global guidance for subsequent chunking and evaluation data generation, ensuring that the generated content does not deviate from the original text's main idea. The initial evaluation dataset is a collection of evaluation data initially generated by the evaluation data generation model based on semantic constraints and text blocks. It may contain irrelevant, repetitive, or other non-compliant text data. Semantic relevance refers to the degree of association between the initial evaluation data and the content of the source document, that is, whether the answer in the evaluation data can be directly or indirectly derived from the source document.

[0022] In this step, the evaluation data generation model receives the source document and global semantic guidance prompts sent by the user, and then outputs the final evaluation dataset. In this end-to-end implementation, semantics is integrated from beginning to end, coupled with automated quality control functions, solving the illusion and duplication problems of large-scale model-generated evaluation sets, and significantly improving the consistency and diversity of the dataset with the original text. The process of the evaluation data generation model generating the evaluation dataset is as follows: Under the guidance of semantic prompts, the evaluation data generation model generates semantic constraint information summarizing the theme and logical structure of the source document; based on the semantic constraint information, the source document is segmented; and evaluation data generation prompts are generated according to the semantic constraint information and each text segment; an initial evaluation dataset is generated based on the evaluation data generation prompts; for each initial evaluation dataset generated, the semantic relevance between the initial evaluation dataset and the source document is identified; and semantic deduplication is performed on the initial evaluation datasets that satisfy the semantic relevance, resulting in the final evaluation dataset.

[0023] In this embodiment, the evaluation data generation model first analyzes the source document based on global semantic guidance prompts, outputting structured semantic constraint information. This information summarizes the document's global theme and logical framework. Then, guided by this information, the evaluation data generation model segments the complete source document into multiple smaller text blocks. Each text block is a continuous segment of the document. Next, the evaluation data generation model dynamically generates a targeted evaluation data generation prompt, such as: "Please generate a single-choice question based on the following text block and the document's overall theme 'Implementation of XX Standard'," driving the model to generate preliminary initial evaluation data (such as question-answer pairs or multiple-choice questions). This step acknowledges and allows the model to generate a certain percentage of substandard questions (such as irrelevant or duplicate questions). For each initial evaluation data point generated, semantic relevance is immediately identified to determine whether the initial evaluation data truly originates from or can be derived from the original text. Questions that pass the verification are collected and semantic deduplication is performed, i.e., highly semantically repetitive questions are removed, ultimately forming a high-quality, diverse evaluation dataset for output.

[0024] S102: Using a text data generation model tested on an evaluation dataset, perform a text generation task to generate text data that matches the text generation task.

[0025] A text data generation model is a model designed to perform specific text generation tasks, specifically models used to complete actual text generation tasks (such as intelligent question answering, report writing, and summary generation). During the training or evaluation phase, this model is tested and optimized using the evaluation dataset generated in the aforementioned steps, thereby improving its performance. Finally, this optimized model is used to process the user's text generation task, outputting high-quality text data. Of course, this embodiment can be used not only for constructing evaluation datasets but also for constructing large model training datasets, QA question-answering pairs, etc.

[0026] In the technical solution provided in this application embodiment, by inputting the source document and global semantic guidance prompts into the evaluation data generation model, semantic constraint information summarizing the document's theme and logical structure is first generated. This enables the overall understanding and extraction of the core content and structural relationships of the source document, thereby providing global semantic guidance for the subsequent generation process and effectively preventing the generated content from deviating from the original text's main idea. Furthermore, the source document is segmented based on the semantic constraint information, ensuring that each text block achieves comprehensive content coverage while maintaining semantic coherence. Combining the semantic constraint information and each text block, targeted evaluation data and prompts are generated, ensuring that the generated initial evaluation dataset can evenly cover all parts of the document. The content is semantically relevant; then, semantic relevance is identified for each generated initial evaluation data point to ensure that each evaluation data point originates from or is closely related to the original text content, thereby significantly reducing the proportion of irrelevant or erroneous data caused by model illusion. At the same time, semantic deduplication is performed on the evaluation data that meets the semantic relevance requirements to automatically identify and remove duplicate content items, thereby improving the diversity and information density of the final evaluation dataset. Finally, the text data generation model, which has been fully tested and optimized through this evaluation dataset, is used to perform various text generation tasks, improving the performance of the text generation model, and thus significantly improving the semantic consistency, content relevance, and structural rationality of the final generated text data.

[0027] Based on the above embodiments, considering that traditional fixed-block segmentation methods ignore the semantic structure of documents and only cut text blocks to a fixed length, resulting in incomplete semantics and logical breaks in the text blocks, which in turn affects the quality of subsequent evaluation data generation; at the same time, the block size is mismatched with the document complexity, either causing difficulties for the model to process due to excessively large blocks, or causing semantic fragmentation due to excessively small blocks. This embodiment also provides an exemplary implementation method for segmenting source documents based on semantic constraint information, which may include the following: The complexity information of the source document is determined based on the logical structure of the semantic constraint information. The block size is determined based on the complexity information, and the overlapping area is determined based on the block size. The semantic boundaries of the source document are determined based on the document structure in the logical structure of the semantic constraint information, and each semantic boundary is used as the block position. The source document is cut based on the block size, overlapping area, and block position, and the cut text blocks are matched with the document structure.

[0028] Complexity information is a quantitative or qualitative indicator determined based on the source document's breadth of topics, logical hierarchy, content density, document length, structural hierarchy, and content depth, used to adapt to different chunking strategies. Chunk size, the length of a text block, is measured in tokens and dynamically adjusted according to document complexity to ensure that chunks can carry complete semantics while facilitating model processing. For example, if the model determines the document's logic is complex, a smaller chunk size (e.g., 512 tokens) is used; otherwise, a larger size (e.g., 2048 tokens) is used. Overlapping areas refer to shared content between adjacent text blocks, used to prevent the fragmentation of complete semantic units at the segmentation point, avoiding semantic breaks caused by chunking. Semantic boundaries are naturally occurring semantic segmentation points in the source document, such as chapter titles, paragraph ends, and list item ends. These positions are ideal for chunking the document, maximizing the semantic integrity within each text block and ensuring that chunking conforms to the document's logical structure. Chunking positions are specific chunking points determined based on semantic boundaries, ensuring that chunking remains consistent with the document structure.

[0029] For example, based on the generated semantic constraint information, its logical structure is analyzed to determine the complexity information of the source document, such as the number of topics involved, chapter levels, and density of technical terms. The chunk size is dynamically adjusted according to the complexity information: for documents with lower complexity, a larger chunk size is set (e.g., 2048 tokens); for documents with higher complexity, a smaller chunk size is set (e.g., 512 tokens), ensuring that the chunks fully preserve semantics while adapting to the model's processing capabilities. A preset overlap coefficient (e.g., 0.25) is obtained, and the overlap area size is determined by multiplying the chunk size by the preset overlap coefficient. For example, when the chunk size is 1024 tokens, the overlap area size is 256 tokens. The document structure in the semantic constraint information's logical structure is analyzed to identify the semantic boundaries of the source document, including H1-H6 headings, paragraph endings, list item endings, etc. These semantic boundaries are used as chunking locations. Each segmented text block is matched with the document structure to ensure that each text block corresponds to a specific chapter or logical unit of the document, maintaining semantic coherence.

[0030] As shown above, the dynamically adjusted block size in this embodiment is adapted to the document complexity, avoiding semantic fragmentation or processing difficulties caused by fixed blocks, and improving the rationality of block segmentation. The selection of block positions based on semantic boundaries ensures that the blocks conform to the document's own logical structure, reducing semantic breaks and laying the foundation for generating high-quality evaluation data. The setting of overlapping areas further ensures semantic coherence, avoiding the omission of key semantic information due to block segmentation, and improving the utilization rate of document content.

[0031] Considering that traditional block segmentation methods lack continuity and tend to cut at non-semantic boundaries, resulting in incomplete semantics in text blocks; furthermore, the segmentation process does not consider the relationship between adjacent text blocks, easily omitting key semantic information across text blocks and affecting the comprehensiveness of evaluation data. Therefore, this embodiment also provides an exemplary segmentation method, which may include the following: The window size of the sliding window is determined by the size of the blocks; the sliding window is used to traverse the source document, and overlapping areas are set during the sliding process, and the source document is divided at each block position.

[0032] The sliding window is a dynamic cutting tool used to traverse the document. It moves within the document at a fixed window size to achieve continuous cutting. The window size is the length of the sliding window, which is the basic length of the text block formed by each cut, and it is consistent with the block size.

[0033] This embodiment uses the block size as the window size of the sliding window. The sliding window traverses the source document. During the sliding process, the sliding step size of the window is controlled according to the calculated overlapping area size, so that there is partial overlap between adjacent windows. When the boundary of the sliding window reaches the predetermined semantic boundary (i.e., the block position), the segmentation operation is performed at that position, and the content within the window is output as a text block.

[0034] As can be seen from the above, this embodiment uses a sliding window combined with overlapping areas to cut text blocks, which can systematically traverse the entire document to avoid omissions, and can also alleviate the loss of edge information that may be caused by cutting through overlapping, thus further ensuring the comprehensiveness of content extraction and the continuity of context.

[0035] Considering that a fixed-size overlapping region cannot adapt to blocks of different sizes, when the block size is large, the fixed overlapping region may not cover the key semantically related parts; when the block size is small, the fixed overlapping region may lead to too much duplicate content, affecting processing efficiency. Based on this, this embodiment also provides an implementation process for determining the overlapping region according to the block size, which may include the following: Obtain the preset overlap coefficient; determine the size of the overlapping area based on the product of the block size and the preset overlap coefficient.

[0036] The preset overlap coefficient is a fixed ratio coefficient determined based on a large amount of experimental data or prior knowledge. It is used to dynamically calculate the size of the overlapping area, taking into account both semantic coherence and processing efficiency. In this embodiment, the preset overlap coefficient can be, for example, 0.25.

[0037] In this embodiment, a preset overlap coefficient can be obtained first, which is, for example, a value between 0.1 and 0.3 (such as 0.25). Then, the product of the block size and the preset overlap coefficient is calculated, and the product result is determined as the size of the overlapping region. For example, if the block size is 1024 words and the preset overlap coefficient is 0.25, then the size of the overlapping region is 256 words. By combining a fixed coefficient with a dynamic block size, the size of the overlapping region can adaptively change with the granularity of the document processing unit, achieving a good balance between ensuring semantic continuity and controlling computational redundancy.

[0038] In this application, the source document switching text block is a semantically preserving document segmentation method. It employs a semantic boundary-based sliding window algorithm, adaptively adjusting the window size according to document complexity; the window size is typically 512-2048 tokens. Boundary alignment rules are used: priority is given to segmenting at headings (e.g., H1-H6), paragraph ends, and list item ends, with dynamic overlapping areas: overlap size = window size × 0.25. Introducing semantic boundary alignment and dynamic overlap at the segmentation stage provides semantically complete processing units for subsequent generation and verification, serving as a crucial prerequisite for ensuring the effectiveness of the entire process. Through overlapping segmentation and semantic global understanding verification, the problem of semantic bias in traditional methods is solved.

[0039] Considering that inaccurate global semantic extraction of documents can lead to generated evaluation data that only focuses on local information, ignoring the overall logic and core theme of the document, resulting in semantic bias and one-sidedness, and failing to fully reflect the document's knowledge system, this embodiment also provides an implementation method for generating semantic constraint information summarizing the source document's theme and logical structure under the guidance of semantic prompts, which may include: Guided by semantic prompts, determine the topic of the source document; guided by semantic prompts, determine the logical structure of the chapters in the source document; guided by semantic prompts, determine the content of the target entities in the source document and the relationship network between the target entities; based on the topic, the logical structure of the chapters, the content of the target entities, and the relationship network between the target entities, generate semantic constraint information.

[0040] Here, "topic" refers to the core discussion object of the source document, a high-level summary of the document's content, such as "Implementation requirements of a certain industry safety standard." "Chapter logical structure" refers to the logical connections and order of the various chapters in the source document, such as "Background introduction → Specification requirements → Implementation steps → Acceptance criteria." "Target entity content" refers to the core information units in the document, such as key concepts, terms, and data, including key concepts, people, locations, terms, standard numbers, scope of application, and technical parameters mentioned in the source document. "Relationship network between target entities" refers to the relationships between target entities, including subordinate, causal, constraint, and comparative relationships, such as "A specific standard number → Scope of application: Medical equipment → Core requirements: Safety performance."

[0041] In this embodiment, the complete source document is input into the evaluation data generation model, and a semantic summary containing at least three levels is generated under the guidance of global semantic guidance prompts: The first layer is the global core theme (e.g., this document discusses the implementation requirements of a certain standard); the second layer is the chapter logical structure (e.g., Chapter 1: Background → Chapter 2: Specification Requirements → Chapter 3: Implementation Guidelines); the third layer is the key entities and their relationship network (e.g., specific standard number → scope of application: medical equipment → core requirements: safety performance). This embodiment uses global semantic guidance as a pre-constraint for the generation process, improving the relevance of the subsequently generated evaluation questions to the original text's main theme from an average of 0.65 to 0.9 (experimental data).

[0042] Considering that large language models are prone to generating semantically repetitive content when generating evaluation data, resulting in high dataset redundancy and low information density, this not only consumes storage resources but also affects the training effect of text data generation models and increases manual screening costs. Therefore, this embodiment also provides an exemplary implementation method for semantic deduplication processing of the initial evaluation dataset that meets semantic relevance, which may include the following: If the target initial evaluation data corresponding to the current text block satisfies semantic relevance, then the target initial evaluation data is converted into a target semantic vector. Based on the clustering or similarity calculation results of the target semantic vector and the semantic vectors of the current evaluation data that already satisfy semantic relevance, if there is evaluation data that semantically overlaps with the target initial evaluation data, then the target initial evaluation data is deleted. If there is no evaluation data that semantically overlaps with the target initial evaluation data, then the target initial evaluation data is added to the current evaluation dataset as evaluation data. If all text blocks of the source document have been processed, then the current evaluation dataset is aggregated and formatted, and output as the evaluation dataset.

[0043] The target initial evaluation data consists of the initial evaluation data generated from the current text block and passed semantic relevance verification. The target semantic vector is a high-dimensional vector converted from the text content (question stem-answer) of the target initial evaluation data, which can be used to quantify the semantic features of the text. For example, the semantic vector can be a numerical vector mapped to a high-dimensional space by a text embedding model (such as BERT or Sentence-Transformer) to represent its semantics. Clustering or similarity calculation refers to analyzing the similarity between the target semantic vector and the semantic vectors of existing evaluation data through algorithms to identify semantically repetitive content. Aggregation and formatting processing involves organizing and integrating all quality-controlled evaluation data into a unified format to form a structurally standardized and easy-to-use final dataset.

[0044] The entire process in this embodiment can be a loop, executing the following steps for each text block and its generated initial evaluation data (taking questions as an example): If the target initial evaluation data corresponding to the current text block satisfies semantic relevance, then the target initial evaluation data is converted into a target semantic vector. Based on the clustering or similarity calculation results of the target semantic vector and the semantic vectors of the currently semantically relevant evaluation data, if there is evaluation data that semantically overlaps with the target initial evaluation data, then the target initial evaluation data is deleted; if not, the target initial evaluation data is added to the current evaluation dataset as evaluation data. That is, the semantic similarity between the new question and the questions in the existing question bank is calculated. If the similarity exceeds a threshold, it is considered semantically duplicated and is removed; otherwise, it is included in the question bank. If all text blocks of the source document have been processed, the current evaluation dataset is aggregated and formatted, and then output. After traversing all text blocks, all the questions that are finally retained are collected, organized (e.g., standardized format, sorted), and output as the final evaluation dataset.

[0045] like Figure 2 As shown, to address the issue of initial evaluation data not meeting requirements, the quality control process is embedded into the production process, forming a cyclical production-quality control process, rather than the traditional separate production-quality inspection model: First, relevance verification is performed immediately: For each question-answer pair, along with its source text block, it is submitted again to the large language model for rapid logical judgment to determine whether the answer is supported by the original text. An example prompt is: "Strictly based on the given text, can this question be directly or indirectly derived from it? Is the answer supported by the text? Only answer 'yes' or 'no'." This step automatically and in batches filters out questions generated by illusions or whose answers lack textual support. For questions that pass verification, the model is required to annotate the original text fragments supporting the answer, enhancing the reliability and interpretability of the dataset. For example, 'Chapter 2, Section 1: Performance Requirements'. Finally, semantic deduplication filtering is performed: The text content (question stem-answer) of all questions that pass verification is converted into high-dimensional semantic vectors. Through clustering or similarity calculation, highly semantically repetitive questions are automatically identified and filtered out to ensure the diversity of the final dataset. After traversing all document blocks and completing the above generation-quality control loop, the system aggregates and formats all quality control-passing questions, outputting a ready-to-use, high-quality evaluation dataset.

[0046] As shown above, this embodiment embeds a fully automated quality control closed loop based on large model self-verification and semantic embedding deduplication within the automated generation process of the evaluation dataset. This solution addresses the two major drawbacks of illusion and repetition in content generated by large models. It organically combines a global semantic guidance framework with the quality control closed loop. The former constrains the generation direction upstream, reducing the probability of generating irrelevant content; the latter rigorously filters downstream, ensuring the purity of the final result. Together, they achieve an improvement from uncontrollable generation to controllable, high-quality output.

[0047] To further address the illusion of answers lacking original textual support due to the verification of the correlation between evaluation data and source documents, and the problem that the inability to clearly identify the source of answers affects the credibility and usability of the dataset, this embodiment also provides corresponding solutions, which may include the following: Generate semantic verification prompts to indicate the semantic relevance between the evaluation data and the source document to the model. Input the semantic verification prompts, the initial evaluation data, and the source text block from which the initial evaluation data was generated into the evaluation data generation model. If the evaluation data generation model determines that the initial evaluation data and the source document have semantic relevance, generate annotation prompts to indicate that the evaluation data is supported by the source document. Input the annotation prompts into the evaluation data generation model, and the evaluation data generation model annotates the target document segments in the source document that support the initial evaluation data.

[0048] Semantic verification prompts are instruction texts used to guide the evaluation data generation model in verifying the semantic relevance between the initial evaluation data and the source document, requiring the model to provide a clear yes or no judgment. Annotation prompts are instruction texts used to guide the model in marking specific segments in the source document that support the evaluation data, improving data traceability. Target document segments are text portions in the source document that directly or indirectly support the answers in the evaluation data, such as specific chapters, paragraphs, or sentences.

[0049] In this embodiment, semantic verification prompts are generated to prompt the model to verify the semantic relevance between the evaluation data and the source document. The semantic verification prompts, initial evaluation data, and the source text block from which the initial evaluation data was generated are then input into the evaluation data generation model. For example, a semantic verification prompt might be: "Strictly based on the given text, determine whether this question can be directly or indirectly derived from it? Is the answer supported by the text? Answer only 'yes' or 'no'." If the evaluation data generation model determines that the initial evaluation data and the source document have semantic relevance, then annotation prompts are generated to prompt the model that the evaluation data is supported by the source document. Annotation prompts might be: "Please find the text fragments in the original text that directly support the above answer." The annotation prompts are input into the evaluation data generation model, which then annotates the target document fragments in the source document that support the initial evaluation data. The model will output annotation information similar to "Chapter 2, Paragraph 1" or a specific quoted sentence.

[0050] As shown above, this embodiment addresses the large-scale model illusion problem by allowing the model to verify the reliability of its generated answers based on source text blocks and tracing back to the original text, thus achieving automated and batch filtering of question validity. This significantly reduces the burden of manual verification. Compared to traditional manual annotation, the automated process supports processing millions of token-level documents per day, while increasing traceability and improving the credibility and interpretability of the generated dataset.

[0051] Finally, this embodiment also provides a method for quickly generating a test question bank from a single PDF document to assess trainees' mastery: Input: Staff convert the PDF document to text format, which serves as the source document. Simultaneously, the system's built-in global semantic guidance prompts are automatically loaded.

[0052] Data Processing Flow: The system calls the evaluation data generation model (e.g., GPT-4). The model first reads the entire document and generates semantic constraint information: "Topic: AI security technical requirements; Context: Scope - Terminology - Basic requirements - Detection methods; Key entities: Adversarial examples, data privacy, model robustness...". Then, based on this information, the model intelligently segments this hundreds-of-pages standard document into hundreds of semantically complete text blocks according to chapter and paragraph boundaries. The system traverses each text block. For text blocks about adversarial examples, combined with the global topic, a prompt is generated: "Please generate a true / false question based on the following technical description of 'adversarial examples'." The model generates the question: "Adversarial example attacks can only target image recognition models." Subsequently, embedded quality control is initiated: the model immediately performs a self-check, determining the answer to the question to be "false" based on the original text, and associating it with the supporting paragraphs in the original text. Next, the system converts this question into a semantic vector, compares it with the generated question bank, and retains it after confirming no duplicates.

[0053] Output and Application: After traversing all text blocks, the system outputs a well-formatted, comprehensive evaluation dataset (question bank) with original text references for each question. This question bank is used to test and fine-tune the AI ​​teaching assistant (text data generation model). The optimized assistant shows significantly improved accuracy in answering students' questions about AI safety.

[0054] Final text generation: The student asks the teaching assistant, "How to prevent adversarial example attacks?" Based on the optimized model, the assistant generates an accurate, comprehensive, and standard-compliant text answer.

[0055] As can be seen from the above, this embodiment reduces the work that originally required experts to manually compile the question bank for several weeks to an automated process in hours, while ensuring the quality of the question bank and its high consistency with the source documents, and ultimately effectively improving the level of subsequent text generation services.

[0056] It should be noted that there is no strict order of execution for the steps in this application. As long as they conform to a logical order, these steps can be executed simultaneously or in a certain preset order. Figures 1-2 This is just an illustrative example and does not mean that this is the only possible execution order.

[0057] This application also provides a corresponding apparatus for the text data generation method, further enhancing the practicality of the method. The apparatus can be described from both a functional module perspective and a hardware perspective. The text data generation apparatus provided in this application is described below. This apparatus is used to implement the text data generation method provided in this application. In this embodiment, the text data generation apparatus may include or be divided into one or more program modules. These one or more program modules are stored in a storage medium and executed by one or more processors to complete the text data generation method disclosed in Embodiment 1. The program module referred to in this embodiment is a series of computer program instruction segments capable of performing a specific function, which is more suitable than the program itself for describing the execution process of the text data generation apparatus in the storage medium. The following description will specifically introduce the functions of each program module in this embodiment. The text data generation apparatus described below can be referred to in correspondence with the text data generation method described above.

[0058] From the perspective of functional modules, see Figure 3 , Figure 3 This is a structural diagram of the text data generation apparatus provided in this embodiment under one specific implementation. The apparatus may include: The data input module 301 is used to input the source document and global semantic guidance prompts into the evaluation data generation model.

[0059] The model processing module 302 is used to generate semantic constraint information summarizing the theme and logical structure of the source document under the guidance of semantic guidance prompts, to process the source document into blocks based on the semantic constraint information, and to generate prompts based on the semantic constraint information and each text block; to generate an initial evaluation dataset based on the prompts generated from the evaluation data; and to identify the semantic relevance between the initial evaluation data and the source document for each initial evaluation data generated, and to perform semantic deduplication on the initial evaluation datasets that meet the semantic relevance requirements to obtain the evaluation dataset.

[0060] The text data output module 303 is used to perform a text generation task using a text data generation model tested through an evaluation dataset, in order to generate text data that matches the text generation task.

[0061] For example, in some embodiments of this example, the model processing module 302 can also be used to: determine the complexity information of the source document based on the logical structure of the semantic constraint information, determine the block size based on the complexity information, and determine the overlapping area based on the block size; determine the semantic boundary of the source document based on the document structure in the logical structure of the semantic constraint information, and use each semantic boundary as the block position; cut the source document based on the block size, the overlapping area and each block position, and match each cut text block with the document structure.

[0062] As an exemplary implementation of the above embodiments, the model processing module 302 can also be used to: use the block size as the window size of the sliding window; traverse the source document using the sliding window, set overlapping areas during the sliding process, and segment the source document at each block position.

[0063] As another exemplary implementation of the above embodiments, the model processing module 302 can also be used to: obtain a preset overlap coefficient; and determine the size of the overlapping region based on the product of the block size and the preset overlap coefficient.

[0064] For example, in some other embodiments of this embodiment, the model processing module 302 described above can also be used to: determine the topic of the source document under the prompt of semantic guidance words; determine the chapter logical structure of the source document under the prompt of semantic guidance words; determine the target entity content of the source document and the relationship network between each target entity under the prompt of semantic guidance words; and generate semantic constraint information based on the topic, chapter logical structure, target entity content and the relationship network between each target entity.

[0065] For example, in some other embodiments of this example, the model processing module 302 can also be used to: if the target initial evaluation data corresponding to the current text block satisfies semantic relevance, then convert the target initial evaluation data into a target semantic vector; based on the clustering or similarity calculation results of the target semantic vector and the semantic vectors of the current evaluation data that already satisfies semantic relevance, if there is evaluation data that semantically overlaps with the target initial evaluation data, then delete the target initial evaluation data; if there is no evaluation data that semantically overlaps with the target initial evaluation data, then add the target initial evaluation data as evaluation data to the current evaluation dataset; if all text blocks of the source document have been processed, then aggregate and format the current evaluation dataset and output it as the evaluation dataset.

[0066] For example, in some other embodiments of this embodiment, the model processing module 302 can also be used to: generate semantic verification prompts to prompt the model to verify the semantic relevance between the evaluation data and the source document, and input the semantic verification prompts, the initial evaluation data, and the source text block that generated the initial evaluation data into the evaluation data generation model; if the evaluation data generation model determines that the initial evaluation data and the source document have semantic relevance, then generate annotation prompts to prompt the model to determine that the evaluation data is supported by the source document; input the annotation prompts into the evaluation data generation model, and the evaluation data generation model annotates the target document fragments in the source document that support the initial evaluation data.

[0067] The text data generation device mentioned above is described from the perspective of functional modules. Furthermore, this application also provides an electronic device, which is described from the perspective of hardware.Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. The electronic device includes a memory 40 for storing a computer program; and a processor 41 for executing the computer program to implement the steps of the text data generation method mentioned in any of the above embodiments.

[0068] The processor 41 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 41 may also be a controller, microcontroller, microprocessor, or other data processing chip. The processor 41 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 41 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 41 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, the processor 41 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0069] The memory 40 may include one or more computer non-volatile storage media, which may be non-transitory. The memory 40 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the memory 40 may be an internal storage unit of an electronic device, such as a server hard drive. In other embodiments, the memory 40 may be an external storage device of an electronic device, such as a plug-in hard drive on a server, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Furthermore, the memory 40 may include both internal and external storage units of the electronic device. The memory 40 can be used not only to store application software and various types of data installed on the electronic device, such as code in the process of executing the text data generation method, but also to temporarily store data that has been output or will be output. In this embodiment, the memory 40 is used to store at least the following computer program 401, which, after being loaded and executed by the processor 41, can implement the relevant steps of the text data generation method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 40 may also include an operating system 402 and data 403, and the storage method may be temporary storage or permanent storage. The operating system 402 may include Windows, Unix, Linux, etc. The data 403 may include, but is not limited to, data corresponding to the text data generation results.

[0070] In some embodiments, the aforementioned electronic device may further include a display screen 42, an input / output interface 43, a communication interface 44 (or network interface), a power supply 45, and a communication bus 46. The display screen 42 and input / output interface 43, such as a keyboard, are user interfaces. Exemplary user interfaces may also include standard wired interfaces, wireless interfaces, etc. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a display screen or display unit, used to display information processed in the electronic device and to display a visual user interface. The communication interface 44 may exemplary include wired and / or wireless interfaces, such as a Wi-Fi interface, a Bluetooth interface, etc., typically used to establish communication connections between the electronic device and other electronic devices. The communication bus 46 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0071] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, such as sensors 47 that perform various functions.

[0072] It is understood that if the text data generation method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes, but is not limited to, various media capable of storing program code, such as: USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, registers, hard disks, multimedia cards, card-type memories (such as SD or DX memories), magnetic memories, removable disks, CD-ROMs, magnetic disks, or optical disks. Based on this, this application also provides a non-volatile storage medium storing a computer program, which, when executed by a processor, performs the steps of the text data generation method as described in any of the above embodiments.

[0073] It is understood that if the text data generation method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, the computer software product may not need to be stored in a physical storage medium. For example, it can be directly transmitted to a computer or other device with information processing capabilities via a wired or wireless network to execute all or part of the steps of the methods in the various embodiments of this application. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, or all or part of the technical solution, can be embodied in the form of a software product. Based on this, this application also provides a computer program product, which stores a computer program, and when the computer program is executed by a processor, it performs the steps of the text data generation method as described in any of the above embodiments.

[0074] The foregoing has provided a detailed description of a text data generation method, electronic device, computer-readable storage medium, and computer program product provided in this application. The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Whether the units and algorithm steps of the various examples described in the disclosed embodiments are executed in electronic hardware or computer software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, and such implementations should not be considered beyond the scope of this application. Several improvements and modifications can be made to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for generating text data, characterized in that, include: Input the source document and global semantic guidance prompts into the evaluation data to generate the model; The evaluation data generation model generates semantic constraint information summarizing the theme and logical structure of the source document under the guidance of the semantic guidance words. Based on the semantic constraint information, the source document is divided into blocks, and evaluation data and prompt words are generated according to the semantic constraint information and each text block. Based on the evaluation data, prompt words are generated to create an initial evaluation dataset. For each initial evaluation dataset generated, the semantic relevance between the initial evaluation dataset and the source document is identified, and semantic deduplication is performed on the initial evaluation datasets that meet the semantic relevance requirements to obtain the evaluation dataset. A text generation task is performed using a text data generation model tested on the evaluation dataset to generate text data that matches the text generation task.

2. The text data generation method according to claim 1, characterized in that, The source document is segmented based on the semantic constraint information, including: The complexity information of the source document is determined based on the logical structure of the semantic constraint information, the block size is determined based on the complexity information, and the overlapping area is determined based on the block size. The semantic boundaries of the source document are determined based on the document structure in the logical structure of the semantic constraint information, and each semantic boundary is used as a block position. The source document is segmented based on the segment size, the overlapping area, and the position of each segment, and the resulting text blocks are matched with the document structure.

3. The text data generation method according to claim 2, characterized in that, The source document is segmented based on the block size, the overlapping area, and the position of each block, including: Use the block size as the window size of the sliding window; The source document is traversed using a sliding window, with overlapping areas set during the sliding process, and the source document is segmented at each cutting position.

4. The text data generation method according to claim 2, characterized in that, Determining the overlapping area based on the block size includes: Obtain the preset overlap coefficient; The size of the overlapping area is determined based on the product of the block size and the preset overlap coefficient.

5. The text data generation method according to claim 1, characterized in that, Guided by the semantic prompts, semantic constraint information summarizing the source document's topic and logical structure is generated, including: The topic of the source document is determined based on the semantic guidance prompts. Guided by the semantic prompts, the logical structure of the source document's chapters is determined; Guided by the semantic prompts, the target entity content of the source document and the relationship network between the target entities are determined. Semantic constraint information is generated based on the topic, the logical structure of the chapter, the content of the target entities, and the relationship network between the target entities.

6. The text data generation method according to claim 1, characterized in that, The initial evaluation dataset that satisfies semantic relevance is semantically deduplicated to obtain the evaluation dataset, which includes: If the target initial evaluation data corresponding to the current text block satisfies semantic relevance, then the target initial evaluation data is converted into a target semantic vector; Based on the clustering results or similarity calculation results of the target semantic vector and the semantic vectors of the current evaluation data that already meet the semantic relevance, if there is evaluation data that semantically overlaps with the target initial evaluation data, then the target initial evaluation data is deleted; if there is no evaluation data that semantically overlaps with the target initial evaluation data, then the target initial evaluation data is added to the current evaluation dataset as evaluation data. If all text blocks of the source document have been processed, the current evaluation dataset will be aggregated and formatted, and then output as the evaluation dataset.

7. The text data generation method according to any one of claims 1 to 6, characterized in that, Identifying the semantic relevance between the initial evaluation data and the source document includes: Generate semantic verification prompts to indicate the semantic relevance between the model verification evaluation data and the source document, and input the semantic verification prompts, the initial evaluation data, and the source text block that generated the initial evaluation data into the evaluation data generation model; If the evaluation data generation model determines that the initial evaluation data has semantic relevance to the source document, then it generates annotation prompt words to indicate that the evaluation data is supported by the source document. The annotation prompts are input into the evaluation data generation model, which then annotates target document fragments in the source document that support the initial evaluation data.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the text data generation method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the text data generation method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the text data generation method according to any one of claims 1 to 7.

Images