Data set construction method and device, electronic equipment and storage medium
By using a large language model and human-computer collaborative review method, unstructured text is parsed into structured text, segmented, and thought chains are generated. This solves the problems of time-consuming, labor-intensive, and noisy dataset construction, and achieves efficient and high-quality dataset construction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-04
- Publication Date
- 2026-04-07
AI Technical Summary
Existing technologies for constructing datasets are time-consuming, labor-intensive, inefficient, and noisy, making it difficult to meet the requirements for high quality and consistency.
By using a large language model to generate thought chains and combining it with human-computer collaborative review, unstructured text is parsed into structured text, data tags are used for segmentation and matching, and inference models and domain expert review are combined to generate a high-quality dataset.
It improves the efficiency and quality of dataset construction, ensures the logical consistency of the thought process and the consistency of annotation, reduces noise, and enables the construction of datasets on different topics as needed.
Smart Images
Figure CN121808302A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a dataset construction method, apparatus, electronic device, and storage medium. Background Technology
[0002] In the field of data processing technology, the quality of datasets affects the development of various data consumers, such as model training.
[0003] In related technologies, datasets are constructed using purely manual annotation, by organizing experts in the field to directly read, extract, and write thought chains from the original documents in order to build the dataset.
[0004] However, constructing a dataset using purely manual annotation is time-consuming, labor-intensive, inefficient, and results in a large amount of noise in the dataset. Summary of the Invention
[0005] The purpose of this application is to provide at least one method, apparatus, electronic device and storage medium for constructing a dataset, which can at least solve the problems of time-consuming, labor-intensive, inefficient and noisy dataset construction, and can at least achieve efficient and high-quality dataset construction.
[0006] To address the aforementioned technical problems, at least one embodiment of this application provides a dataset construction method, comprising: acquiring unstructured raw text; parsing the unstructured raw text into structured text; segmenting the structured text according to semantic units to obtain multiple text blocks; acquiring data tags for the structured text; extracting multiple target text blocks matching the data tags from the multiple text blocks; processing the multiple target text blocks using a large language model to generate a thought chain corresponding to the data tags; and performing human-machine collaborative review on the thought chain, wherein the human-machine collaborative review includes using a reasoning model to review the logical consistency and factual accuracy of the thought chain, and, based on the obtained processing results, having a domain expert corresponding to the unstructured raw text review and correct the results to obtain a target thought chain for constructing the dataset.
[0007] By using model generation combined with human-computer collaborative review, the efficiency and quality of constructing thought chains can be greatly improved. Furthermore, since the thought chains in the dataset are all generated by language models, the consistency of annotation can be better guaranteed between different thought chains, reducing the noise in the dataset and further improving the quality of the dataset.
[0008] Furthermore, by "parsing unstructured raw text into structured text", a clear and standardized input is provided for all subsequent processing steps, which fundamentally improves data quality, overcomes the processing difficulties caused by messy raw text formats, reduces the loss of logical information, and enhances the standardization and basic quality of processing.
[0009] Furthermore, by acquiring data tags for the structured text and using these data tags to generate thought chains, the system can extract and construct datasets with different themes or focuses from the same original text according to different task requirements (i.e., different data tags), achieving on-demand construction and overcoming the drawbacks of traditional methods in generating single, generalized datasets.
[0010] Furthermore, by extracting multiple target text blocks that match the data tags and using these multiple target text blocks to generate thought chains, it is ensured that the materials used to generate thought chains are highly relevant core information, eliminating interference from irrelevant text. This directly improves the quality and accuracy of the final generated content, while also reducing the amount of data that the large language model needs to process and improving the efficiency of thought chain generation.
[0011] Furthermore, human-machine collaborative review ensures high-quality datasets by adopting a two-stage model of "machine initial review + expert review". First, the "reasoning model performs logical consistency review" to filter out low-level errors, and then "domain experts review and correct" to handle complex and professional logical problems. This approach not only frees up experts to focus on high-value work, but also ensures the authority and high reliability of the final output (target thinking chain) through human-machine collaboration.
[0012] Furthermore, the overall dataset construction process is not closely tied to specific technical fields, making it easy to use the dataset construction method to build datasets in various technical fields.
[0013] In some optional embodiments, the structured text is in Markdown format.
[0014] Markdown format effectively preserves rich text structures such as headings, lists, and tables. These structures themselves carry important logical hierarchy information, providing a natural and accurate basis for subsequent "segmentation by semantic unit," allowing the segmented text blocks to better maintain semantic integrity. Furthermore, regardless of whether the original text is PDF, DOCX, or other formats, it is uniformly converted to Markdown, greatly simplifying the design of subsequent text processing modules and improving the system's robustness and the breadth of file types it can handle.
[0015] In some optional embodiments, the step of extracting multiple target text blocks matching the data tag from the multiple text blocks includes: converting each text block and the data tag into vectors respectively to obtain multiple text block vectors and data tag vectors; calculating the similarity between the data tag vector and each text block vector to obtain multiple similarities; and, based on the similarity corresponding to each text block, taking the text blocks that meet the similarity requirements as target text blocks to obtain multiple target text blocks matching the data tag.
[0016] By calculating the similarity between the data tag vector and the vectors of each text block, semantic content-based matching is achieved. This enables more accurate identification of text blocks that are semantically related to the "data tag," even if the text blocks do not use the exact same keywords, thus greatly improving the recall and precision of the retrieval.
[0017] In some optional embodiments, the step of processing the plurality of target text blocks using a large language model to generate the thought chain corresponding to the data label includes: inputting the data label and the plurality of target text blocks into a Reranker model, re-sorting the plurality of target text blocks using the Reranker model; and processing the re-sorted plurality of target text blocks using the large language model to generate the thought chain.
[0018] After initially identifying multiple target text blocks, a more refined Reranker model is used to reorder them, prioritizing the most relevant and core text blocks. This provides a focused input for subsequent large language models, guiding them to focus on the most critical information and generating a more accurate and logically clear thought process.
[0019] In some optional embodiments, the thought chain corresponding to the data tag conforms to the domain structure specification corresponding to the unstructured raw text.
[0020] By constraining the thought chain to conform to the domain structure specifications corresponding to the unstructured original text, it is ensured that the automatically generated thought chain is not just fluent text, but structured data that conforms to the inherent logic and norms of a specific domain. This makes the final dataset directly applicable and possesses high practical value and professionalism.
[0021] In some optional embodiments, the method further includes: fine-tuning and optimizing the large language model based on the target thought chain and the thought chain.
[0022] By fine-tuning and optimizing the large language model based on the target thought chain and the thought chain, a positive loop of "application-feedback-learning" is formed. The high-quality "target thought chain" obtained after expert review and correction is used as a supervision signal to fine-tune the model, enabling the model to learn from errors and perform better and better in subsequent generation. This significantly improves the long-term performance and adaptive capability of the system, so that subsequent models can directly generate higher-quality data chains, reduce the workload of domain experts in review and modification, and improve the generation efficiency of the target thought chain.
[0023] At least one embodiment of this application also provides a dataset construction apparatus, comprising: a first acquisition module for acquiring unstructured raw text and parsing the unstructured raw text into structured text; a segmentation module for segmenting the structured text into blocks according to semantic units to obtain multiple text blocks; a second acquisition module for acquiring data tags for the structured text; an extraction module for extracting multiple target text blocks matching the data tags from the multiple text blocks; a generation module for processing the multiple target text blocks using a large language model to generate thought chains corresponding to the data tags; and an auditing module for performing human-machine collaborative auditing on the thought chains, wherein the human-machine collaborative auditing includes using a reasoning model to audit the logical consistency and factual accuracy of the thought chains, and based on the obtained processing results, a domain expert corresponding to the unstructured raw text audits and corrects the results to obtain target thought chains for constructing the dataset.
[0024] In some optional embodiments, the extraction module is used to convert each text block and the data tag into vectors respectively, to obtain multiple text block vectors and data tag vectors; calculate the similarity between the data tag vector and each text block vector to obtain multiple similarities; and, based on the similarity corresponding to each text block, select the text blocks that meet the similarity requirements as target text blocks to obtain multiple target text blocks that match the data tag.
[0025] In some optional embodiments, the generation module is configured to input the data tags and the plurality of target text blocks into the Reranker model, reorder the plurality of target text blocks using the Reranker model, and process the reordered plurality of target text blocks using the large language model to generate the thought chain.
[0026] In some optional embodiments, the apparatus further includes a training module for fine-tuning and optimizing the large language model based on the target thought chain and the thought chain.
[0027] At least one embodiment of this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the dataset construction method described above.
[0028] At least one embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the dataset construction method described above.
[0029] At least one embodiment of this application also provides a computer program product, including a computer program that, when executed by a processor, implements the dataset construction method described above. Attached Figure Description
[0030] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.
[0031] Figure 1 This is a flowchart of a dataset construction method provided in one embodiment of this application; Figure 2 This is a schematic diagram of a dataset construction apparatus provided in another embodiment of this application. Detailed Implementation
[0032] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.
[0033] To facilitate understanding of the embodiments of this application, the relevant content regarding dataset construction will be introduced first.
[0034] The reasoning capabilities of Large Language Models (LLMs) heavily rely on high-quality datasets that contain detailed reasoning steps. However, general-purpose datasets (such as HotpotQA, a general-purpose dataset) primarily focus on factual question answering and lack domain-specific, expert-verified logical reasoning processes.
[0035] The demand for structured data in various professional fields: In fields such as finance, law, and healthcare, there is a need to transform massive amounts of unstructured text (such as financial reports and judgments) into structured data with clear logical hierarchies for model training and the establishment of auditable and traceable AI systems. The core of this process is ETL (Extract, Transform, Load) and high-quality annotation.
[0036] In related technologies, dataset construction schemes include the following: 1. Purely manual annotation Method: Organize domain experts to directly read, extract, and write thought chains from original documents. For example, the construction of the CCI4.0 dataset (a dataset).
[0037] Limitations: Purely manual annotation is a typical labor-intensive ETL method, which is costly (approximately 40 man-hours per thousand data points). Furthermore, due to differences in background and understanding among experts in the field, the consistency of the mental chain annotation is difficult to guarantee (consistency is approximately 68%), resulting in high noise in the dataset.
[0038] 2. Rule-based ETL Method: A predefined set of templates or regular expression rules is used to automatically extract entities and relationships from text, generating structured data. For example, this is used in the data generation of some clinical decision support systems.
[0039] Limitations: Poor generalization ability of rules and low rate of reasonable data generation. When the domain or text style changes, most rules need to be rewritten, resulting in high maintenance costs and an inability to adapt to diverse data sources.
[0040] 3. General model combined with RAG (Retrieval-Augmented Generation) Methods: Generate thought chains directly using large language models (such as GPT4), or supplement them by retrieving fragments from a knowledge base using traditional RAG techniques.
[0041] Limitations: General large models often exhibit "illusions" in specialized fields, generating reasoning that does not conform to the logic of the domain. Traditional RAG only performs simple information retrieval and splicing, and its ETL process is "flat," failing to structure and compress the retrieved information into a hierarchical chain of thought, resulting in unclear logic in the generated chain of thought.
[0042] In addition, traditional ETL methods share the following common problems: 1. Severe Information Loss: When processing raw texts such as legal documents or medical reports, crucial logical chains and contextual information are lost. For example, when processing financial analysis reports, a large number of core reasoning points are lost.
[0043] 2. High cost and poor consistency: Relying on experts to manually annotate thought chains (such as the argumentation process of legal cases) is not only inefficient, but also has low consistency in thought chain annotation due to the lack of unified standards, which seriously affects the quality of the dataset.
[0044] 3. Rigid processes and difficult to migrate: The ETL rules and annotation paradigms designed for specific domains have low reusability, which means that the entire dataset construction process needs to be redesigned every time a new domain is entered.
[0045] To address the aforementioned technical problems, this application proposes a dataset construction method. The implementation details of the dataset construction method in this embodiment are described below. The following content is only for ease of understanding and is not necessary for implementing this solution.
[0046] Example 1: The dataset construction method of this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 1 As shown, it includes S101-S106.
[0047] S101, Obtain the unstructured raw text and parse the unstructured raw text into structured text.
[0048] In some examples, the unstructured raw text can be obtained by having another device send the unstructured raw text to the electronic device executing this method, after which the electronic device completes the acquisition of the unstructured raw text; or by directly constructing the unstructured raw text in the electronic device; or by receiving unstructured raw text input by the user through an input device configured on the electronic device.
[0049] The embodiments of this application do not limit the specific domain of the unstructured raw text. For example, it could be from the fields of bidding and evaluation, law, medicine, finance, etc.
[0050] The embodiments of this application do not limit the specific format of the unstructured raw text. For example, the unstructured raw text may be in PDF format or DOCX format, etc.
[0051] The embodiments of this application do not limit the specific format of the structured text. In some examples, the structured text is in Markdown format.
[0052] Markdown format effectively preserves rich text structures such as headings, lists, and tables. These structures themselves carry important logical hierarchy information, providing a natural and accurate basis for subsequent "segmentation by semantic unit," allowing the segmented text blocks to better maintain semantic integrity. Furthermore, regardless of whether the original text is PDF, DOCX, or other formats, it is uniformly converted to Markdown, greatly simplifying the design of subsequent text processing modules and improving the system's robustness and the breadth of file types it can handle.
[0053] S102, the structured text is divided into blocks according to semantic units to obtain multiple text blocks.
[0054] Regarding how to segment structured text according to semantic units, the embodiments of this application are not limited, and any method of segmenting text according to semantic units can be applied here.
[0055] In some examples, structured text is segmented according to semantic units to obtain multiple text blocks. This may include: recursively segmenting the structured text according to the heading structure to obtain multiple intermediate text blocks; and for each intermediate text block, segmenting the intermediate text block according to natural paragraphs and fixed window separation or list item separation.
[0056] The method for separating natural paragraphs and fixed windows is as follows: when the number of characters in a natural paragraph is no more than a threshold, the entire natural paragraph is treated as a single text block. If the number of characters in the natural paragraph exceeds the threshold, the natural paragraph is divided into segments using a fixed window (with a fixed number of characters, for example, 150 characters, which can be set based on experience). During segmentation, adjacent text blocks overlap by a preset number of characters (for example, 50 characters, which can be set based on experience) to maintain the continuity of the context.
[0057] Using Markdown format for structured text, a structured text file looks like this: "#verdict ## I. Information of the Parties Plaintiff and Defendant's Detailed Letter ##II. Plaintiff's Claims Plaintiff's claims and factual grounds ##III. The Defendant's Defense Defendant's response ##IV. Findings by this Court The facts determined by the court ##V. This court holds that The court's reasoning and legal analysis VI. Judgment The final judgment text.
[0058] Then, by recursively segmenting according to the heading structure in the structured text, the resulting intermediate text blocks are: "Intermediate text block 1: Party information: detailed information of the plaintiff and defendant; " Intermediate text block 2: Plaintiff's claims: Plaintiff's litigation requests and factual grounds; Intermediate text block 3: Defendant's defense: Defendant's response; Intermediate text block 4: This court has ascertained the following facts as determined by the court; Intermediate text block 5: This court holds that: the court's reasoning and legal analysis; Intermediate text block 6: Judgment result: Final judgment text.
[0059] For example, an intermediate text block would look like this: "##IV. The findings of this court" 1. On May 1, 2023, A borrowed money from B and issued an IOU.
[0060] 2. After the loan matured, A repeatedly called B to urge repayment.
[0061] 3. B argued that the loan was for debt repayment, but failed to provide any evidence to prove it.
[0062] Based on the list items, the middle text block can be divided into three text blocks, as follows: "Text block 1: On May 1, 2023, A borrowed money from B and issued a promissory note; Text block 2: After the loan matured, A repeatedly called B to demand repayment; Text block 3: B argues that the loan was for debt repayment, but provides no evidence to prove it.
[0063] Segmenting based on semantic units enables subsequent processing to operate on more refined and meaningful units, laying the foundation for accurate retrieval in S104.
[0064] S103, Obtain data tags for structured text.
[0065] Data tags are used to represent the subject meaning of data and are the targets used to extract target text blocks in subsequent S104. For example, for some structured text, "cloud computing services" can be a data tag so that a thought chain with the subject meaning of "cloud computing services" can be generated based on the data tag.
[0066] Data tags for structured text represent textual content within that text. For example, a text describing influenza contains the textual content corresponding to "cough and fever." Conversely, a legal text does not contain the textual content corresponding to "communication technology."
[0067] In some examples, acquiring data tags for structured text may include: receiving data tags for the structured text input by a user through an input device; or receiving data tags for the structured text sent by other devices.
[0068] By acquiring data labels for the structured text, electronic devices can subsequently extract and construct datasets with different themes or focuses from the same original text according to different task requirements (i.e., different data labels), realizing on-demand construction and overcoming the drawbacks of traditional methods in generating single and generalized datasets.
[0069] S104, Extract multiple target text blocks that match the data labels from multiple text blocks.
[0070] In some examples, extracting multiple target text blocks that match data labels from multiple text blocks may include: converting each text block and data label into vectors respectively to obtain multiple text block vectors and data label vectors; calculating the similarity between the data label vector and each text block vector to obtain multiple similarity scores; and, based on the similarity score corresponding to each text block, selecting the text blocks that meet the similarity requirements as target text blocks to obtain multiple target text blocks that match data labels.
[0071] In some cases, an encoder can be used to convert each text block and data label into a vector.
[0072] The embodiments of this application do not limit how the similarity between the data label vector and each text block vector is calculated. For example, cosine distance, L1 distance, or Euclidean distance can be used to calculate the similarity between the data label vector and each text block vector.
[0073] In some examples, the similarity requirement is that the similarity is greater than a similarity threshold. The embodiments of this application do not limit the specific value of the similarity threshold; for example, the similarity threshold is 90%.
[0074] By calculating the similarity between the data tag vector and the vectors of each text block, semantic content-based matching is achieved. This enables more accurate identification of text blocks that are semantically related to the "data tag," even if the text blocks do not use the exact same keywords, thus greatly improving the recall and precision of the retrieval.
[0075] S105 uses a large language model to process multiple target text blocks and generate thought chains corresponding to data labels.
[0076] Regarding the specific type of large language model, the embodiments of this application are not limited. Any large language model capable of performing reasoning tasks to generate thought chains based on prompts can be applied here. For example, Wenxin Yiyan, DeepSeek, Chat GPT series, etc.
[0077] In some cases, the content of these multiple target text blocks can be used directly to construct a prompt, which is then input into a large language model, which generates a thought chain based on the prompt.
[0078] In some examples, using a large language model to process multiple target text blocks and generate thought chains corresponding to data labels can include: inputting data labels and multiple target text blocks into a Reranker model, using the Reranker model to reorder the multiple target text blocks; and using a large language model to process the reordered multiple target text blocks to generate thought chains.
[0079] After initially identifying multiple target text blocks, a more refined Reranker model is used to reorder them, prioritizing the most relevant and core text blocks. This provides a focused input for subsequent large language models, guiding them to focus on the most critical information and generating a more accurate and logically clear thought process.
[0080] In some cases, the thought chains corresponding to the data labels conform to the domain structure specifications corresponding to the unstructured raw text.
[0081] The embodiments of this application do not limit the domain to which the unstructured raw text belongs. Nor do the embodiments of this application limit the domain structure specifications corresponding to different domains.
[0082] For example, taking the bidding and tendering field as an example, the domain structure specification definition for this bidding and tendering field has multiple fields, including evaluation items, evaluation basis, technical analysis, compliance judgment, and score. Evaluation items describe the review content required in the tender documents; evaluation basis describes the evaluation knowledge referenced for the evaluation items; technical analysis describes the technical analysis of the corresponding content in the tender documents for the evaluation items; compliance judgment describes the compliance conclusion of the tender documents relative to the evaluation items; and score describes the score of the tender documents on the evaluation items. [wu1] To ensure that the thought chain corresponding to the data label conforms to the domain structure specification corresponding to the unstructured original text, the content of the domain structure specification needs to be added when constructing hints based on multiple target text blocks. This will allow the large language model to generate a thought chain that conforms to the requirements of the domain structure specification when generating the thought chain.
[0083] By constraining the thought chain to conform to the domain structure specifications corresponding to the unstructured original text, it is ensured that the automatically generated thought chain is not just fluent text, but structured data that conforms to the inherent logic and norms of a specific domain. This makes the final dataset directly applicable and possesses high practical value and professionalism.
[0084] S106, Perform human-machine collaborative review on the thought chain to obtain the target thought chain used to construct the dataset.
[0085] Among them, human-machine collaborative review includes using reasoning models to review the logical consistency and factual accuracy of the thought chain, and then having domain experts review and correct the results based on the unstructured original text.
[0086] The specific inference model used is not limited in the embodiments of this application; for example, it could be GPT4 or Qwen3-72B, etc. Utilizing a strong inference model to verify the logical consistency and factual accuracy of the generated thought chain is a machine-processed process that can filter out obvious logical breaks or factual errors.
[0087] Domain experts, based on the results of machine processing, can examine more complex, ambiguous, or logical relationships involving deep domain knowledge.
[0088] In some cases, to facilitate review by domain experts, a forward annotation method is used for the thought chain: for each text block, its original document source, version, author and other meta-information are automatically recorded; and a reverse query method is also allowed: reviewers can use the built-in reverse index function to locate the specific paragraph of the original evidence in the unstructured original text from any node in the thought chain.
[0089] By using model generation combined with human-computer collaborative review, the efficiency and quality of constructing thought chains can be greatly improved. Furthermore, since the thought chains in the dataset are all generated by language models, the consistency of annotation can be better guaranteed between different thought chains, reducing the noise in the dataset and further improving the quality of the dataset.
[0090] Furthermore, by "parsing unstructured raw text into structured text", a clear and standardized input is provided for all subsequent processing steps, which fundamentally improves data quality, overcomes the processing difficulties caused by messy raw text formats, reduces the loss of logical information, and enhances the standardization and basic quality of processing.
[0091] Furthermore, by acquiring data tags for the structured text and using these data tags to generate thought chains, the system can extract and construct datasets with different themes or focuses from the same original text according to different task requirements (i.e., different data tags), achieving on-demand construction and overcoming the drawbacks of traditional methods in generating single, generalized datasets.
[0092] Furthermore, by extracting multiple target text blocks that match the data tags and using these multiple target text blocks to generate thought chains, it is ensured that the materials used to generate thought chains are highly relevant core information, eliminating interference from irrelevant text. This directly improves the quality and accuracy of the final generated content, while also reducing the amount of data that the large language model needs to process and improving the efficiency of thought chain generation.
[0093] Furthermore, human-machine collaborative review ensures high-quality datasets by adopting a two-stage model of "machine initial review + expert review". First, the "reasoning model performs logical consistency review" to filter out low-level errors, and then "domain experts review and correct" to handle complex and professional logical problems. This approach not only frees up experts to focus on high-value work, but also ensures the authority and high reliability of the final output (target thinking chain) through human-machine collaboration.
[0094] Furthermore, the overall dataset construction process is not closely tied to specific technical fields, making it easy to use the dataset construction method to build datasets in various technical fields.
[0095] In some cases, the large language model can also be fine-tuned and optimized based on the target thought chain and thought chain.
[0096] By fine-tuning and optimizing the large language model based on the target thought chain and the thought chain, a positive loop of "application-feedback-learning" is formed. The high-quality "target thought chain" obtained after expert review and correction is used as a supervision signal to fine-tune the model, enabling the model to learn from errors and perform better and better in subsequent generation. This significantly improves the long-term performance and adaptive capability of the system, so that subsequent models can directly generate higher-quality data chains, reduce the workload of domain experts in review and modification, and improve the generation efficiency of the target thought chain.
[0097] In this application, unstructured raw text is parsed into structured text, and retrieval enhancement generation technology (the processes of S102, S103-S105 can be regarded as the process of retrieval enhancement generation) is used to efficiently segment, vectorize and recall the structured text. This enables the accurate extraction of key information related to "data tags" from redundant text, solving the problem that the text is too long and exceeds the context limit of the large model, resulting in the inability to generate thought chains.
[0098] In addition, this application integrates "original text parsing" - "structured text" - "RAG text compression (multiple target text blocks)" - "large language model reasoning" - "thinking chain" into a seamless end-to-end process. This integration not only simplifies the operation steps, but also ensures the efficiency and logical consistency of the entire process from the original document to the final key point.
[0099] Example 2: Another embodiment of this application relates to a dataset construction apparatus. The implementation details of this dataset construction apparatus are described below. The following details are for ease of understanding and are not essential for implementing this solution. A schematic diagram of the dataset construction apparatus in this embodiment can be seen as follows: Figure 2 As shown, the system includes: a first acquisition module 21, used to acquire unstructured raw text and parse it into structured text; a segmentation module 22, used to segment the structured text into blocks according to semantic units to obtain multiple text blocks; a second acquisition module 23, used to acquire data tags for the structured text; an extraction module 24, used to extract multiple target text blocks that match the data tags from the multiple text blocks; a generation module 25, used to process the multiple target text blocks using a large language model to generate thought chains corresponding to the data tags; and an auditing module 26, used to perform human-machine collaborative auditing on the thought chains. The human-machine collaborative auditing includes using a reasoning model to audit the logical consistency and factual accuracy of the thought chains. Based on the obtained processing results, domain experts corresponding to the unstructured raw text audit and correct the results to obtain the target thought chains used to construct the dataset.
[0100] In some optional embodiments, the extraction module 24 is used to convert each text block and data label into vectors respectively, to obtain multiple text block vectors and data label vectors; calculate the similarity between the data label vector and each text block vector to obtain multiple similarities; and, based on the similarity corresponding to each text block, take the text block that meets the similarity requirement as the target text block to obtain multiple target text blocks that match the data label.
[0101] In some optional embodiments, the generation module 25 is used to input data labels and multiple target text blocks into the Reranker model, reorder the multiple target text blocks using the Reranker model, and process the reordered multiple target text blocks using a large language model to generate a thought chain.
[0102] In some alternative embodiments, the apparatus further includes a training module for fine-tuning and optimizing a large language model based on a target thought chain and thought chains.
[0103] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.
[0104] Example 3: Another embodiment of this application relates to an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the dataset construction method in the above embodiments.
[0105] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.
[0106] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.
[0107] Example 4: Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.
[0108] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0109] Example 5: Another embodiment of this application relates to a computer program product, including a computer program. When executed by a processor, the computer program implements the method embodiments described above.
[0110] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.
Claims
1. A method for constructing a dataset, characterized in that, include: Obtain unstructured raw text and parse the unstructured raw text into structured text; The structured text is divided into blocks according to semantic units to obtain multiple text blocks; Obtain data tags for the structured text; Extract multiple target text blocks that match the data tags from the multiple text blocks; The multiple target text blocks are processed using a large language model to generate the thought chain corresponding to the data label; The thought chain is subject to human-machine collaborative review, which includes using a reasoning model to review the logical consistency and factual accuracy of the thought chain. Based on the results, the domain expert corresponding to the unstructured original text reviews and corrects the thought chain to obtain the target thought chain used to construct the dataset.
2. The dataset construction method according to claim 1, characterized in that, The structured text is in Markdown format.
3. The dataset construction method according to claim 1, characterized in that, The step of extracting multiple target text blocks that match the data tags from the multiple text blocks includes: Each text block and the data label are converted into a vector, resulting in multiple text block vectors and data label vectors. Calculate the similarity between the data tag vector and each text block vector to obtain multiple similarity scores; Based on the similarity of each text block, the text blocks that meet the similarity requirements are taken as target text blocks, resulting in multiple target text blocks that match the data labels.
4. The dataset construction method according to claim 1, characterized in that, The process of using a large language model to process the multiple target text blocks and generate the thought chain corresponding to the data label includes: The data labels and the plurality of target text blocks are input into the Reranker model, and the Reranker model is used to reorder the plurality of target text blocks. The large language model is used to process the reordered target text blocks to generate the thought chain.
5. The dataset construction method according to any one of claims 1-4, characterized in that, The thought chain corresponding to the data tag conforms to the domain structure specification corresponding to the unstructured original text.
6. The dataset construction method according to any one of claims 1-4, characterized in that, Also includes: Based on the target thought chain and the thought chain, the large language model is fine-tuned and optimized.
7. A dataset construction apparatus, characterized in that, include: The first acquisition module is used to acquire unstructured raw text and parse the unstructured raw text into structured text; The segmentation module is used to segment the structured text into blocks according to semantic units to obtain multiple text blocks; The second acquisition module is used to acquire data tags for the structured text; An extraction module is used to extract multiple target text blocks that match the data tags from the multiple text blocks; The generation module is used to process the multiple target text blocks using a large language model to generate the thought chain corresponding to the data label; The review module is used to perform human-machine collaborative review of the thought chain. The human-machine collaborative review includes using a reasoning model to review the logical consistency and factual accuracy of the thought chain. Based on the obtained processing results, the domain expert corresponding to the unstructured original text reviews and corrects the results to obtain the target thought chain used to construct the dataset.
8. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the dataset construction method as described in any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the dataset construction method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the dataset construction method according to any one of claims 1 to 6.