Automatic corpus generation method and system based on multi-source structured data

Abstract

The invention provides an automatic corpus generation method and system based on multi-source structured data, and relates to the technical field of data processing. Automatic identification and integration of a multi-source data structure are realized through an adaptive interface adapter, highly-structured and low-structured data are dynamically processed in combination with a context sensing template and a hybrid generation framework, and accuracy and consistency of output corpora are ensured based on a multi-dimensional evaluation mechanism and a term mapping table. The technical problems of difficult multi-source data integration, high manual annotation cost, professional term conversion obstacle and unstable generation quality are effectively solved, and the method has the advantages that the multi-source structured data integration efficiency is improved, the manual annotation cost is reduced, the professional term conversion accuracy is ensured, and the generated corpus quality stability is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and system for automatic generation of multi-source structured data corpora. Background Technology

[0002] With the deepening development of big data and artificial intelligence technologies, the application of structured data in key fields such as healthcare, finance, law, and the military continues to expand. This data exists in a wide range of highly heterogeneous forms, including relational database tables, spreadsheet files, JSON data, and knowledge graphs. However, existing technologies face significant challenges in automatically generating high-quality text corpora from multi-source structured data. Single-model-driven methods, such as those relying on optical character recognition systems or rule-based template generation techniques, are typically optimized for only a single data source type and cannot effectively integrate and process mixed-format inputs. For example, while OCR technology can extract text from images or scanned documents, its ability to recognize technical terms is limited, especially when processing structured data such as database tables or JSON, resulting in semantic gaps or terminology errors in the output. Rule-based methods require the manual writing of numerous customized rules, which is not only costly to develop but also lacks flexibility, making it difficult to adapt to changes in different data structures and domain requirements. For instance, in the financial sector, when generating transaction reports, rules need frequent adjustments to accommodate changes in data format. Manual annotation and integration are particularly critical in specialized scenarios. The healthcare industry needs to convert electronic medical record codes (such as ICD-10) into patient-understandable descriptions, while the legal field requires converting case records into natural language summaries. These tasks rely heavily on professionals investing significant time in the conversion process, resulting in low efficiency and susceptibility to human error, severely hindering the large-scale application of data processing. Existing data processing tools primarily focus on data cleaning and format conversion, such as ETL tools or general conversion software, but lack in-depth text generation capabilities. They cannot dynamically perceive changes in the data source structure, leading to output text that is often awkwardly spliced, semantically incoherent, or inconsistent in terminology. Specifically, existing technologies suffer from the following prominent problems: weak multi-source data integration capabilities; significant differences in data formats across different systems; a lack of a unified adaptive interface for automatically identifying and processing mixed-type documents such as database tables, Excel files, JSON, and knowledge graphs, forcing users to perform multiple manual conversions and interventions; high costs of manual annotation, with professionals expending considerable time and effort converting structured data into natural language, making it difficult to meet the needs of large-scale corpus generation; significant obstacles in understanding professional domain terminology, with industry codes (such as medical E11.9 representing "type 2 diabetes") difficult for existing models to accurately parse into natural language, affecting the readability and accuracy of the output text; and unstable quality of generated text, with existing methods lacking dynamic evaluation mechanisms, resulting in output content that may lack fluency, accuracy, or diversity. For example, template generation is too rigid to handle data variations, while purely generative models are prone to producing semantically inconsistent content. These problems collectively lead to low utilization of multi-source structured data, severely restricting the training of artificial intelligence models and the expansion of practical application scenarios.

[0003] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0004] The purpose of this application is to provide an automatic generation method based on multi-source structured data corpus, which has the advantages of improving the efficiency of multi-source structured data integration, reducing manual annotation costs, ensuring the accuracy of professional terminology conversion, and improving the quality and stability of the generated corpus.

[0005] Firstly, the automatic generation method based on multi-source structured data corpus provided in this application adopts the following technical solution: An automatic generation method based on multi-source structured data corpus includes: Data is extracted from multi-source structured data, including database tables, Excel files, JSON data, and knowledge graphs. The data source structure is automatically identified through an adaptive interface adapter, supporting streaming or batch collection. Context-aware templates are generated based on a domain knowledge base. These templates contain static text and dynamic slots for matching data fields. A framework combining template-driven and generative models is used to generate text corpora: template imputation is used for highly structured data; semantic completion is performed using a Transformer-based generative model for low-structured data. The quality of the generated corpus is evaluated using multi-dimensional metrics, including fluency, accuracy, and diversity, and the generation strategy is dynamically adjusted accordingly. Specialized codes are converted into natural language based on a terminology mapping table, ensuring terminology consistency and accuracy.

[0006] Optionally, in the step of extracting data from multi-source structured data, the adaptive interface adapter adopts a plug-in architecture, dynamically loads dedicated connectors for different data sources, and automatically identifies the data source structure through metadata awareness technology to generate a unified data model. It also includes a load balancing mechanism that dynamically allocates and extracts resources based on data traffic, and integrates a data cleaning module to remove noisy data and duplicate entries, ensuring the quality of input data.

[0007] Optionally, in the step of generating context-aware templates based on a domain knowledge base, the domain knowledge base includes a terminology dictionary, grammar rules, and style guidelines, and identifies entities and relationships in structured data through semantic parsing technology, mapping them to dynamic template slots; It also includes a template optimization mechanism, which compares the generation effects of different templates through A / B testing and adjusts template rules based on user feedback to improve slot matching accuracy.

[0008] Optionally, in the step of generating text corpus using a hybrid framework of template-driven and generative models, the generative model adopts a sequence-to-sequence architecture based on Transformer, introduces an attention mechanism to capture long-distance dependencies, and optimizes the coherence of the generated text through an adversarial training strategy. It also includes a caching mechanism to store high-frequency template results to improve processing efficiency.

[0009] Optionally, in the step of evaluating the quality of the generated corpus through multi-dimensional indicators, the evaluation indicators include fluency indicators, accuracy indicators, and diversity indicators, wherein the fluency indicator is calculated based on the perplexity of the language model, and the accuracy indicator is calculated through semantic consistency comparison. It also includes a human feedback loop, allowing users to rate the generated corpus and retrain the generative model based on the rating results, dynamically adjusting the generation strategy.

[0010] Optionally, in the step of converting professional codes into natural language based on a terminology mapping table, the terminology mapping table is extracted from a standard terminology database and dynamically updated using machine learning methods; It also includes a terminology consistency check module, which uses a rules engine to detect terminology usage conflicts and automatically recommends alternative terms to ensure the accuracy of terminology conversion.

[0011] Optionally, the method is applicable to general structured data processing scenarios, and can be adapted to different domain requirements through configurable parameters, wherein: The multi-source data extraction step supports dynamically loading data source connectors and automatically adjusts the extraction strategy according to the format and structure of the input data to ensure cross-domain data compatibility. The context-aware template generation step integrates a configurable rule engine, allowing users to customize template rules and style guidelines to generate text corpora that adapt to different industry standards; The hybrid generation framework execution steps introduce a domain adaptation mechanism, which fine-tunes the parameters of the generative model to make the text output conform to the semantic and syntactic norms of a specific domain. The terminology standardization process supports switching between multiple terminology databases and dynamically loads the corresponding terminology mapping table according to the application scenario, ensuring the accuracy and consistency of professional code conversion.

[0012] Secondly, this application provides an automatic generation system based on multi-source structured data corpus, comprising: The data extraction module is used to extract data from multi-source structured data, including database tables, Excel files, JSON data and knowledge graphs. It automatically identifies the data source structure through an adaptive interface adapter and supports streaming or batch collection. The template generation module is used to generate context-aware templates based on a domain knowledge base. The templates contain static text and dynamic slots for matching data fields. The corpus generation module is used to generate text corpora using a hybrid framework of template-driven and generative models: template filling is used for highly structured data; semantic completion is performed using a Transformer-based generative model for low-structured data. The strategy generation module is used to evaluate the quality of the generated corpus through multi-dimensional indicators, including fluency, accuracy and diversity, and dynamically adjust the generation strategy. The natural language conversion module is used to convert technical codes into natural language based on a terminology mapping table, ensuring terminology consistency and accuracy.

[0013] Thirdly, this application provides a computer device, the device comprising: a memory and a processor, wherein the processor, when executing computer instructions stored in the memory, performs the method described above.

[0014] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method described above.

[0015] In summary, this application achieves automatic identification and integration of multi-source data structures through an adaptive interface adapter, dynamically processes highly structured and unstructured data by combining context-aware templates and a hybrid generation framework, and ensures the accuracy and consistency of the output corpus by relying on a multi-dimensional evaluation mechanism and a terminology mapping table. It effectively solves the technical problems of difficulty in multi-source data integration, high cost of manual annotation, obstacles to professional terminology conversion, and unstable generation quality. It has the advantages of improving the efficiency of multi-source structured data integration, reducing manual annotation costs, ensuring the accuracy of professional terminology conversion, and improving the stability of the generated corpus quality. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiments of this application; Figure 2 This is a flowchart illustrating the first embodiment of the automatic generation method based on multi-source structured data corpus of this application; Figure 3 This is a structural block diagram of the first embodiment of the automatic generation system based on multi-source structured data corpus of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0018] Reference Figure 1 , Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiments of this application.

[0019] like Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0020] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0021] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and an automatic generation program based on multi-source structured data corpus.

[0022] exist Figure 1 In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in this application can be set in the computer device, and the computer device calls the automatic generation program based on multi-source structured data corpus stored in the memory 1005 through the processor 1001, and executes the automatic generation method based on multi-source structured data corpus provided in the embodiments of this application.

[0023] This application provides an embodiment of an automatic generation method based on multi-source structured data corpus, referring to... Figure 2 , Figure 2This is a flowchart illustrating the first embodiment of the automatic generation method based on multi-source structured data corpus of this application.

[0024] In this embodiment, the automatic generation method based on multi-source structured data corpus includes the following steps: Step S10: Extract data from multi-source structured data, including database tables, Excel files, JSON data and knowledge graphs, and automatically identify the data source structure through an adaptive interface adapter, supporting streaming or batch collection; Step S20: Generate a context-aware template based on the domain knowledge base. The template contains static text and dynamic slots for matching data fields. Step S30: Generate text corpus using a hybrid framework of template-driven and generative models: for highly structured data, use template filling; for low-structured data, use a Transformer-based generative model for semantic completion. Step S40: Evaluate the quality of the generated corpus through multi-dimensional indicators, including fluency, accuracy and diversity, and dynamically adjust the generation strategy; Step S50: Convert specialized codes into natural language based on the terminology mapping table, ensuring terminology consistency and accuracy. In traditional data processing technologies, the integration of multi-source heterogeneous data faces structural obstacles.

[0025] Specifically, due to the diversity of data source formats, including database tables, Excel files, JSON data, and knowledge graphs, the lack of a unified adaptive interface necessitates customized development for data extraction, increasing system complexity and the risk of error propagation. Manual annotation involves extensive rule writing and intervention, significantly extending the data processing cycle and increasing maintenance costs. The conversion mechanism for specialized terminology is flawed; for example, medical codes or industry abbreviations cannot be accurately mapped to natural language expressions, resulting in semantic distortion. The quality assessment of generated text lacks dynamic optimization capabilities, exhibiting fluency fluctuations, accuracy deviations, and insufficient diversity, directly impacting the practicality and reliability of the output corpus.

[0026] For example, in medical information systems, electronic health record databases, Excel files of laboratory test results, JSON data from patient questionnaires, and disease knowledge graphs need to be integrated to generate clinical summary reports. Because current technology cannot automatically identify the structural characteristics of heterogeneous data sources, operators are forced to manually configure extraction rules, leading to frequent interruptions in the data acquisition process and increasing the risk of formatting errors. In the terminology processing stage, ICD-10 code E11.9 failed to be correctly interpreted as "type 2 diabetes," causing professional bias in the report content. The template generation stage relies on manually written static rules, which are difficult to adapt to dynamic changes in data fields, resulting in semantic breaks in the text. Simultaneously, generative models exhibit logical inconsistencies in their output when processing low-structured data, further reducing the readability and clinical reference value of the report. If these problems are not addressed, the automation capabilities of the data processing system will be fundamentally limited, preventing streaming data acquisition and real-time processing, leading to system response delays and decreased resource utilization. Continued reliance on manual annotation will expand the range of operational errors and hinder the expansion of large-scale datasets. The unreliability of terminology conversion may cause misunderstandings in key areas, posing potential risks in scenarios such as medical diagnosis or financial analysis. Uncontrollable fluctuations in the quality of generated text will weaken users' trust in the system, ultimately limiting the deployment feasibility and technology promotion scope of data-driven applications. To address this, this embodiment proposes an automatic text generation method based on multi-source structured data corpora, including: extracting data from multi-source structured data, with data sources including database tables, Excel files, JSON data, and knowledge graphs, and automatically identifying the data source structure through an adaptive interface adapter, supporting streaming or batch collection; generating context-aware templates based on a domain knowledge base, with templates containing static text and dynamic slots for matching data fields; generating text corpora using a hybrid framework of template-driven and generative models: template filling is used for highly structured data; semantic completion is performed using a Transformer-based generative model for low-structured data; evaluating the quality of the generated corpus through multi-dimensional metrics, including fluency, accuracy, and diversity, and dynamically adjusting the generation strategy; and converting specialized codes into natural language based on a terminology mapping table to ensure terminology consistency and accuracy.

[0027] This embodiment describes an automatic data generation method based on multi-source structured data corpora. When extracting data from multi-source structured data, the adaptive interface adapter automatically identifies the data source structure by dynamically determining its organization through analysis of the data source's format characteristics. Specifically, rule-based structure parsing methods can be used, such as pre-defining identification logic based on file extensions or metadata identifiers, or statistical sampling techniques can be used to infer field types and relationships from data samples. This supports streaming or batch acquisition of database tables, Excel files, JSON data, and knowledge graphs. Furthermore, generating context-aware templates based on a domain knowledge base involves creating a text framework containing fixed expressions and variable placeholders. Static text and dynamic slots are used to match data fields. In practical applications, context-aware templates can be manually configured through a user interface, such as specifying the mapping relationship between text paragraphs and data fields, or automatically summarizing common patterns based on historical generation records, such as using clustering algorithms to identify high-frequency data combinations and generate suggested templates. When generating text corpora using a hybrid framework of template-driven and generative models, template filling for highly structured data involves inserting data values ​​into fixed positions within a template according to predefined rules. Specifically, string replacement functions can be used to directly embed field values. For low-structured data, semantic completion using a Transformer-based generative model leverages sequence generation capabilities to handle data uncertainty. Furthermore, semantic completion can be achieved by loading pre-trained language model parameters to perform contextual reasoning, such as fine-tuning for domain adaptation based on open-source models. When evaluating the quality of generated corpora using multi-dimensional metrics, fluency, accuracy, and diversity metrics can be calculated by using language model probability scores to quantify text coherence, or by verifying data accuracy through field value consistency comparisons, such as statistically analyzing the matching ratio between generated content and source data, while simultaneously using vocabulary distribution analysis to assess output diversity. Converting professional codes into natural language based on terminology mapping tables involves establishing a correspondence between codes and standard expressions. Specifically, terminology mapping tables can be quickly queried using locally stored key-value pairs, such as CSV format mapping tables that retrieve natural language descriptions through code indexes, or by dynamically obtaining conversion results through integration with external dictionary services. Therefore, this embodiment effectively solves the problems of difficulty in integrating multi-source heterogeneous data, high cost of manual annotation, inaccurate conversion of professional terms, and unstable quality of generated text by integrating adaptive data extraction, context-aware template generation, hybrid generation framework, multi-dimensional quality assessment, and terminology standardization processing, and achieves the technical effect of automatically creating high-quality text corpora from multi-source structured data.

[0028] This method integrates multi-source data extraction, context-aware template generation, hybrid text generation, dynamic quality assessment, and terminology standardization to achieve a technical workflow for automatically generating high-quality text corpora from heterogeneous data sources. Specifically, the extraction of multi-source structured data is executed by an adaptive interface adapter. This adapter automatically identifies the structural features of data sources such as database tables, Excel files, JSON data, and knowledge graphs, completing streaming or batch acquisition tasks without manual intervention, thereby converting input data of different formats into a unified intermediate representation. Furthermore, the context-aware templates generated based on the domain knowledge base are constructed as a composite structure containing static text and dynamic slots. The dynamic slots are mapped to structured data fields, ensuring that the template content can be dynamically adjusted according to the actual data content. In the text generation stage, the hybrid framework intelligently allocates processing strategies based on the degree of data structuring: highly structured data is directly generated by filling in templates to avoid semantic bias; low-structured data uses a Transformer-based generative model for semantic completion, leveraging an attention mechanism to capture the inherent correlations within the data. The generated corpus then undergoes multi-dimensional quality evaluation. Fluency is quantified using a language model perplexity metric, accuracy is verified based on semantic consistency, and diversity measures the variability of the output text. The evaluation results drive real-time optimization of the generation strategy. Finally, a terminology mapping table converts specialized codes into natural language expressions. This mapping table extracts core terminology relationships from a standard terminology database, ensuring that the conversion process conforms to domain specifications.

[0029] In specific implementation, actual application scenarios in the medical field can be considered. As one implementation method, database table data is extracted from the hospital's electronic medical record system. The adaptive interface adapter automatically parses the field structure of the patient information table, identifying key fields such as name and diagnosis code. The domain knowledge base generates a template containing the static text "Patient [Name]'s diagnosis result is [Disease]", where [Name] and [Disease] serve as dynamic slots to match data fields. For highly structured fields such as name, the template is directly filled to generate a fixed expression. For low-structured fields such as symptom descriptions, the generative model completes the information into natural language such as "persistent cough with fever" based on context. Evaluation of the generated corpus reveals ambiguity in disease terminology conversion. The system immediately calls the terminology mapping table to convert the ICD-10 code E11.9 into the standardized expression "type 2 diabetes," and dynamically increases the context window length of the generative model based on fluency indices falling below a threshold.

[0030] Therefore, this technical solution effectively solves the problem of integrating multi-source heterogeneous data; the adaptive interface adapter eliminates the need for customized development for different data sources; the high cost of manual annotation is overcome, and the automatic generation function of context-aware templates replaces manual rule writing; the inaccuracy of professional terminology conversion is improved, and the terminology mapping table ensures the domain adaptability of the encoding conversion; the unstable quality of generated text is significantly alleviated, and the multi-dimensional evaluation and dynamic adjustment mechanism ensures the reliability and adaptability of the output corpus. The entire process, through the coordinated operation of each stage, achieves automated and standardized data-to-text conversion while maintaining the rigor and operability of the technical implementation.

[0031] In practice, an adaptive interface adapter was proposed to automatically identify the data source structure and support streaming or batch acquisition. However, in its implementation, the adapter lacks dynamic expansion capability, which makes it unable to adapt to new data sources in time. The lack of a load balancing mechanism causes unreasonable resource allocation, which affects processing efficiency. Furthermore, the lack of integrated data cleaning function causes noisy data and duplicate entries to interfere with input quality, which ultimately restricts the automation level of multi-source data extraction and the reliability of subsequent corpus generation.

[0032] To address this, this embodiment further proposes that in the step of extracting data from multi-source structured data, the adaptive interface adapter adopts a plug-in architecture, dynamically loads dedicated connectors for different data sources, and automatically identifies the data source structure through metadata awareness technology to generate a unified data model; it also includes a load balancing mechanism to dynamically allocate extraction resources according to data traffic, and integrates a data cleaning module to remove noisy data and duplicate entries to ensure the quality of input data.

[0033] Among them, plug-in architecture refers to a software design pattern that supports runtime functional expansion. It can be implemented using modular components and standardized interface definitions, aiming to improve the system's ability to adapt to new data sources in real time. Dynamically loading dedicated connectors can be understood as a mechanism that automatically matches processing modules based on the characteristics of input data format. It can be implemented using reflection technology or service registration and discovery patterns, aiming to eliminate manual configuration processes and ensure seamless access to heterogeneous data sources. Metadata awareness technology specifically infers structural characteristics by parsing the metadata information of the data source. It can be implemented using database schema analysis or file header information extraction, aiming to avoid dependence on predefined rules and improve the accuracy of structure recognition. Generating a unified data model refers to the process of converting heterogeneous data into a standardized intermediate representation. It can be implemented using general data objects such as key-value pair sets, aiming to provide a structurally consistent input foundation for subsequent processing. The load balancing mechanism can be understood as a strategy that dynamically optimizes resource allocation based on real-time data traffic. It can be implemented using adaptive scheduling algorithms or resource pooling technology, aiming to avoid idle or overloaded computing resources. The data cleaning module is a component that processes noise and redundant data. It can be implemented using rule matching engines or statistical anomaly detection methods, aiming to ensure the purity and consistency of input data.

[0034] Specifically, the adaptive interface adapter dynamically loads dedicated connectors through a plug-in architecture, and combines metadata-aware technology to analyze the structural characteristics of the data source in real time and generate a unified data model, providing standardized input for subsequent corpus generation. Simultaneously, a load balancing mechanism continuously monitors data traffic fluctuations and dynamically adjusts the resource allocation ratio of the server cluster to ensure stable processing capabilities under high concurrency scenarios. The integrated data cleaning module performs noise filtering and duplicate item detection concurrently during data extraction, ensuring that the input data quality meets the generation requirements. These technical features form an organic whole, enabling the system to automatically adapt to the data source type based on runtime needs, optimize resource utilization efficiency, and eliminate data quality issues at the source, thereby supporting the efficient and automated processing of multi-source structured data.

[0035] In practical implementation: In financial transaction data processing scenarios, when the system receives transaction record tables and market data in JSON format from a relational database, the adaptive interface adapter automatically loads the SQL connector and JSON parsing connector; metadata awareness technology identifies the column definitions of the database tables and the field hierarchy of the JSON data, generating a unified data model; the load balancing mechanism allocates extraction tasks to idle server nodes based on real-time transaction traffic; the data cleaning module removes duplicate transaction entries and abnormal values ​​through preset rules, outputting a clean data stream.

[0036] Through the above solution, this embodiment effectively solves the problem of new data source adaptation delay caused by insufficient dynamic expansion capability of the adapter, optimizes the processing efficiency bottleneck caused by unreasonable resource allocation, and eliminates the interference of noisy data and duplicate entries on input quality, significantly improving the automation level of multi-source data extraction and the reliability of corpus generation process.

[0037] In practice, a method was proposed to generate context-aware templates based on a domain knowledge base to match data fields. However, in this process, the domain knowledge base lacks a systematic integration of terminology dictionaries, grammar rules, and style guidelines, resulting in template generation failing to accurately reflect the semantic norms of the professional domain. At the same time, the entity and relation recognition mechanism in structured data is imperfect, leading to semantic bias when mapped to dynamic slots. Furthermore, the template rules are fixed and lack dynamic optimization capabilities, making it impossible to iteratively adjust based on actual generation results and user feedback, resulting in insufficient slot matching accuracy and affecting the accuracy and adaptability of the generated text.

[0038] To address this, this embodiment further proposes a step for generating context-aware templates based on a domain knowledge base. The domain knowledge base includes a terminology dictionary, grammar rules, and style guidelines. It identifies entities and relationships in structured data through semantic parsing technology and maps them to dynamic template slots. It also includes a template optimization mechanism that compares the generation effects of different templates through A / B testing and adjusts template rules based on user feedback to improve slot matching accuracy.

[0039] In practical applications, a domain knowledge base refers to a database that integrates professional domain knowledge. It can be implemented by using a terminology dictionary to store professional terms and their explanations, grammatical rules to define language structure specifications, and style guidelines to standardize expression habits. Its purpose is to ensure that the generated templates meet the semantic requirements of the domain. Semantic parsing technology can be understood as a technical means of analyzing the semantics of data. Specifically, it can be implemented by named entity recognition or relation extraction algorithms. Its purpose is to deeply analyze entities and relations in structured data and avoid semantic deviations caused by superficial field matching. The template optimization mechanism can be specifically a closed-loop system that dynamically adjusts templates. For example, it can compare the generation quality indicators of different templates through A / B testing and iteratively update template rules based on user rating feedback. Its purpose is to continuously improve the slot matching accuracy and adapt to data variation and changes in user needs.

[0040] Specifically, the solution in this embodiment systematically constructs a domain knowledge base, integrating terminology dictionaries, grammar rules, and style guidelines to form a unified knowledge system, providing a domain-specific foundation for template generation. Simultaneously, it utilizes semantic parsing technology to perform deep semantic analysis on structured data, accurately identifying entities and relationships and mapping them to dynamic template slots, ensuring semantic consistency between data elements and slots. Based on this, a template optimization mechanism quantitatively evaluates the generation effects of different templates through A / B testing and dynamically adjusts template rules based on user feedback, forming a closed-loop process of "generation-evaluation-optimization," thereby collaboratively addressing issues such as incomplete knowledge coverage, inaccurate mapping, and rigid rules.

[0041] As a specific implementation method, in medical data processing scenarios, the domain knowledge base can include a medical terminology dictionary (such as an ICD-10 encoding mapping table), clinical report grammar rules, and medical document style guidelines; semantic parsing technology can identify disease entities (such as "E11.9") and treatment relationships (such as "insulin injection") in electronic medical records and map them to dynamic slots in diagnostic report templates; the template optimization mechanism compares the generation quality of different report templates through A / B testing and adjusts template rules based on doctor rating feedback, such as optimizing the slot filling logic of the diabetes description template, thereby improving slot matching accuracy.

[0042] Through the above solution, this embodiment effectively solves the problems of inaccurate expression caused by incomplete domain knowledge coverage, mapping deviation caused by misjudgment of entity relationships, and insufficient adaptability caused by rigid template rules, ensuring the accuracy of terminology, semantic coherence, and scene adaptability of the generated text in the professional field.

[0043] In some of the above embodiments of this example, a framework that combines template-driven and generative models is proposed to generate text corpora. However, in its implementation, the text generated by the generative model lacks coherence, resulting in semantic breaks or illusory content, and the processing efficiency is low, especially in high-frequency data scenarios where repeated calculations cause resource waste.

[0044] In response, this embodiment further proposes a framework for generating text corpora using a hybrid template-driven and generative model. The generative model adopts a sequence-to-sequence architecture based on Transformer, introduces an attention mechanism to capture long-distance dependencies, and optimizes the coherence of the generated text through an adversarial training strategy. It also includes a caching mechanism to store high-frequency template results to improve processing efficiency.

[0045] Among them, the Transformer-based sequence-to-sequence architecture refers to a deep learning model structure specifically designed for sequence transformation tasks. It can be implemented using an encoder-decoder framework, aiming to efficiently process the semantic sequence of input data and avoid the information decay problem in long text generation of traditional models. The attention mechanism can be understood as a technical component that dynamically calculates the correlation weights of each position in the input sequence. It can be implemented using a multi-head self-attention layer, aiming to focus on key information fragments to maintain long-distance contextual logical consistency. The adversarial training strategy is specifically a method to optimize output quality through the interaction between the generator and the discriminator. It can be implemented by minimizing the difference between the generated text and the real text distribution, aiming to constrain the generated content to be close to the natural language pattern and suppress semantic breaks. The caching mechanism refers to a temporary storage structure for storing the results of high-frequency data processing. It can be implemented using an in-memory database or a hash table, aiming to avoid redundant calculations and improve system resource utilization.

[0046] Specifically, the solution in this embodiment combines a Transformer-based sequence-to-sequence architecture with an attention mechanism, enabling the generative model to dynamically identify key semantic relationships in the input data and ensure the logical coherence of long text generation. At the same time, the adversarial training strategy utilizes the discriminator's real-time feedback on the generated text to continuously optimize the generator's output quality and effectively reduce the generation of hallucinatory content. The caching mechanism directly reuses historical templates to fill results in high-frequency data scenarios, avoiding redundant calculation processes. This complements the dynamic optimization of the generative model, jointly ensuring that the system improves processing efficiency while maintaining text quality.

[0047] As a specific implementation method, the generative model can specifically adopt a fine-tuned T5-base model as the core of the sequence-to-sequence architecture. Its attention mechanism achieves long-distance dependency capture through four sets of parallel self-attention heads. In the adversarial training strategy, the discriminator can specifically adopt a convolutional neural network structure to distinguish the semantic consistency between generated text and real corpus. The caching mechanism can specifically be implemented as a memory-based key-value storage structure, which temporarily stores the filling results of high-frequency templates in the form of a hash table. When the same structured data is input again, the historical results are directly called.

[0048] Through the above solution, this embodiment effectively solves the problem of semantic coherence of generated text, avoids semantic breaks caused by missing long-distance dependencies, and significantly reduces the consumption of computing resources in high-frequency data scenarios through the caching mechanism, thereby improving the overall processing efficiency.

[0049] In some of the above implementations of this embodiment, a multi-dimensional index is proposed to evaluate the quality of the generated corpus. However, in its implementation, the evaluation index lacks specific quantitative methods and user feedback mechanisms, resulting in highly subjective evaluation results that cannot accurately reflect the quality of the corpus. This leads to a lag in the adjustment of the generation strategy and affects the continuous optimization of the corpus's fluency, accuracy, and diversity.

[0050] In response, this embodiment further proposes steps for evaluating the quality of the generated corpus using multi-dimensional indicators, including: The evaluation metrics include fluency, accuracy, and diversity. The fluency metric is calculated based on the perplexity of the language model, and the accuracy metric is calculated through semantic consistency comparison. It also includes a human feedback loop, allowing users to rate the generated corpus and retrain the generative model based on the rating results, dynamically adjusting the generation strategy.

[0051] In practical applications, fluency index refers to an objective quantitative indicator that measures the fluency of generated text. It can be implemented using probabilistic evaluation methods based on pre-trained language models, such as using a language model based on the Transformer architecture to calculate the probability distribution of text sequences. The purpose is to provide reproducible mathematical evidence through statistical characteristics and overcome the arbitrariness of human judgment. Among them, the accuracy index can be understood as a metric for evaluating the semantic consistency between the generated content and the source data. Specifically, it can be achieved by using a semantic similarity model to conduct vector space comparison analysis. For example, by calculating the similarity score between the generated text and the original data in the embedding space, the aim is to accurately capture content deviations and avoid misjudgments caused by simple literal matching. Specifically, the human feedback loop refers to a closed-loop mechanism in which users participate in the evaluation of corpus quality and drive model optimization. It can collect user opinions through interactive scoring interfaces or structured feedback forms. Its purpose is to transform human subjective cognition into dynamic optimization of model parameters and achieve continuous iteration for quality improvement.

[0052] Specifically, the solution in this embodiment first calculates the perplexity of the generated text using a language model to quantify fluency, and simultaneously uses semantic consistency comparison technology to evaluate content accuracy, forming a multi-dimensional quality assessment result. Subsequently, the system collects user rating feedback on the generated corpus, uses these ratings as supervisory signals to retrain the generative model, and dynamically adjusts the model parameters according to actual usage effects. Based on this, the system optimizes the generation strategy in real time according to the updated model and evaluation indicators, ensuring that the subsequently generated corpus continuously improves in terms of fluency, accuracy, and diversity, thus forming a complete closed loop from evaluation to optimization.

[0053] As a specific implementation method, in a medical data processing scenario, after the system generates electronic medical record description text, it first calculates the perplexity of the text's language model to assess fluency, and verifies the degree of matching between the diagnostic description and the original ICD encoding through semantic consistency comparison. At the same time, a scoring interface is provided to doctors, allowing them to score the clinical accuracy of the generated text. After collecting these scores, the system uses reinforcement learning methods to fine-tune the generative model, making the model more inclined to generate descriptions that conform to medical standards. For example, when doctors score diabetes-related descriptions low, the model automatically adjusts relevant parameters to improve the generation quality of similar scenarios in the future.

[0054] Through the above technical solutions, the evaluation process effectively combines objective quantification with user feedback, solves the problem of strong subjectivity in evaluation results, and enables the generation strategy to be adjusted in a timely manner according to the actual usage effect. This ensures that the generated corpus has both natural language expressiveness and data accuracy in professional scenarios, and effectively improves the ability to continuously optimize the quality of the corpus.

[0055] Specifically, in the process of converting professional codes into natural language based on a terminology mapping table, the terminology mapping table only relies on a static standard terminology database for extraction, which cannot respond in a timely manner to the dynamic evolution of domain terms (such as the introduction of new codes or the obsolescence of old codes), resulting in the conversion results lagging behind the actual application needs. At the same time, there is a lack of an active detection mechanism for terminology usage conflicts. When terminology ambiguity or duplicate mapping occurs in different data sources or contexts, the system cannot automatically identify and correct it, resulting in inconsistent terms or semantic errors in the output text, which seriously affects the reliability and readability of professional domain text corpora.

[0056] To address this, this embodiment further proposes that in the step of converting professional codes into natural language based on a terminology mapping table, the terminology mapping table is extracted from a standard terminology database and dynamically updated using machine learning methods; it also includes a terminology consistency check module, which detects terminology usage conflicts through a rule engine and automatically recommends alternative terms to ensure the accuracy of terminology conversion.

[0057] The dynamic updating of the terminology mapping table refers to its ability to automatically adjust its content based on real-time data streams. This can be achieved using incremental learning algorithms or online learning models, such as fine-tuning the model by continuously receiving newly labeled terminology samples. The goal is to enable the system to adapt to the evolution of domain terminology without human intervention, thereby maintaining the timeliness of the conversion results. The terminology consistency check module is a component used to monitor the consistency of terminology usage. It can be implemented using rule-based inference engines or context-aware conflict detection algorithms. For example, it analyzes the applicability of terms in different contexts through a predefined set of semantic rules. The goal is to automatically identify and resolve terminology ambiguities, ensuring the professional standardization of the output text.

[0058] Specifically, this embodiment achieves closed-loop optimization of the terminology conversion process by deeply integrating the dynamic update mechanism of the terminology mapping table with the terminology consistency check module. During data processing, the terminology mapping table continuously extracts basic mapping relationships from the standard terminology database and uses machine learning methods to automatically learn and update newly added or changed terms, ensuring that the mapping table content is synchronized with domain development. Simultaneously, the terminology consistency check module scans the output text in real time during conversion, detecting potential terminology conflicts through a rule engine. When inconsistent use of the same code in different contexts is identified, an alternative term recommendation mechanism is immediately triggered, providing standardized correction suggestions. This design enables the system to dynamically adapt to domain changes while maintaining terminology accuracy, effectively avoiding conversion errors caused by an outdated terminology database or contextual ambiguity.

[0059] In practical implementation: In medical data processing scenarios, when the system receives electronic medical record data containing ICD-10 codes, the terminology mapping table first loads the basic mapping relationships from the standard medical terminology database, and automatically identifies and updates newly emerging codes (such as newly added diabetes subtype codes) through a machine learning model; during the conversion process, the terminology consistency check module monitors the use of "diabetes" related terms. If it detects that "diabetes" is incorrectly used in a cardiovascular context and "type 2 diabetes" should be used instead, the rule engine immediately recommends the correct terminology replacement to ensure that the terminology in the final generated patient report is accurate and consistent.

[0060] Through the above solution, this embodiment effectively solves the timeliness and terminology conflict problems caused by the static mapping table in the terminology conversion process, significantly improves the accuracy and consistency of professional coding to natural language conversion, and thus enhances the reliability and readability of the generated text corpus in professional fields.

[0061] In some of the above embodiments of this example, a method is proposed to process multi-source structured data. However, in its implementation, it lacks the ability to automatically adapt to different fields, which makes it unable to flexibly cope with the specific data formats, terminology rules and industry standards of industries such as medical, financial and legal. It requires repeated manual adjustments and configurations, which increases the processing complexity, error rate and implementation cost, and makes it difficult to achieve efficient cross-domain expansion.

[0062] In response, this embodiment further proposes a method applicable to general structured data processing scenarios, adapting to different domain requirements through configurable parameters, wherein: The multi-source data extraction step supports dynamically loading data source connectors and automatically adjusts the extraction strategy according to the format and structure of the input data to ensure cross-domain data compatibility. The context-aware template generation step integrates a configurable rule engine, allowing users to customize template rules and style guidelines to generate text corpora that adapt to different industry standards; The hybrid generation framework execution steps introduce a domain adaptation mechanism, which fine-tunes the parameters of the generative model to make the text output conform to the semantic and syntactic norms of a specific domain. The terminology standardization process supports switching between multiple terminology databases and dynamically loads the corresponding terminology mapping table according to the application scenario, ensuring the accuracy and consistency of professional code conversion.

[0063] The multi-source data extraction step supports dynamically loading data source connectors, meaning the system can automatically load the corresponding interface module based on the detected data source type. This can be achieved using a plug-in architecture combined with a service discovery mechanism. For example, it can dynamically retrieve dedicated connectors adapted to database tables or JSON data through a registry center. The purpose is to avoid repeatedly developing dedicated interfaces for different data sources (such as the HL7 standard in the medical field and the FIX protocol in the financial field), thereby improving data integration efficiency. The context-aware template generation step integrates a configurable rule engine, which can be understood as providing an execution environment for user-defined template generation logic. This can be achieved using a rule expression-based parser or an open-source rule engine framework. For example, it can define the paragraph structure of a financial report by configuring an XML rule file. The purpose is to reduce reliance on repeated manual template modifications and ensure that the output content conforms to the specific expression specifications of legal documents or military briefings. The hybrid generation framework execution steps introduce a domain adaptation mechanism, which specifically refers to the technical means of optimizing the text generation model for a specific domain. This can be achieved by using efficient parameter fine-tuning methods or domain-specific layer injection strategies. For example, adjusting some weights of the Transformer model through low-rank adaptation techniques aims to solve the semantic bias problem that general generation models are prone to in professional domains (such as medical diagnostic reports), improving the professionalism of the output without retraining the entire model. The terminology standardization processing steps support switching between multiple terminology databases, which is a technical solution for dynamically selecting terminology mapping resources according to the scenario. This can be achieved by using scenario identifier matching or context-aware loading strategies. For example, automatically switching to the ICD-10 medical terminology database based on the metadata tags of the input data aims to address conversion errors caused by differences in terminology across different domains (such as legal article encoding and financial product codes), ensuring the reliability of terminology understanding.

[0064] Specifically, the solution in this embodiment automatically identifies the format characteristics of the input data and dynamically loads matching connectors through a multi-source data extraction step, providing a unified data model for subsequent processing. Based on this, the context-aware template generation step calls a configurable rule engine to parse user-defined industry rules, generating template structures adapted to specific domains (such as legal contracts or financial statements). The hybrid generation framework execution step uses a domain-adaptive mechanism to fine-tune the parameters of the generative model, ensuring that the text output strictly adheres to the semantic specifications of the target domain. The terminology standardization processing step switches the terminology mapping table in real time based on application scenario identifiers, ensuring the accuracy of professional coding conversion. These steps form a closed-loop workflow, with each stage seamlessly connected through configurable parameters: when the system detects medical domain data input, it automatically triggers HL7 connector loading, medical style template generation, clinical corpus model fine-tuning, and ICD-10 terminology database switching, thereby completing the conversion from heterogeneous data to professional text without manual intervention.

[0065] As a specific implementation method, the solution in this embodiment is implemented as follows in a medical data processing scenario: When receiving an electronic medical record database table, the multi-source data extraction step dynamically loads the HL7 dedicated connector through a service discovery mechanism, automatically identifying field structures such as patient ID and diagnosis code; the context-aware template generation step calls the integrated Drools rule engine to generate a template containing dynamic slots for "patient complaint" and "clinical diagnosis" based on the user-preset medical report style guidelines; the hybrid generation framework execution step activates the fine-tuned clinical domain generation model, generating a medically compliant description of "type 2 diabetes" based on the diagnosis code E11.9; the terminology standardization processing step synchronously loads the ICD-10 terminology mapping table, converting the surgical code 0DTJ4ZZ into a natural language expression for "laparoscopic cholecystectomy". The entire process automatically coordinates the parameter configurations of each module through domain identifiers, ensuring that the output text simultaneously meets the terminology accuracy and report format requirements of the medical industry. Through the above solution, this embodiment effectively solves the compatibility and customization challenges in cross-domain applications, significantly reduces the complexity of repeated manual configuration adjustments, avoids increased error rates due to data format differences, and enables the method to automatically adapt to the specific needs of diverse fields such as medicine and finance, thereby improving the implementation efficiency and scalability of structured data processing. Furthermore, this application embodiment also proposes a computer-readable storage medium storing a program automatically generated based on multi-source structured data corpus. When the program is executed by a processor, it implements the steps of the method for automatically generating multi-source structured data corpus as described above.

[0066] Reference Figure 3 , Figure 3This is a structural block diagram of the first embodiment of the automatic generation system based on multi-source structured data corpus of this application.

[0067] like Figure 3 As shown, the automatic generation system based on multi-source structured data corpus proposed in this application includes: Data extraction module 10 is used to extract data from multi-source structured data, including database tables, Excel files, JSON data and knowledge graphs, and automatically identifies the data source structure through an adaptive interface adapter, supporting streaming or batch collection. Template generation module 20 is used to generate context-aware templates based on a domain knowledge base. The templates contain static text and dynamic slots for matching data fields. The corpus generation module 30 is used to generate text corpora using a hybrid framework of template-driven and generative models: for highly structured data, template filling is used; for low-structured data, a Transformer-based generative model is used for semantic completion. The strategy generation module 40 is used to evaluate the quality of the generated corpus through multi-dimensional indicators, including fluency, accuracy and diversity, and to dynamically adjust the generation strategy. Natural Language Conversion Module 50 is used to convert professional codes into natural language based on a terminology mapping table, ensuring terminology consistency and accuracy.

[0068] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of this application. In specific applications, those skilled in the art can make settings as needed, and this application does not impose any restrictions on this.

[0069] This embodiment achieves automatic identification and integration of multi-source data structures through an adaptive interface adapter. It dynamically processes highly structured and unstructured data by combining context-aware templates and a hybrid generation framework. Furthermore, it ensures the accuracy and consistency of the output corpus by relying on a multi-dimensional evaluation mechanism and a terminology mapping table. This effectively solves the technical problems of difficulty in integrating multi-source data, high cost of manual annotation, obstacles to the conversion of professional terms, and unstable generation quality. It has the advantages of improving the efficiency of multi-source structured data integration, reducing the cost of manual annotation, ensuring the accuracy of professional terminology conversion, and improving the stability of the generated corpus quality.

[0070] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this application. In practical applications, those skilled in the art can select some or all of it to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0071] In addition, for technical details not described in detail in this embodiment, please refer to the method for automatic generation based on multi-source structured data corpus provided in any embodiment of this application, which will not be repeated here.

[0072] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0073] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application. The above are only preferred embodiments of this application and do not limit the patent scope of this application. All equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for automatically generating text based on multi-source structured data corpus, characterized in that, include: Data is extracted from multi-source structured data, including database tables, Excel files, JSON data, and knowledge graphs. The data source structure is automatically identified through an adaptive interface adapter, supporting streaming or batch collection. Context-aware templates are generated based on a domain knowledge base. These templates contain static text and dynamic slots for matching data fields. A framework combining template-driven and generative models is used to generate text corpora: template imputation is used for highly structured data; semantic completion is performed using a Transformer-based generative model for low-structured data. The quality of the generated corpus is evaluated using multi-dimensional metrics, including fluency, accuracy, and diversity, and the generation strategy is dynamically adjusted accordingly. Specialized codes are converted into natural language based on a terminology mapping table, ensuring terminology consistency and accuracy.

2. The method according to claim 1, characterized in that, In the step of extracting data from multi-source structured data, the adaptive interface adapter adopts a plug-in architecture, dynamically loads dedicated connectors for different data sources, and automatically identifies the data source structure through metadata awareness technology to generate a unified data model. It also includes a load balancing mechanism that dynamically allocates and extracts resources based on data traffic, and integrates a data cleaning module to remove noisy data and duplicate entries, ensuring the quality of input data.

3. The method according to claim 1, characterized in that, In the step of generating context-aware templates based on a domain knowledge base, the domain knowledge base includes a terminology dictionary, grammar rules, and style guidelines, and identifies entities and relationships in structured data through semantic parsing technology and maps them to dynamic template slots; It also includes a template optimization mechanism, which compares the generation effects of different templates through A / B testing and adjusts template rules based on user feedback to improve slot matching accuracy.

4. The method according to claim 1, characterized in that, In the step of generating text corpus using a hybrid framework of template-driven and generative models, the generative model adopts a sequence-to-sequence architecture based on Transformer, introduces an attention mechanism to capture long-distance dependencies, and optimizes the coherence of the generated text through an adversarial training strategy. It also includes a caching mechanism to store high-frequency template results to improve processing efficiency.

5. The method according to claim 1, characterized in that, In the step of evaluating the quality of the generated corpus through multi-dimensional indicators, the evaluation indicators include fluency indicators, accuracy indicators and diversity indicators, wherein the fluency indicator is calculated based on the perplexity of the language model, and the accuracy indicator is calculated through semantic consistency comparison. It also includes a human feedback loop, allowing users to rate the generated corpus and retrain the generative model based on the rating results, dynamically adjusting the generation strategy.

6. The method according to claim 1, characterized in that, In the step of converting professional codes into natural language based on a terminology mapping table, the terminology mapping table is extracted from a standard terminology database and dynamically updated using machine learning methods. It also includes a terminology consistency check module, which uses a rules engine to detect terminology usage conflicts and automatically recommends alternative terms to ensure the accuracy of terminology conversion.

7. The method according to claim 1, characterized in that, The method is applicable to general structured data processing scenarios and can be adapted to different domain requirements through configurable parameters, wherein: The multi-source data extraction step supports dynamically loading data source connectors and automatically adjusts the extraction strategy according to the format and structure of the input data to ensure cross-domain data compatibility. The context-aware template generation step integrates a configurable rule engine, allowing users to customize template rules and style guidelines to generate text corpora that adapt to different industry standards; The hybrid generation framework execution steps introduce a domain adaptation mechanism, which fine-tunes the parameters of the generative model to make the text output conform to the semantic and syntactic norms of a specific domain. The terminology standardization process supports switching between multiple terminology databases and dynamically loads the corresponding terminology mapping table according to the application scenario, ensuring the accuracy and consistency of professional code conversion.

8. An automatic generation system based on multi-source structured data corpus, characterized in that, include: The data extraction module is used to extract data from multi-source structured data, including database tables, Excel files, JSON data and knowledge graphs. It automatically identifies the data source structure through an adaptive interface adapter and supports streaming or batch collection. The template generation module is used to generate context-aware templates based on a domain knowledge base. The templates contain static text and dynamic slots for matching data fields. The corpus generation module is used to generate text corpora using a hybrid framework of template-driven and generative models: template filling is used for highly structured data; semantic completion is performed using a Transformer-based generative model for low-structured data. The strategy generation module is used to evaluate the quality of the generated corpus through multi-dimensional indicators, including fluency, accuracy and diversity, and dynamically adjust the generation strategy. The natural language conversion module is used to convert technical codes into natural language based on a terminology mapping table, ensuring terminology consistency and accuracy.

9. A computer device, characterized in that, The device includes a memory and a processor, wherein the processor, when executing computer instructions stored in the memory, performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images