A multi-dimensional generalizable data generation method, device and software system for large model training

Abstract

The application relates to a multi-dimensional generalizable data generation method, device and software system for large model training, and belongs to the natural language processing field. In view of the technical problems of insufficient diversified expression, easy deviation of core semantics and lack of a multi-dimensional language transformation system in the prior art, the application provides a multi-dimensional generalizable data generation method for large model training. The method refines core semantics by input analysis on an original sentence, then performs transformation and combination expansion of four dimensions of semantics, syntax, pragmatics and vocabulary around the core semantics, and generates a rich set of transformation instructions. Then, the core semantic elements and the transformation instructions are structured into a structured prompt, and a large language model is input to generate a candidate sentence. Finally, qualified generated text is constructed into a structured data set and applied to a downstream natural language processing system. The application can generate much richer and more natural equivalent expressions than traditional methods, and strictly guarantees semantic consistency.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and relates to a method, apparatus and software system for generating multi-dimensional generalizable data for training large models. Background Technology

[0002] With the rapid development of artificial intelligence and human-computer interaction technologies, Natural Language Processing (NLP) has become a core technology for applications such as intelligent dialogue systems, voice assistants, and text mining. In building these systems, acquiring massive, diverse, and semantically equivalent training data is crucial, as it directly determines the model's ability to understand the user's true intent and the system's robustness.

[0003] However, existing technologies generally suffer from the following drawbacks when generating such data:

[0004] Limited Dimensionality: Traditional data augmentation methods, such as simple synonym replacement, backtranslation, or sentence transformation based on fixed templates, can only expand on a single dimension, usually vocabulary or syntax. This results in insufficient diversity in the generated data, failing to fully simulate the rich expressions of human language at multiple levels, including semantics, syntax, pragmatics, and vocabulary.

[0005] Semantic consistency is difficult to guarantee: Some automated data augmentation algorithms, especially when performing deep transformations, lack effective monitoring and constraint mechanisms to ensure that the core semantics remain consistent. This often leads to "semantic drift" or "semantic distortion," where the meaning of the generated sentence deviates from the original sentence. Training with this poor-quality data can actually impair the model's accuracy.

[0006] Lack of systematicity and controllability: Existing technologies are often fragmented applications of various individual techniques, failing to integrate multi-dimensional transformations such as semantics, syntax, pragmatics, and lexicality into a systematic and controllable generative framework. This makes the generation process difficult to manage, and the richness, grammatical correctness, and fluency of the generated results are also difficult to guarantee.

[0007] Furthermore, existing technologies have the following shortcomings: Chinese patent CN202311733517.7, concerning speech recognition methods, apparatuses, devices, and storage media based on Application Programming Interface (API) information, designs a process from understanding the API to generating instructions for a large model, but does not design for the extensive generalization of data. Chinese patent CN202410494963.5, concerning the construction of complex instruction training data for model training, processes, formats, and generalizes initial training data to generate new training data; these all involve enhancing existing data and do not design for instruction control data generated from the API.

[0008] Therefore, how to design an automated method that can systematically and multidimensionally generate high-quality equivalent representations while keeping the core semantics unchanged is a technical problem that urgently needs to be solved in the field of natural language processing. Summary of the Invention

[0009] In view of this, the purpose of this invention is to provide a method, apparatus, and software system for generating multi-dimensional generalizable data for large-scale model training. This invention aims to address the shortcomings of existing technologies, such as insufficient diversity of expression, easy deviation of core semantics, and lack of a multi-dimensional language transformation system, by providing a method for generating multi-dimensional generalizable data for large-scale model training. This method, while maintaining the consistency of the core semantics of the original sentence, systematically transforms data from four dimensions—semantics, syntax, pragmatics, and lexical—and their sub-dimensions. It can also be integrated with large language models and related natural language processing tools for automated generation, thereby obtaining rich and high-quality diverse text expressions, providing more robust and diverse training and testing data for voice assistants, dialogue systems, and other natural language processing applications.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] I. Overall Process and Core Structure

[0012] 1. Input processing and semantic extraction

[0013] First, we conduct a demand analysis on the original sentence input by the user to determine its usage scenario (e.g., voice assistant, dialogue system), its field (e.g., automotive, healthcare, finance), and potential contextual limitations.

[0014] Natural Language Processing (NLP) parsing tools such as spaCy, Stanza, and Stanford CoreNLP are used to perform syntactic, dependency, and semantic role analysis on the input text to extract key elements, including intent, entity, action, and context.

[0015] The original sentence is semantically vectorized using Sentence-BERT or SimCSE to establish "semantic embedding points," which are used to verify whether the similarity between the newly generated sentence and the original sentence meets the standard.

[0016] 2. Multi-dimensional transformation and combination extension

[0017] Semantic dimension: This includes functional restatement and paraphrasing, such as replacing with synonyms or functional descriptions and negation logic transformations. For complex application programming interface (API) calls, such as SQL queries, hierarchical generation strategies and combinations can be adopted. For example, the model space can be combined into a certain pattern in a certain region, so that the generated sentence is logically equivalent to the original sentence.

[0018] Syntactic dimension: This involves sentence transformation, such as the conversion between imperative and interrogative sentences, declarative and exclamatory sentences, inversion or word order reorganization (e.g., highlighting key components), passive voice, relative clauses, adverbial clauses, and other grammatical structure transformations.

[0019] Pragmatic dimension: This includes expressing needs such as being more conversational and contextualized, and controlling tone and style, such as politeness, humor, and technicality, to make the generated text more in line with real communicative scenarios.

[0020] Lexical dimension: Nouns, verbs, and modifiers are replaced using thesaurus or word vector model and masked language model to enrich vocabulary diversity.

[0021] Cross-dimensional combination: Combine the transformation methods of the four main dimensions mentioned above, such as performing syntactic transformation first and then word substitution, or combine it with pragmatic adjustment to form multi-dimensional and diversified expressions.

[0022] 3. Model Generation and Hints Project

[0023] Large Language Model (LLM): Select a Transformer model such as ChatGPT, Grok, LLaMA, or fine-tuned on a specific industry corpus, and use its powerful natural language generation capabilities to perform the above multi-dimensional transformations.

[0024] Prompt Engineering: Design structured prompts that clearly specify the required transformation types and language styles, and achieve generation diversity and controllability by adjusting parameters such as Temperature, Top-k, and Top-p.

[0025] Auxiliary tools include spaCy or Stanford Parser for syntactic analysis, WordNet or thesaurus for word replacement, and Sentence-BERT or SimCSE for similarity verification, which can be used in combination via application programming interfaces or local deployment.

[0026] 4. Post-processing verification and quality control

[0027] Semantic consistency verification: The generated sentence and the original sentence are embedded into vector spaces respectively, and semantic consistency is measured by calculating cosine similarity or Euclidean distance; if it is lower than a set threshold, such as 0.85, it is judged as semantic deviation and needs to be removed or regenerated.

[0028] Language fluency and grammar compliance checks: Use the Grammarly application programming interface, LanguageTool, or let the large language model perform "self-checking" to filter out incoherent or erroneous expressions; if the perplexity is too high, it is necessary to roll back and regenerate.

[0029] Deduplication and diversity assessment: Deduplication is performed using vector clustering such as K-means or edit distance such as Levenshtein, and overall diversity is evaluated using metrics such as Distinct-n and Self-BLEU; if the duplication rate is high, the randomness of the generation strategy is increased.

[0030] 5. Data Construction and Application

[0031] Dataset format: JSON or CSV, recording information such as "original input sentence", "generated output sentence", "transformation dimension used", "similarity", and "application scenario" to facilitate retrieval and secondary processing.

[0032] Application scenarios: This diverse dataset can be used for training data augmentation in Natural Language Understanding (NLU) for voice assistants, diverse responses in chatbots, text intent classification, sentiment analysis, and other fields, significantly enhancing the system's robustness to the diverse expressions of natural language.

[0033] II. Key Technologies and Specific Points

[0034] 1. Semantic extraction and protection mechanism

[0035] Semantic hierarchical representation: Adopting a method combining syntactic analysis and semantic role labeling to accurately divide the structure of "intention + object + action + context" to form a semantic skeleton, which ensures that the core elements remain unchanged in subsequent transformations.

[0036] Similarity threshold screening: Select a threshold such as 0.85 to 0.90 through training or experience. If the similarity between the generated text and the original sentence is too low, the generated text will be automatically discarded. This measure forms a "semantic protection" barrier to prevent the generated sentences from deviating from the topic.

[0037] 2. Implementation of the four-dimension transformation strategy

[0038] Semantic dimension:

[0039] Functional restatement: Retrieve the domain thesaurus or use the nearest neighbor of word vectors to obtain or generate near-synonymous phrases.

[0040] Negative expression: Construct word pairs such as "turn on" and "do not turn off" at the logical level, or other functionally equivalent expressions such as "enable" and "don't shut down".

[0041] Syntactic dimension:

[0042] Sentence pattern transformation: Pre-set multiple sentence templates such as "May I X?" and "Can you X a little?", and automatically fill in the templates in combination with the dependency parsing result of the original item.

[0043] Inversion / sentence order adjustment: Use the syntax tree or dependency relationship to adjust the word order regularly, and call the model to verify the readability of the sentence.

[0044] Passive voice: After determining the subject-predicate-object structure of the sentence, automatically generate a passive structure sentence, for example, "XX is turned on".

[0045] Pragmatic dimension:

[0046] Expression of demands: Inject emotional or scenario-based expressions, for example, "I feel a little cold, please turn on the seat heater".

[0047] Tone and style: Achieve richer text styles through explicit instructions of prompt engineering, such as "rewrite this sentence in a polite and formal tone" or by means of style transfer technology.

[0048] Lexical dimension:

[0049] Entity noun replacement: For key nouns in the sentence, such as device and function names, call the thesaurus or a customized domain term list, for example, convert "heating" to "warm air".

[0050] Verb replacement: Convert "turn on" to "enable" or "start", etc., supplemented by synonymy level or stylistic adaptability, such as colloquial or formal.

[0051] Modifier adjustment: Insert or remove adverbs and adjectives, such as "Please turn on the seat heating quickly", or replace, for example, "immediately" with "right away".

[0052] 3. Automated generation driven by large language models

[0053] Hint design: List or selectively list the required transformation dimensions or style preferences in the hints, and provide examples to reduce model generation deviations.

[0054] Generation controls: Set Temperature, for example, 0.7 to 0.9 to improve text diversity, or use Top-k, for example, k=50 to avoid rare words affecting readability.

[0055] Fine-tuning or adaptation: For specific business scenarios, such as automotive and medical, the model can be fine-tuned by selecting scenario corpora to make the generated results closer to industry terminology.

[0056] 4. Post-processing and quality assessment mechanism

[0057] Semantic validation: In the aforementioned "semantic tracking" stage, the vector representations of the original and generated texts have been saved. The validity of the text is determined by checking if the cosine similarity is greater than a threshold. During manual sampling, key field rules, such as requiring "seat heating" to be retained or appearing, can also be used.

[0058] Language fluency: Each generated sentence is fed into a grammar checker such as Grammarly or LanguageTool to check the number of grammatical errors or readability score. If the error exceeds a certain threshold, it is judged as unqualified.

[0059] Deduplication strategy: Use vector clustering or string editing distance threshold to remove candidates with high repetition, such as those with a distance below a certain value, to ensure data diversity.

[0060] Diversity metrics: A custom Distinct-n metric is used to quantify the degree of data diversity by statistically analyzing the overlap between N-gram coverage and the Self-BLEU corpus.

[0061] 5. Data construction and integration with downstream systems

[0062] The filtered, qualified generated text is packaged with the corresponding original sentence into multi-line JSON data, where each record is explicitly marked:

[0063] Transformation dimensions already used include semantic, syntactic, pragmatic, lexical, or cross-dimensional combinations;

[0064] The final semantic similarity with the original sentence is the similarity score;

[0065] The results of the language fluency check can be stored if necessary.

[0066] This generated data can be injected into the training pipeline of a dialogue system or voice assistant through standard interfaces such as RESTful application programming interfaces or multilingual software development kits (SDKs), allowing downstream systems to obtain richer corpora during training or validation.

[0067] The beneficial effects of this invention are as follows:

[0068] (1) This invention has uniquely established a transformation framework integrating "semantics, syntax, pragmatics and vocabulary" and supports cross-dimensional combinations. It can generate richer and more natural equivalent expressions than traditional methods and fully cover human language habits.

[0069] (2) Through the pioneering dual mechanism of “semantic core extraction and locking” and “semantic benchmark similarity verification”, the problem of “semantic drift” in data augmentation is effectively solved, ensuring that all generated sentences are highly consistent with the original sentence intent and guaranteeing the validity of the data.

[0070] (3) It integrates the generation capabilities of large language models with multi-level post-processing verification processes, such as grammar checking and perplexity assessment, to ensure that the generated text is not only diverse, but also grammatically correct, fluent in expression, and conforms to natural language norms.

[0071] (4) This invention provides a modular and configurable framework. Users can customize domain lexicons, syntactic templates and pragmatic styles according to specific application scenarios such as automotive, finance and medical fields, and can easily integrate different natural language processing tools or large language models, which has strong flexibility and scalability.

[0072] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0073] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0074] Figure 1 The overall process of a multi-dimensional generalizable data generation method for training large models;

[0075] Figure 2 For each generalization dimension;

[0076] Figure 3 To generate a data quality check chart. Detailed Implementation

[0077] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0078] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0079] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0080] like Figure 1 As shown, the present invention provides a multi-dimensional generalizable data generation method for training large models, the overall process of which includes:

[0081] S100: Input and parsing.

[0082] The system receives the original sentence input by the user. Natural Language Processing (NLP) tools such as spaCy and Stanza are used to process the sentence, including word segmentation, part-of-speech tagging, syntactic analysis, and semantic role labeling. This step aims to extract the core semantic skeleton of the sentence. For example, for the sentence "Please help me turn on the driver's seat heater," the core elements parsed are: {Intent: "Turn on device", Action: "Turn on", Entity: "Seat heater", Attribute: "Driver's seat"}. Simultaneously, a sentence embedding model such as Sentence-BERT is used to encode the original sentence into a vector V_orig, which serves as the benchmark for subsequent semantic verification.

[0083] S200: Multi-dimensional transformation engine.

[0084] This module is the core of this invention. For example... Figure 2 As shown, it generates a transformation strategy (S201) around the locked "core semantics" (201) in four dimensions.

[0085] Semantic dimension transformation (202): including functional restatements such as “heat” to “raise temperature” and logical transformations such as “turn on” to “do not turn off”.

[0086] Syntactic dimension transformation (203): including sentence transformation, such as converting imperative sentences into interrogative sentences, word order adjustment, such as inversion, and voice transformation, such as converting active voice into passive voice.

[0087] Pragmatic dimension transformation (204): This includes adjustments to tone and style, such as converting commands into polite ones, formal ones into colloquial ones, and adding contextual expressions, such as “I’m a little cold,…”.

[0088] Lexical dimension transformation (205): including synonym or near-synonym replacement, such as "open" to "start", "start", and the addition, deletion and modification of modifiers.

[0089] These transformation strategies can be applied individually or combined across dimensions (206), for example, performing syntactic transformations first and then word substitutions, thus forming a rich set of transformation instructions.

[0090] S300: Model generation.

[0091] The core semantic elements extracted in S100 and the transformation instructions generated in S200 are combined to construct a structured prompt. For example... Figure 2As shown, a typical prompt (400) includes the original sentence (401), core semantic constraints (402), explicit transformation instructions (403), and output format requirements (404). This prompt is input into a pre-selected or fine-tuned Large Language Model (LLM) (S301), which generates one or more candidate sentences according to the instructions (S302). The diversity of the generated results can be controlled by adjusting the generation parameters of the Large Language Model, such as Temperature and Top-p.

[0092] S400: Post-processing and quality control.

[0093] The candidate sentences generated by the large language model are subjected to rigorous multi-level screening.

[0094] S401: Semantic consistency check. For example... Figure 3 As shown, each candidate sentence (301) is encoded into a vector V_gen (302) through the same sentence embedding model, and then the cosine similarity between V_gen and the original sentence vector V_orig is calculated (303). The similarity score is compared with a preset threshold, such as 0.85 (304), and only sentences with scores higher than the threshold can pass the verification (305); otherwise, they are judged as semantic drift and discarded (306).

[0095] S402: Language quality assessment. Sentences with grammatical errors or incoherent expressions are filtered out using grammar checking tools such as LanguageTool or by allowing large language models to self-correct.

[0096] S403: Diversity Assessment and Deduplication. Using methods such as edit distance or vector clustering, sentences that are highly repetitive with existing sentence database content are removed to ensure the diversity of the final dataset.

[0097] S500: Data Construction and Application.

[0098] The qualified sentences that pass all checks (S501) are stored together with metadata such as the original sentence, transformation dimension, similarity score, and application scenario in a structured format such as JSON or CSV (S502). The final generated dataset can be directly used for training or testing downstream systems (S503), such as enhancing the Natural Language Understanding (NLU) model of voice assistants or expanding the response knowledge base of chatbots.

[0099] Example: Enhanced Customer Service Scenario Inquiry Commands

[0100] This embodiment uses a flight query command from a customer service robot as an example to demonstrate the versatility of this method.

[0101] 1. Input and Parsing (S100):

[0102] Original sentence: "Find flights from Beijing to Shanghai on October 1, 2025, in business class."

[0103] Core semantics: {Intent: "Query", Object: "Flight", Attribute: "Date and Cabin Class"}.

[0104] Semantic baseline: Generate the Sentence-BERT vector V_orig for this sentence.

[0105] 2. Multi-dimensional Transformation (S200): Generation Transformation Strategy:

[0106] [Syntax]: Transform an interrogative sentence into a declarative sentence by adding an interrogative word.

[0107] [Pragmatic]: A more polite and formal way of asking the question.

[0108] [Vocabulary]: Replace "Where are we?" with a functional description.

[0109] [Combination]: [pragmatics] plus [syntax] plus [vocabulary].

[0110] 3. Generation, Verification, and Construction (S300-S500):

[0111] Original sentence: "Find flights from Beijing to Shanghai, date October 1, 2025, class of business."

[0112] [Syntactic Transformation] → "Are there any business class tickets for a direct flight from Beijing to Shanghai on October 1, 2025?" Similarity: 0.94.

[0113] [Pragmatic Transformation] → "Could you please check if there are any direct business class flights from Beijing to Shanghai on National Day?" Similarity: 0.92.

[0114] [Word Transformation] → "Excuse me, could you help me search for direct one-way flights from Beijing to Shanghai on National Day? Are there any business class seats available?" Similarity: 0.96.

[0115] [Combined Transformation] → "I want to go to Shanghai for National Day, give me a direct business class ticket from Beijing." Similarity: 0.91.

[0116] Application Programming Interface Examples

[0117] Application Programming Interface Call: find_flight

[0118] parameter:

[0119] origin: "Beijing"

[0120] Destination: Shanghai

[0121] date: "2025-10-01"

[0122] Class: "Business Class"

[0123] exclude_stops: True excludes transit points.

[0124] Query 0: "Find flights from Beijing to Shanghai, date October 1, 2025, in business class."

[0125] Semantic transformation prompt example

[0126] Role and Task: You are an expert focused on enhancing AI training data. Your core task is to generate one or more new queries that are semantically equivalent but more diverse and natural in expression, based on a structured application programming interface (API) call instruction and a corresponding base query.

[0127] Core principles: semantic equivalence and generalized expression. The generated new query must be able to trigger the exact same operations and parameters as the original application programming interface call, but its wording should mimic the way people speak, ask questions, or give instructions in the real world.

[0128] Generalization and transformation strategies: When generating new queries, please use one or more of the following strategies in combination:

[0129] 1. Intent Variation:

[0130] Replace direct imperative verbs such as search, find, create, and set with more euphemistic, colloquial, or varied expressions.

[0131] For example, "search" can be transformed into "help me take a look", "what recommendations do you have", or "I want to find something".

[0132] 2. Parameter Naturalization:

[0133] Numerical or enumerated values: Translate precise numerical values ​​or procedural enumerated values, such as sort: "rating_desc", into verbal descriptions such as "highest rated" or "best rated".

[0134] Boolean values: Convert boolean values ​​such as true or false into common language. For example, is_open: true means "is open now"; exclude_stops: true means "direct flight" or "no transfer".

[0135] Entity name: Use aliases, nicknames or more vague descriptions to replace formal names, for example, "Peking University" is converted to "Peking U"; "mom" is converted to "mommy".

[0136] 3. Contextualization of Time / Space:

[0137] Convert absolute dates and times such as 2025-12-25, 21:00 into relative or culturally related expressions, for example, "on Christmas Day", "9 o'clock the day after tomorrow evening".

[0138] Convert precise geographic coordinates or administrative divisions into vague descriptions based on landmarks or directions, for example, area: "Haidian District" is converted to "near Zhongguancun".

[0139] 4. Sentence Restructuring:

[0140] Disrupt or rearrange the order in which each parameter appears in the sentence.

[0141] Naturally integrate multiple parameters into a single phrase.

[0142] Table 1 shows examples of generated generalized data.

[0143] Table 1

[0144]

[0145] Finally, it is stated that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that modifications or equivalent substitutions can be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution, all of which shall be covered within the scope of the claims of the present invention.

Claims

1. A method for generating multi-dimensional generalizable data for training large models, characterized in that: Includes the following steps: S100: Input and parsing: Receive the original sentence, process the original sentence using a natural language processing (NLP) parsing tool to extract the core semantic skeleton of the original sentence, and encode the original sentence into a semantic baseline vector V_orig using a sentence embedding model; S200: A multi-dimensional transformation engine that generates one or more transformation strategies around the core semantic skeleton. The transformation strategies include at least one of semantic dimension transformation, syntactic dimension transformation, pragmatic dimension transformation and lexical dimension transformation. S300: Model generation, constructing a structured prompt from the core semantic skeleton and the transformation strategy, and inputting the structured prompt into a large language model (LLM) to generate one or more candidate sentences; S400: Post-processing and quality control, performing multi-level screening of the candidate sentences; specifically including: S401: Semantic consistency verification. Each candidate sentence is encoded into a vector V_gen using the same sentence embedding model. Then, the cosine similarity between V_gen and the semantic baseline vector V_orig is calculated. The cosine similarity score is compared with a preset threshold. Only candidate sentences with scores higher than the preset threshold pass the verification. S402: Language quality assessment, filtering out candidate sentences with grammatical errors or incoherent expressions by using grammar checking tools or allowing the LLM to self-correct; S403: Diversity assessment and deduplication, using methods such as edit distance or vector clustering to remove candidate sentences that are highly repetitive with the existing sentence database content; S500: Data construction and application, which stores the qualified sentences that have been selected together with metadata in a structured format, and uses the qualified sentences for training or testing of downstream natural language processing (NLP) systems.

2. The method for generating multi-dimensional generalizable data for large model training according to claim 1, characterized in that: The S100 step specifically includes: S101: Receive the original sentence input by the user; S102: Use the NLP parsing tool to perform word segmentation, part-of-speech tagging, syntactic analysis and semantic role tagging on the original sentence to extract the core semantic skeleton of the original sentence; S103: Encode the original sentence into the semantic baseline vector V_orig using the sentence embedding model, which serves as the baseline for subsequent semantic verification.

3. The method for generating multi-dimensional generalizable data for large model training according to claim 1, characterized in that: The transformation strategy also includes cross-dimensional combination, that is, combining at least two of the semantic dimension transformation, syntactic dimension transformation, pragmatic dimension transformation and lexical dimension transformation.

4. The method for generating multi-dimensional generalizable data for large model training according to claim 1, characterized in that: The preset threshold range is 0.85 to 0.

90.

5. The method for generating multi-dimensional generalizable data for large model training according to claim 1, characterized in that: The semantic dimension transformation includes functional restatement and logical transformation; The syntactic dimension transformation includes sentence transformation, word order adjustment, and voice transformation; The pragmatic dimension transformation includes adjustments to tone and style, and the addition of contextualized expressions; The lexical dimension transformation includes synonym / near-synonym replacement and the addition, deletion and modification of modifiers.

6. The method for generating multi-dimensional generalizable data for large model training according to claim 1, characterized in that: The multi-level screening also includes a perplexity assessment of the qualified sentences to ensure fluency.

7. A multi-dimensional generalizable data generation device for training large models, characterized in that: include: The input and parsing module is used to receive the original sentence, process the original sentence using natural language processing (NLP) parsing tools to extract the core semantic skeleton of the original sentence, and encode the original sentence into a semantic baseline vector V_orig using a sentence embedding model. A multi-dimensional transformation engine module is used to generate one or more transformation strategies around the core semantic skeleton, wherein the transformation strategies include at least one of semantic dimension transformation, syntactic dimension transformation, pragmatic dimension transformation and lexical dimension transformation. The model generation module is used to construct a structured prompt from the core semantic skeleton and the transformation strategy, and input the structured prompt into the large language model LLM to generate one or more candidate sentences; The post-processing and quality control module is used to perform multi-level filtering on the candidate sentences; The post-processing and quality control module specifically includes: The semantic consistency verification unit is used to encode each candidate sentence into a vector V_gen through the same sentence embedding model, then calculate the cosine similarity between V_gen and the semantic baseline vector V_orig, and compare the cosine similarity score with a preset threshold. Only candidate sentences with scores higher than the preset threshold pass the verification. The language quality assessment unit is used to filter out candidate sentences that contain grammatical errors or are not fluent by using grammar checking tools or by allowing the LLM to self-correct. The diversity assessment and deduplication unit is used to remove candidate sentences that are highly repetitive with existing sentence database content using methods such as edit distance or vector clustering. The transformation strategy also includes cross-dimensional combination, that is, combining at least two of the semantic dimension transformation, syntactic dimension transformation, pragmatic dimension transformation and lexical dimension transformation. The data construction and application module is used to store qualified sentences that have passed the screening along with metadata in a structured format, and to use the qualified sentences for training or testing of downstream natural language processing (NLP) systems.

8. A computer system, characterized in that: It includes a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the method as described in any one of claims 1 to 6.

Images