Demand-driven sample set automatic generation system

Abstract

The invention discloses a sample set automatic generation system based on demand driving, and the system comprises a demand analysis module which is used for receiving a target demand inputted by a user, and automatically extracting a data type, a distribution feature and a constraint condition; the sample generation module is used for being in communication connection with the demand analysis module and generating an initial sample set meeting the demand according to an analysis result; the optimization evaluation module is used for carrying out quality evaluation on the generated sample set and outputting optimization suggestions; the iteration generation module is used for adjusting a generation strategy according to the optimization suggestion and regenerating an improved sample set; according to the method, automatic creation and optimization of the sample set can be realized by constructing a demand-driven intelligent sample generation framework, and the problems of much manual intervention and weak adaptive capacity in a traditional method are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of sample set generation, and in particular to a demand-driven automatic sample set generation system. Background Art

[0002] In areas such as machine learning model training and data analysis, sample set construction is a key step. Traditional sample set generation methods have the following drawbacks: Highly manual dependency: Sample rules must be manually designed or rely on preset templates, which is inefficient and difficult to adapt to complex needs; Poor dynamic adaptability: Unable to automatically adjust sample distribution according to actual application scenarios (such as image classification and text generation); Data redundancy problem: The generated samples lack specificity and are prone to introducing noise or duplicate data, which affects the model training effect. Summary of the Invention

[0003] The purpose of the present invention is to provide a demand-driven automatic sample set generation system to address the above-mentioned problems in the prior art, thereby solving all or one of the above-mentioned problems in the prior art.

[0004] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows: The present invention provides a demand-driven automatic sample set generation system, comprising: The requirement parsing module is used to receive the target requirements input by the user and automatically extract the data type, distribution characteristics and constraints; A sample generation module, configured to communicate with the requirement analysis module and generate an initial sample set that meets the requirements based on the analysis results; The optimization evaluation module is used to evaluate the quality of the generated sample set and output optimization suggestions; The iterative generation module is used to adjust the generation strategy according to the optimization suggestions and regenerate the improved sample set.

[0005] As an improved solution, the demand analysis module further includes: Natural language processing unit, used to parse text requirements entered by users; Parameter extraction unit, used to automatically identify structured parameters such as numerical ranges and category labels from text; Domain knowledge base, used to store predefined industry templates.

[0006] As an improved solution, the sample generation module includes multiple generation engines: A rule engine for generating structured samples based on templates; Synthesis engine, used to generate unstructured samples such as images and audio using GANs or diffusion models; The hybrid engine is used to combine the rule engine and the synthesis engine to generate multimodal combination samples.

[0007] As an improved solution, the optimization evaluation module includes: Diversity detection submodule, used to evaluate the uniformity of sample distribution through statistical indicators; The compliance detection submodule is used to verify whether the sample complies with the constraints specified by the user; The model fitness submodule is used to input samples into the pre-trained model and calculate the prediction error as the basis for optimization.

[0008] As an improved solution, the iterative generation module supports two optimization modes: Local optimization mode: adjust the parameters of the generation engine; Global optimization mode: switch the generation engine type or reorganize the sample generation process.

[0009] As an improved solution, the sample set automatic generation system has a built-in sample cache library for storing historical generation samples and their optimization records to avoid repeated calculations; the sample cache library supports a fast retrieval function based on similarity matching. As an improved solution, The sample set automatic generation system provides a visual configuration interface for allowing users to define required parameters by dragging components; the configuration interface automatically generates corresponding parameter codes.

[0010] As an improved solution, the sample set automatic generation system integrates a multi-tenant management module to support allocation of independent sample generation environments to different users.

[0011] As an improved solution, the sample set automatic generation system has anomaly detection capability, which is used to monitor the abnormal status during the generation process in real time, and automatically trigger a rollback mechanism when an anomaly occurs to restore to the previous version of the sample set.

[0012] As an improved solution, the sample set automatic generation system is also used to provide an API service interface to support integration with external platforms; The API interface definition complies with the OpenAPI 3.0 specification and includes sample generation, status query, and batch export functions.

[0013] The beneficial effects of the technical solution of the present invention are: the present invention can realize the automatic creation and optimization of sample sets by building a demand-driven intelligent sample generation framework through the mutual cooperation of system modules, thereby solving the problems of frequent manual intervention and weak adaptability in traditional methods; automatically extract key parameters in user needs (such as data types, distribution characteristics, and constraints); support the collaborative generation of cross-modal data such as text, images, and voice; optimize sample quality based on feedback iteration, and reduce manual verification costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 4 is a schematic diagram of the architecture of the demand-driven automatic sample set generation system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present invention.

[0017] In the description of the present invention, it should be noted that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of the present invention.

[0018] The terms "first," "second," and the like in the specification and claims herein and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device. Example

[0019] This embodiment provides a demand-driven automatic sample set generation system. Figure 1 As shown, including: The requirement parsing module is used to receive the target requirements input by the user and automatically extract the data type, distribution characteristics and constraints; A sample generation module, configured to communicate with the requirement analysis module and generate an initial sample set that meets the requirements based on the analysis results; The optimization evaluation module is used to evaluate the quality of the generated sample set and output optimization suggestions; The iterative generation module is used to adjust the generation strategy according to the optimization suggestions and regenerate the improved sample set.

[0020] As an improved solution, the demand analysis module further includes: Natural language processing unit, used to parse text requirements entered by users; Parameter extraction unit, used to automatically identify structured parameters such as numerical ranges and category labels from text; Domain knowledge base, used to store predefined industry templates.

[0021] As an improved solution, the sample generation module includes multiple generation engines: A rule engine for generating structured samples based on templates; Synthesis engine, used to generate unstructured samples such as images and audio using GANs or diffusion models; The hybrid engine is used to combine the rule engine and the synthesis engine to generate multimodal combination samples.

[0022] As an improved solution, the optimization evaluation module includes: Diversity detection submodule, used to evaluate the uniformity of sample distribution through statistical indicators; The compliance detection submodule is used to verify whether the sample complies with the constraints specified by the user; The model fitness submodule is used to input samples into the pre-trained model and calculate the prediction error as the basis for optimization.

[0023] As an improved solution, the iterative generation module supports two optimization modes: Local optimization mode: adjust the parameters of the generation engine; Global optimization mode: switch the generation engine type or reorganize the sample generation process.

[0024] As an improved solution, the sample set automatic generation system has a built-in sample cache library for storing historical generation samples and their optimization records to avoid repeated calculations; the sample cache library supports a fast retrieval function based on similarity matching. As an improved solution, The sample set automatic generation system provides a visual configuration interface for allowing users to define required parameters by dragging components; the configuration interface automatically generates corresponding parameter codes.

[0025] As an improved solution, the sample set automatic generation system integrates a multi-tenant management module to support allocation of independent sample generation environments to different users.

[0026] As an improved solution, the sample set automatic generation system has anomaly detection capability, which is used to monitor the abnormal status during the generation process in real time, and automatically trigger a rollback mechanism when an anomaly occurs to restore to the previous version of the sample set.

[0027] As an improved solution, the sample set automatic generation system is also used to provide an API service interface to support integration with external platforms; The API interface definition complies with the OpenAPI 3.0 specification and includes sample generation, status query, and batch export functions.

[0028] It should be noted that the examples herein are only for explaining the present invention and are not intended to limit the scope of protection of the present invention.

[0029] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0030] It should also be understood that in the embodiments herein, the term "and / or" merely describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" could represent: A alone, A and B simultaneously, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0031] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0032] Those skilled in the art will clearly understand that, for the sake of convenience and brevity of description, the specific logical process of the method described above can refer to the corresponding working processes of the systems, devices and units in the aforementioned method embodiments, and will not be repeated here.

[0033] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices, or units, or can be an electrical, mechanical, or other form of connection.

[0034] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments herein.

[0035] In addition, the functional units in the various embodiments herein may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0036] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0037] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A demand-driven automatic sample set generation system, characterized by: The system comprises at least: The requirement parsing module is used to receive the target requirements input by the user and automatically extract the data type, distribution characteristics and constraints; A sample generation module, configured to communicate with the requirement analysis module and generate an initial sample set that meets the requirements based on the analysis results; The optimization evaluation module is used to evaluate the quality of the generated sample set and output optimization suggestions; The iterative generation module is used to adjust the generation strategy according to the optimization suggestions and regenerate the improved sample set.

2. The demand-driven automatic sample set generation system according to claim 1, characterized in that: The demand analysis module further includes: Natural language processing unit, used to parse text requirements entered by users; Parameter extraction unit, used to automatically identify structured parameters such as numerical ranges and category labels from text; Domain knowledge base, used to store predefined industry templates.

3. The demand-driven automatic sample set generation system according to claim 1 or 2, characterized in that: The sample generation module includes multiple generation engines: A rule engine for generating structured samples based on templates; Synthesis engine, used to generate unstructured samples such as images and audio using GANs or diffusion models; The hybrid engine is used to combine the rule engine and the synthesis engine to generate multimodal combination samples.

4. The demand-driven automatic sample set generation system according to claim 1 or 2, characterized in that: The optimization evaluation module includes: Diversity detection submodule, used to evaluate the uniformity of sample distribution through statistical indicators; The compliance detection submodule is used to verify whether the sample complies with the constraints specified by the user; The model fitness submodule is used to input samples into the pre-trained model and calculate the prediction error as the basis for optimization.

5. The demand-driven automatic sample set generation system according to claim 1 or 2, characterized in that: The iterative generation module supports two optimization modes: Local optimization mode: adjust the parameters of the generation engine; Global optimization mode: switch the generation engine type or reorganize the sample generation process.

6. The demand-driven automatic sample set generation system according to claim 1 or 2, characterized in that: The sample set automatic generation system has a built-in sample cache library for storing historically generated samples and their optimization records to avoid repeated calculations; the sample cache library supports a fast retrieval function based on similarity matching and reuses existing samples to improve efficiency.

7. The demand-driven automatic sample set generation system according to claim 1 or 2, characterized in that: The sample set automatic generation system provides a visual configuration interface for allowing users to define required parameters by dragging components; the configuration interface automatically generates corresponding parameter codes.

8. The demand-driven automatic sample set generation system according to claim 1 or 2, characterized in that: The sample set automatic generation system integrates a multi-tenant management module to support allocation of independent sample generation environments to different users.

9. The demand-driven automatic sample set generation system according to claim 1 or 2, characterized in that: The sample set automatic generation system has anomaly detection capabilities, which is used to monitor abnormal conditions during the generation process in real time, and automatically trigger a rollback mechanism when an anomaly occurs to restore to the previous version of the sample set.

10. The demand-driven automatic sample set generation system according to claim 1 or 2, characterized in that: The sample set automatic generation system is also used to provide an API service interface to support integration with external platforms; The API interface definition complies with the OpenAPI 3.0 specification and includes sample generation, status query, and batch export functions.

Images