Data generation method and device, equipment, storage medium and product

Abstract

The invention discloses a data generation method and device, equipment, a storage medium and a product. The method comprises the following steps: acquiring data of a plurality of single stages to generate corresponding generated data; assembling the generated data of each single stage based on the associated information to obtain multi-stage data expected to be generated; wherein the association information represents an association relationship between the generated data of each single stage. Thus, the generated data corresponding to the multiple single-stage data can be generated on the basis of the associated information and assembled, the data scale of the generated multi-stage data can be expanded on the basis of the combination of the multiple single-stage generated data, effective convergence can be achieved in the data quality dimension, the data generation quality is improved, and the data generation efficiency is improved. Therefore, the multi-stage data generation effect is effectively improved, and the application requirements of downstream tasks are met.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data generation method, device, equipment, storage medium and product. Background Art

[0002] Data is one of the three elements in the development of artificial intelligence technology. With the development of computing power, the complexity of model parameters continues to increase. As the model effect "works miracles with great effort", the importance of data is becoming increasingly prominent.

[0003] Data generation technology has become a research focus in the era of deep learning. Data generation refers to the use of artificial intelligence technology, usually through a large language model (LLM), to generate the data required by users.

[0004] Most of the related technologies focus on data generation in general fields, which makes it difficult to meet the data generation needs of vertical segments (for example, network security). In addition, the related technologies often target single-stage data generation solutions, which often have the defect of limited scalability. In addition, when multi-stage data generation is performed based on single-stage experience, it is difficult to guarantee the data quality dimension, which affects the application effect of downstream tasks. Summary of the invention

[0005] In view of this, embodiments of the present application provide a data generation method, apparatus, device, storage medium and product, aiming to improve the quality of multi-stage data generation.

[0006] The technical solution of the embodiment of the present application is implemented as follows: In a first aspect, an embodiment of the present application provides a data generation method, comprising: Obtaining generated data corresponding to multiple single-stage data generation; The generated data of each single stage are assembled based on the association information to obtain the expected multi-stage data; wherein the association information represents the association relationship between the generated data of each single stage.

[0007] In the above scheme, the generated data of each single stage is assembled based on the associated information to obtain the desired generated multi-stage data, including: Based on the associated information, select the data topics of each single stage whose matching relationship meets the requirements; Based on the generated data corresponding to the selected data themes of each single stage, the data is assembled in the order of stage combination to obtain the expected multi-stage data.

[0008] In the above solution, the step of obtaining data of multiple single stages to generate corresponding generated data includes: Obtaining identity definition information representing the desired role; Based on the identity definition information, the scenario templates of various preset roles are matched to obtain a matching target scenario template; Configuring theme options for each single stage in the target scenario template; Based on the target scenario template after configuring the theme options, the generated data corresponding to the data generation of each single stage is obtained.

[0009] In the above scheme, the target scenario template after configuring the theme options is used to obtain the generated data corresponding to the data generation of each single stage, including: Generate prompt words for data generation in each single stage based on the target scenario template after configuring the theme options; The prompt words are input into a preset model to obtain generated data corresponding to the data generation of each single stage.

[0010] In the above scheme, the method further includes: Divide multiple preset roles based on data application scenarios; Based on each preset role and setting field, a corresponding scenario template is configured; wherein the setting field is used to define a prompt word framework corresponding to the scenario template.

[0011] In the above solution, each data topic in the topic options includes key elements supporting configuration, and the method further includes: Replacing the key elements based on the configuration file; Based on the replacement of the key elements, the generated prompt words are generalized to obtain batch prompt words; The step of inputting the prompt word into a preset model to obtain corresponding generated data for each single-stage data intelligent generation includes: The batch prompt words are input into a preset model to obtain batch generation data of each single stage.

[0012] In the above scheme, the method further includes: Acquire a data set in each scenario, where each data in the data set is multi-stage data; Based on multi-stage data in the same scene, the associated information of the corresponding scene is constructed.

[0013] In the above solution, the construction of the associated information of the corresponding scene based on the multi-stage data in the same scene includes: For multi-stage data in the same scenario, extracting data themes in each single stage of the multi-stage data; Perform clustering processing based on the data themes extracted in each single stage to obtain clustering information of each single stage; Based on the correlation between the clustering information of two adjacent single stages, the association information is generated.

[0014] In the above solution, the association information is a numerical matrix representing the matching relationship between the data subjects of each single stage or a knowledge set representing the matching relationship between the data subjects of each single stage.

[0015] In a second aspect, an embodiment of the present application provides a data generating device, including: An acquisition module, used for acquiring generated data corresponding to multiple single-stage data generation; The processing module is used to assemble the generated data of each single stage based on the association information to obtain the multi-stage data expected to be generated; wherein the association information represents the association relationship between the generated data of each single stage.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when used to run the computer program, executes the steps of the method described in the first aspect of the embodiment of the present application.

[0017] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect of the embodiment of the present application are implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect of the embodiment of the present application.

[0019] The technical solution provided in the embodiment of the present application obtains the generated data corresponding to the data generation of multiple single stages; assembles the generated data of each single stage based on the associated information to obtain the desired generated multi-stage data; wherein the associated information represents the associated relationship between the generated data of each single stage. In this way, the generated data corresponding to the data generation of multiple single stages can be assembled based on the associated information, which can not only expand the data scale of the generated multi-stage data based on the combination of the generated data of multiple single stages, but also effectively converge on the data quality dimension, improve the data generation quality, and then effectively improve the effect of multi-stage data generation to meet the application requirements of downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a flow chart of a data generation method according to an embodiment of the present application; Figure 2 This is a schematic diagram of the structure of a data generating device according to an embodiment of the present application; Figure 3It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0023] Data generation technology has become a research focus in the era of deep learning. The paper "Attributealignment: Controlling text generation from pre-trained language models" proposes a method called "Attribute Alignment" to control text generation to adapt to target attributes, such as sentiment polarity or specific topics, without changing the parameters of the pre-trained language model (LM-GPT2). The paper "Control, generate, augment: A scalable framework for multi-attribute text generation" introduces a framework called CGA (Control, Generate, Augment), which is a scalable framework for multi-attribute text generation. CGA combines adversarial learning, context-aware loss, and cyclic vocabulary loss strategies to control multiple semantic and syntactic attributes of generated natural English sentences.

[0024] The era of large language models has opened up new possibilities for data generation. The paper "Language models are realistic tabular data generators" proposes a new method called GReaT (Generation of Realistic Tabular data), which uses large models to generate synthetic samples (tabular data) with the characteristics of original data. The main contributions of GReaT include: 1. A new paradigm is proposed to model and generate real heterogeneous tabular data using a large model, connecting tabular and textual data modalities through a text encoding scheme.

[0025] 2. Support arbitrary conditioning. When trained on text encodings with random feature order, the model inherits the arbitrary conditioning capability of LLM and can model the data distribution of any given feature subset and sample the remaining features.

[0026] 3. We provide extensive experimental results, demonstrating that GReaT achieves state-of-the-art generation performance on a variety of datasets, and open-source the experimental results to provide a strong benchmark for the community.

[0027] 4. An easy-to-use Python implementation is provided, which can generate new synthetic samples with only three lines of code.

[0028] The paper "Generating datasets with pretrained language models" proposes a method called DINO (Datasets from Instructions), which uses large models to create text pairs by providing natural language instructions to provide high-quality sentence embeddings for small models.

[0029] Specifically, the DINO method is implemented through the following steps: 1. Use a pre-trained language model (such as GPT-2-XL) to generate text pairs with semantic similarity labels.

[0030] 2. The self-debiasing algorithm is used to ensure that the generated text pairs not only conform to the given similarity labels but also do not conform to other labels to reduce noise.

[0031] 3. Use the generated dataset to train smaller models, such as Sentence-RoBERTa, to improve its performance on multiple semantic text similarity datasets.

[0032] The contributions of the paper include: 1. We introduce the DINO method, which can automatically generate annotated datasets of any size.

[0033] 2. Released STS-Dino, the first fully automatically generated text similarity dataset that does not require any human annotation.

[0034] 3. We show that Sentence-RoBERTa trained on the dataset generated by DINO outperforms strong baseline models on multiple semantic text similarity datasets.

[0035] In addition, the paper also explores the importance of self-debiasing algorithms in improving the quality of generated data, and conducts a small-scale manual evaluation to evaluate the quality of the generated dataset. The experimental results show that although there is a certain degree of noise in the generated dataset, by taking appropriate measures to deal with this noise, the trained model can still achieve excellent performance on multiple semantic text similarity datasets.

[0036] However, the above papers focus more on general data generation, and do not provide specific methodologies for data generation in vertical segments such as network security data, making it difficult to meet the training needs of large models in vertical fields. In addition, in related technologies, data generation technologies provided by large model manufacturers are mostly single-stage generation solutions, which cannot systematically solve multi-stage data generation tasks.

[0037] Specifically, taking the field of network security as an example, the current data generation technology in the field of network security mainly uses technologies such as transfer learning to achieve single-stage data expansion. For example, the attack methods in one field are expanded into data of dozens of attack methods based on the ATT&CK attack matrix. Among them, ATT&CK (Adversarial Tactics, Techniques & Common Knowledge) is a knowledge base developed by MITRE based on the observation of attack tactics and techniques in real environments. Its advantage is that it can greatly expand the scale of data attacks, and its disadvantages are: 1) Single-stage data generation and quality assessment methods have been extensively studied. Although they can effectively expand the data scale, they can only achieve linear expansion (usually up to 20-50 times) and are difficult to achieve larger-scale expansion. 2) If multi-stage data generation is performed based on single-stage experience, it is difficult to converge on the quality dimension. Although a large amount of data can be generated, the data quality cannot be guaranteed (poor diversity, high token duplication; uneven coverage and quality of various scenarios), resulting in the data being difficult to meet the needs of downstream tasks.

[0038] Based on this, in various embodiments of the present application, association information is introduced to characterize the association relationship between the generated data generated by each single stage, and the generated data of multiple single stages are assembled based on the association information to obtain the desired multi-stage data. This can not only expand the generation scale of the multi-stage data, but also reduce invalid or redundant data, improve the generation quality of the multi-stage data, and thus effectively improve the generation effect of the multi-stage data to meet the application requirements of downstream tasks.

[0039] For example, the embodiment of the present application provides a data generation method, which can be independently applied to electronic devices with data processing capabilities, such as terminal devices, servers, etc., or can be implemented by the cooperation between terminal devices and servers; wherein the terminal device can specifically be a computer, a smart phone, a personal digital assistant (PDA), etc.; the server can specifically be an application server or a Web server. In actual deployment, the server can be an independent server or a cluster server. Figure 1 As shown, the method includes: Step 101, obtaining multiple single-stage data to generate corresponding generated data.

[0040] Step 102, assembling the generated data of each single stage based on the association information to obtain the multi-stage data expected to be generated; wherein the association information represents the association relationship between the generated data of each single stage.

[0041] It should be noted that in the embodiment of the present application, the data generation task under the set scenario is implemented in a multi-stage generation manner, and specifically, a preset scenario template can be used to define the multi-stage generation manner. Among them, the generated data corresponding to the acquired multiple single-stage data generation is implemented based on the scenario template corresponding to the set scenario, and the generated data of each single stage is generated based on the subject selected by the user. The subject can be understood as a field that defines the category of the generated data in each single stage. Taking the email security scenario as an example, the multi-stage data generation task can be divided into: the first single stage corresponding to the main text and the second single stage corresponding to the generation of attachments, that is, the selected subject of the first single stage can be determined based on multiple subject options related to the main text, and the selected subject of the second single stage can be determined based on subject options related to the attachments. Among them, the number of subject options for each single stage can be multiple, and other scenarios can be understood by reference, which will not be repeated here.

[0042] This embodiment assembles the generated data of each single stage by introducing related information. On the one hand, it can expand the data scale of the generated multi-stage data based on the combination of multiple single-stage generated data. On the other hand, it can ensure the effective convergence of multi-stage data in the data quality dimension, improve the data generation quality, and further effectively improve the effect of multi-stage data generation to meet the application requirements of downstream tasks.

[0043] It should be noted that the association information in the embodiment of the present application is used to characterize whether the combination of generated data in a single stage meets the data generation requirements. This is conducive to ensuring effective convergence in the quality dimension during multi-stage data generation.

[0044] Exemplarily, the step of obtaining the generated data corresponding to the data generation of multiple single stages in a set scenario includes: Obtaining identity definition information representing the desired role; Based on the identity definition information, the scenario templates of various preset roles are matched to obtain a matching target scenario template; Configuring theme options for each single stage in the target scenario template; Based on the target scenario template after configuring the theme options, the generated data corresponding to the data generation of each single stage is obtained.

[0045] Here, the electronic device can receive the identity definition information input by the user, and the identity definition information is used to define the expected role, which is the role played by the data generator. For example, the data generation agent (Agent) as the data generator generates multi-stage data in the identity of the expected role. After obtaining the identity definition information input by the user, the electronic device can match it with the scenario templates of various roles built in advance to obtain a matching target scenario template. Among them, each scenario template can be understood as a template corresponding to the role for constraining the generation of prompt words, which can effectively improve the generation quality of prompt words.

[0046] Exemplarily, based on the target scenario template after configuring the theme options, the generated data corresponding to the data generation of each single stage is obtained, including: Generate prompt words for data generation in each single stage based on the target scenario template after configuring the theme options; The prompt words are input into a preset model to obtain generated data corresponding to the data generation of each single stage.

[0047] It should be noted that the prompt word is Prompt, which is used to clearly inform the model of the content of the data that needs to be generated, thereby guiding the model to generate data that meets the requirements. The prompt word can be a simple instruction or question, or it can include a complex description with rich context and relevant requirements.

[0048] Here, the preset model may be one or more large language models, for example, a GPT (Generative Pre-Trained Transformer) model. After the prompt words are input into the preset model, the preset model may obtain the generated data of each single stage based on the prompt words of each single stage.

[0049] Exemplarily, the method further includes: Divide multiple preset roles based on data application scenarios; Based on each preset role and setting field, a corresponding scenario template is configured; wherein the setting field is used to define a prompt word framework corresponding to the scenario template.

[0050] Here, data can be divided based on the scenarios of data application to obtain multiple mainstream application scenarios in various business fields. For example, in the field of network security, it can include but is not limited to: email security, IoT device security, banking and financial security, cloud storage and backup, remote office, social media and other scenarios. Among them, one or more preset roles can be divided based on demand in each scenario. For example, multiple preset roles can be divided based on the application stage of the scenario, so as to better meet the data segmentation generation needs of vertical business fields, and can effectively guide users to enter the identity definition information of the role in segments, thereby laying a foundation for improving the quality of data generation.

[0051] It should be noted that the data generation quality of the model depends largely on the input prompt words. Based on this, a scientific and reasonable prompt word framework is particularly important. Among them, each preset role can preset a corresponding prompt word framework and configure the scenario template based on the corresponding setting field. Each prompt word framework has a corresponding setting field. For example, the prompt word framework and the corresponding setting field of the embodiment of the present application include but are not limited to one or more of the following: 1) ICIO framework, which includes the following setting fields: Instruction: Instructions, that is, the specific tasks you want the AI ​​to perform; Context: Background information. Giving AI more background information can guide the model to make responses that are more in line with the needs. Input Data: Input data, telling the model what data needs to be processed; Output Indicator: Output guide, telling the model the type or style of user output.

[0052] 2) CRISPE framework, which includes the following setting fields: Capacity and Role: Capacity and role, what role the model should play; Insight: Provides insights, background, and context behind user requests; Statement: what the user wants the model to do; Personality: the style, personality or way in which the user wants the model to respond; Experiment: Experiment, requesting the model to reply with multiple examples.

[0053] 3) BROKE framework, which includes the following setting fields: Background: Background, explain the background and provide sufficient information for the model; Role: the role you want the model to play; Objectives: what you want to achieve; Key Result: What specific effect should be tested and adjusted? Evolve: Experiment and improve, freely combine the following three improvement methods: a. Improve input: Start from the shortcomings of the answer to improve the background B, goal O and key results R; b. Improve answers: Point out shortcomings in the model’s answers in subsequent conversations; c. Regenerate: Try to generate results multiple times without changing the prompt, and select the best one.

[0054] 4) APE frame, which includes the following setting fields: ACTION: Action, defines the work or activity to be done; PURPOSE: purpose, discuss intention or goal; EXPECTATION: Expectation, stating the expected outcome.

[0055] 5) COAST framework, which includes the following setting fields: CONTEXT: Context sets the stage for the conversation; OBJECTIVE: Purpose, describing the goal; ACTION: Action, explain the required action; SCENARIO: scenario, describing the scene; TASK: Task, describe the task.

[0056] 6) TAG frame, which includes the following setting fields: TASK: task, describe the task; ACTION: Action, explain the required action; GOAL: Goal, explain the final goal.

[0057] 7) RISE framework, which includes the following setting fields: ROLE: role, specifies the role of the model; INPUT: input, describing information or resources; STEPS: Steps, ask for detailed steps; EXPECTATION: Expectation, describing the desired outcome.

[0058] 8) TRACE frame, which includes the following setting fields: TASK: Task, define a specific task; REQUEST: request, describing the requirements; ACTION: Action, indicating the required operation; CONTEXT: context, providing contextual information or situation; EXAMPLE: Example, giving an example to illustrate the effect the user wants to achieve.

[0059] 9) ERA framework, which includes the following setting fields: EXPECTATION: Expectation, describing the desired outcome; ROLE: role, specify the role; ACTION: Action, specifies what actions need to be taken.

[0060] 10) CARE framework, which includes the following setting fields: CONTEXT: context, providing contextual information or situation; ACTION: Action, indicating the required operation; RESULT: Result, describing the desired result; EXAMPLE: Example, giving an example to illustrate the effect the user wants to achieve.

[0061] 11) ROSES framework, which includes the following setting fields: ROLE: role, specify the role; OBJECTIVE: purpose, state the goal; SCENARIO: scenario, describing the situation; EXPECTED SOLUTION: Solution, defines the desired outcome; STEPS: Steps required to achieve the goal.

[0062] 12) PATFU frame, which includes the following setting fields: PROBLEM: Problem, clearly express the problem that needs to be solved; AERA: Field, the field of the problem or the role to be played; TASK: Task, the specific task that needs to be performed to solve this problem; FORMAT: Format, which defines the output format and restrictions in detail; UPDATE: Iterate, record the prompt word version and iterate the prompt word according to the output results.

[0063] It is understandable that multiple preset roles may also share the same prompt word framework, that is, the setting fields of multiple preset roles may be the same, and the embodiment of the present application does not limit this.

[0064] Exemplarily, the generating of prompt words for data generation in each single stage based on the target scenario template after configuring the theme options includes: Based on the auxiliary information and / or preset information input by the user for generating prompt words, the corresponding setting fields in the target scenario template after each single-stage configuration theme option are filled to generate prompt words for data generation in each single stage.

[0065] Here, the electronic device can fill in the set fields in the target scenario template based on the preset information on the platform side to obtain the prompt words output by the reference model, or fill in the set fields in the target scenario template based on the auxiliary information input by the user and the preset information on the platform side to obtain the prompt words output by the reference model. Among them, the auxiliary information input by the user can be information corresponding to one or more fields in the prompt word framework, and the preset information on the platform side can be understood as the configuration information pre-defined for each field in the prompt word framework.

[0066] In one application example, the electronic device can establish a correspondence between the auxiliary information input by the user and / or the platform preset information and the set fields in the target scenario template based on the knowledge graph of the vertical business field. That is, the correspondence is determined by utilizing the relationship between entities and attributes in the knowledge graph, and then each set field is automatically filled in to obtain prompt words for guiding model output.

[0067] Exemplarily, the filling of setting fields in the target scenario template based on the auxiliary information and / or preset information input by the user for generating prompt words to generate prompt words for guiding the model output includes: Based on the relationship between entities and attributes in the pre-constructed knowledge graph, the setting fields in the targeted scenario template corresponding to the auxiliary information and / or preset information input by the user are determined, and the auxiliary information and / or preset information are filled into the corresponding setting fields to obtain prompt words for guiding the model output.

[0068] Here, the data knowledge of the vertical business field can be expressed in the form of a knowledge graph, so that auxiliary information and / or preset information can be automatically filled into the prompt word framework of the targeted scenario template based on the knowledge graph, thereby realizing the automatic assembly of prompt words and improving the automation level of prompt word generation corresponding to each scenario template.

[0069] Exemplarily, the step of assembling the generated data of each single stage based on the associated information to obtain the desired generated multi-stage data includes: Based on the associated information, select the data topics of each single stage whose matching relationship meets the requirements; Based on the generated data corresponding to the selected data themes of each single stage, the data is assembled in the order of stage combination to obtain the expected multi-stage data.

[0070] It should be noted that the association information for each scenario can be pre-constructed. Exemplarily, the association information can be represented based on the topic categories after each single-stage clustering, that is, represented by the correlation between the topic categories between two adjacent single stages, for example, using a numerical matrix that characterizes the matching relationship between the data topics of each single stage, wherein the higher the correlation value in the numerical matrix (also called the correlation matrix), the higher the probability of data combination between the two topic categories, and the lower the correlation value, the lower the probability of data combination between the two topic categories. The order of stage combination can be determined based on the knowledge graph of the application scenario to which it belongs.

[0071] In another example, the associated information can also be a knowledge set that characterizes the matching relationship between data topics of each single stage. For example, each data topic can include content such as "key elements" and "data text", where the key elements can be understood as the data dimensions under the data topic, and the data text can be understood as the data content obtained by assembling the data based on each data dimension. In this way, through the matching relationship between data topics, a combination relationship of key elements between different data topics can be established, and then the combined data text, that is, the multi-stage data text, is obtained, thereby improving the quality of multi-stage data.

[0072] In one example, if the association information is the aforementioned numerical matrix, selecting data topics of each single stage whose matching relationship meets the requirements based on the association information may include: Matching the attributes of the generated data of each single stage with the preset subject categories to divide the generated data of each single stage into different subject categories; Based on the association information, the association degree corresponding to the combination of two adjacent single-stage topic categories is obtained; Select the topic category combination whose correlation is greater than or equal to the set threshold to obtain the data topics of each single stage that meet the requirements.

[0073] In this way, the correlation between the topic combinations of two adjacent single stages can be obtained based on the correlation information corresponding to the clustered topic categories, thereby improving the efficiency of screening single-stage data topics that meet the requirements.

[0074] Exemplarily, each data topic in the topic options includes key elements supporting configuration, and the method further includes: Replacing the key elements based on the configuration file; Based on the replacement of the key elements, the generated prompt words are generalized to obtain batch prompt words; The step of inputting the prompt word into a preset model to obtain the generated data corresponding to the data generation of each single stage includes: The batch prompt words are input into a preset model to obtain batch generation data of each single stage.

[0075] Here, the electronic device can receive a configuration file uploaded by a user, and the configuration file can be a pre-prepared formatted data file, wherein the variables supporting configuration (i.e., the key elements under the theme) can be replaced by the content in the configuration file, for example, the configuration file can include various fields and corresponding value spaces, and the fields corresponding to the variables can be set, and the variables can be replaced based on the value spaces of the fields in the configuration file. For example, taking the generation of news data as an example, the key elements under the data theme can include regions and news topics, wherein the regions in the formatted data file can include various regional information such as countries, provinces, and cities, and the news topics can include: politics, economy, society, sports, science and technology, culture, entertainment, environment, military, health, and other examples. In this way, the variables in the prompt words can be replaced based on the formatted data file, and then batch prompt words are obtained. The electronic device then inputs the batch prompt words into a preset model to obtain batch generated data. The batch generated data can greatly enrich the diversity of the data and the comprehensiveness of the coverage, and avoid the defects of insufficient richness and coverage of the data generated by a single prompt word, thereby effectively improving the quality of the generated data and meeting the data generation needs of the vertical business field. For example, in the email security scenario, the key element of the data subject can be the "vernacular preparation" in the phishing attack scenario. The attribute value space of the "vernacular preparation" can include: subsidy notification, preferential notification, procurement notification, tax refund notification, salary notification, etc., and then the prompt words can be generalized based on the variable replacement results to obtain batch prompt words.

[0076] Exemplarily, the number of the preset models is multiple, and the step of inputting the batch prompt words into the preset models to obtain batch generated data includes: Based on the distributed calling mechanism, the batched prompt words are divided and input into each preset model in batches to obtain batched generated data.

[0077] Here, in order to improve the efficiency of data generation by the data generation agent, in an embodiment of the present application, for multiple preset models, the electronic device can also divide the batched prompt words based on a distributed calling mechanism and input them into each preset model in batches to obtain batched generated data. In this way, the effect of load balancing control can be achieved based on the distributed calling mechanism, thereby meeting the data generation quality while greatly improving the data generation efficiency.

[0078] Exemplarily, the method further includes: Acquire a data set in each scenario, where each data in the data set is multi-stage data; Based on multi-stage data in the same scene, the associated information of the corresponding scene is constructed.

[0079] Here, the data set can be extracted based on high-quality file data. For example, for network security scenarios, seed data can be extracted from high-quality file data and the seed data can be used as the data set.

[0080] Exemplarily, the constructing of association information of corresponding scenes based on multi-stage data in the same scene includes: For multi-stage data in the same scenario, extracting data themes in each single stage of the multi-stage data; Perform clustering processing based on the data themes extracted in each single stage to obtain clustering information of each single stage; Based on the correlation between the clustering information of two adjacent single stages, the association information is generated.

[0081] Here, the clustering information refers to the aforementioned topic categories, that is, dividing the data topics into multiple topic categories, and forming related information based on the correlation between the topic categories. In this way, it is convenient to assemble multiple single-stage generated data based on the correlation between the topic categories between multiple stages when generating data, thereby improving data generation efficiency.

[0082] Here, the correlation between topic categories can be determined by the number of repetitions of multi-stage data between different topic categories, or can be determined based on the similarity of multi-stage data, which is not limited in this embodiment of the present application.

[0083] For example, for the email security scenario in the field of network security, assuming that the data set includes 20,000 complete multi-stage data, the number of data topics in the first stage corresponding to the text data is 10,000, and the number of data topics in the second stage corresponding to the attachment data is 8,000. Through clustering processing, the number of topic categories in the first stage is reduced to 100, and the number of topic categories in the second stage is reduced to 80. That is, the correlation between the 100 topic categories in the first stage and the 80 topic categories in the second stage can be obtained. When the subsequent multi-stage data is generated, the screening efficiency before data combination can be improved based on the clustered correlation information, so that the quality of the generated multi-stage data is reasonable and the data volume is greatly expanded. For example, when multi-stage data is generated, quality assessment and filtering can be performed based on related information to prevent improper multi-stage assembly. Taking the scenario of phishing email attack as an example, stage 1 generates the main text (100 options), and stage 2 adds links / attachments (80 options); if stage 1 selects "tax refund" in the main text and mentions "please check the attachment", stage 2 can use PDF attachments, Excel attachments, or even links, but usually not QR codes. In this way, through optimized combinations, stage 1 selects 80 options and stage 2 selects 60 options. Based on the combination of the two, 4,800 types of phishing email attack data generated based on the combination can be obtained.

[0084] In order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a data generating device, which corresponds to the above-mentioned data generating method, and each step in the above-mentioned data generating method embodiment is also fully applicable to the embodiment of the present data generating device.

[0085] like Figure 2 As shown, the data generation device includes: an acquisition module 201 and a processing module 202. The acquisition module 201 is used to acquire the generated data corresponding to the data generation of multiple single stages; the processing module 202 is used to assemble the generated data of each single stage based on the associated information to obtain the desired generated multi-stage data; wherein the associated information represents the associated relationship between the generated data of each single stage.

[0086] Exemplarily, the processing module 202 is specifically configured to: Based on the associated information, select the data topics of each single stage whose matching relationship meets the requirements; Based on the generated data corresponding to the selected data themes of each single stage, the data is assembled in the order of stage combination to obtain the expected multi-stage data.

[0087] Exemplarily, the acquisition module 201 is specifically used for: Obtaining identity definition information representing the desired role; Based on the identity definition information, the scenario templates of various preset roles are matched to obtain a matching target scenario template; Configuring theme options for each single stage in the target scenario template; Based on the target scenario template after configuring the theme options, the generated data corresponding to the data generation of each single stage is obtained.

[0088] Exemplarily, the acquisition module 201 obtains the generated data corresponding to the data generation of each single stage based on the target scenario template after configuring the theme options, including: Generate prompt words for data generation in each single stage based on the target scenario template after configuring the theme options; The prompt words are input into a preset model to obtain generated data corresponding to the data generation of each single stage.

[0089] Exemplarily, the data generating device further includes: a configuration module 203, configured to: Divide multiple preset roles based on data application scenarios; Based on each preset role and setting field, a corresponding scenario template is configured; wherein the setting field is used to define a prompt word framework corresponding to the scenario template.

[0090] Exemplarily, each data topic in the topic options includes key elements supporting configuration, and the acquisition module 201 is further used to: Replacing the key elements based on the configuration file; Based on the replacement of the key elements, the generated prompt words are generalized to obtain batch prompt words; The batch prompt words are input into a preset model to obtain batch generation data of each single stage.

[0091] Exemplarily, the configuration module 203 is further used for: Acquire a data set in each scenario, where each data in the data set is multi-stage data; Based on multi-stage data in the same scene, the associated information of the corresponding scene is constructed.

[0092] Exemplarily, the configuration module 203 is specifically used for: For multi-stage data in the same scenario, extracting data themes in each single stage of the multi-stage data; Perform clustering processing based on the data themes extracted in each single stage to obtain clustering information of each single stage; Based on the correlation between the clustering information of two adjacent single stages, the association information is generated.

[0093] Exemplarily, the association information is a numerical matrix representing the matching relationship between data subjects in each single stage or a knowledge set representing the matching relationship between data subjects in each single stage.

[0094] In actual application, the acquisition module 201, the processing module 202 and the configuration module 203 can be implemented by a processor in the electronic device. Of course, the processor needs to run the computer program in the memory to implement its functions.

[0095] It should be noted that: when the data generation device provided in the above embodiment performs data generation, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the data generation device provided in the above embodiment and the data generation method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0096] Based on the hardware implementation of the above program modules and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an electronic device. Figure 3 Only an exemplary structure of the electronic device is shown, not all structures, and it can be implemented as needed. Figure 3 Partial or complete structure shown.

[0097] like Figure 3 As shown, the electronic device 300 provided in the embodiment of the present application includes: at least one processor 301, a memory 302, a user interface 303 and at least one network interface 304. The various components in the electronic device 300 are coupled together through a bus system 305. It can be understood that the bus system 305 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 305 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 3 Various buses are labeled as bus system 305 .

[0098] The user interface 303 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0099] The memory 302 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0100] The data generation method disclosed in the embodiment of the present application can be applied to the processor 301, or implemented by the processor 301. The processor 301 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the data generation method can be completed by the hardware integrated logic circuit or software instructions in the processor 301. The above-mentioned processor 301 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 301 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 302, and the processor 301 reads the information in the memory 302, and completes the steps of the data generation method provided in the embodiment of the present application in combination with its hardware.

[0101] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0102] It can be understood that the memory 302 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0103] In an exemplary embodiment, the present application also provides a computer storage medium, which can be a computer-readable storage medium, for example, a memory 302 storing a computer program, and the computer program can be executed by a processor 301 of an electronic device to complete the steps described in the method of the present application embodiment. The computer-readable storage medium can be a memory such as a ROM, a PROM, an EPROM, an EEPROM, a Flash Memory, a magnetic surface memory, an optical disk, or a CD-ROM.

[0104] In an exemplary embodiment, the embodiment of the present application further provides a computer program product, including a computer program, which can be executed by the processor 301 of the electronic device 300 to complete the steps described in the method of the embodiment of the present application.

[0105] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0106] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0107] The above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A data generation method, characterized in that: include: Obtaining generated data corresponding to multiple single-stage data generation; The generated data of each single stage are assembled based on the association information to obtain the expected multi-stage data; wherein the association information represents the association relationship between the generated data of each single stage.

2. The method according to claim 1, characterized in that The generating data of each single stage is assembled based on the association information to obtain the desired generated multi-stage data, including: Based on the associated information, select the data topics of each single stage whose matching relationship meets the requirements; Based on the generated data corresponding to the selected data themes of each single stage, the data is assembled in the order of stage combination to obtain the expected multi-stage data.

3. The method according to claim 1, characterized in that The step of obtaining data from a plurality of single stages to generate corresponding generated data includes: Obtaining identity definition information representing the desired role; Based on the identity definition information, the scenario templates of various preset roles are matched to obtain a matching target scenario template; Configuring theme options for each single stage in the target scenario template; Based on the target scenario template after configuring the theme options, the generated data corresponding to the data generation of each single stage is obtained.

4. The method according to claim 1, characterized in that: The method further comprises: Acquire a data set in each scenario, where each data in the data set is multi-stage data; Based on multi-stage data in the same scene, the associated information of the corresponding scene is constructed.

5. The method according to claim 4, characterized in that The construction of the associated information of the corresponding scene based on the multi-stage data in the same scene includes: For multi-stage data in the same scenario, extracting data themes in each single stage of the multi-stage data; Perform clustering processing based on the data themes extracted in each single stage to obtain clustering information of each single stage; Based on the correlation between the clustering information of two adjacent single stages, the association information is generated.

6. The method according to any one of claims 1 to 5, characterized in that: The association information is a numerical matrix representing the matching relationship between the data subjects of each single stage or a knowledge set representing the matching relationship between the data subjects of each single stage.

7. A data generating device, characterized in that: include: An acquisition module, used for acquiring generated data corresponding to multiple single-stage data generation; The processing module is used to assemble the generated data of each single stage based on the association information to obtain the multi-stage data expected to be generated; wherein the association information represents the association relationship between the generated data of each single stage.

8. An electronic device, characterized in that: include: A processor and a memory for storing a computer program that can be executed on the processor, wherein: The processor is used to execute the steps of the method according to any one of claims 1 to 6 when running a computer program.

9. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images