Data generation method, device, equipment and readable storage medium

Abstract

This application discloses a data generation method, apparatus, device, and readable storage medium. The method includes: decomposing an application programming interface (API) at different granularities to obtain multiple API sub-interfaces; performing type statistics on the multiple API sub-interfaces and determining a generation ratio based on the statistical results; obtaining prompt words corresponding to each of the multiple API sub-interfaces; inputting the prompt words into a text language model for data generation processing to obtain instruction-result pairs; wherein the text language model has learned the technical documentation and usage guidelines of the API; storing the instruction-result pairs in a dataset and performing statistics on the instruction-result pairs in the dataset to obtain distribution data; and determining the dataset as the target dataset if the distribution data matches the generation ratio. The technical effect of this application is that it can generate a target dataset with high instruction coverage and customizable data distribution.

Description

Technical Field

[0001] This application relates to the field of information processing technology, and in particular to a data generation method, apparatus, device, and readable storage medium. Background Technology

[0002] API documentation and usage guides are typically geared towards expert users, aiming to accurately describe API functionality and avoid ambiguity. To this end, standardized language and uniform formats are usually used to describe and define APIs. However, when ordinary or non-expert users use APIs, they often face significant barriers to understanding, requiring them to consult documentation to comprehend the API's functionality and usage. This comprehension gap significantly increases the difficulty of correctly invoking the API.

[0003] Currently, API calls and parameter settings are controlled via natural language processing. Advances in Natural Language Processing (NLP) technology have made the semantic understanding of models more accurately aligned with code. However, while models exhibit excellent generalization capabilities across a wide range of domains, the base model's data often struggles to cover customized API functionalities. In such cases, further pre-training or supervised fine-tuning (SFT) using a large amount of task data is necessary to enhance the model's understanding of specific domains.

[0004] In order for generative models (such as GPT) to master domain-specific APIs, they must be trained on a large dataset containing instruction-result pairs (including the correspondence between input instructions and API results; input instructions are user-input descriptive instructions, and API results are the procedural instructions that describe those instructions). However, current methods for generating such datasets suffer from problems such as low instruction coverage and imbalanced data distribution, further hindering generative models from effectively learning a more comprehensive API.

[0005] In summary, how to effectively solve the problem of generating datasets for training generative models is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this application is to provide a data generation method, apparatus, device, and readable storage medium to obtain a target dataset with high instruction coverage and customizable data distribution.

[0007] To solve the above-mentioned technical problems, this application provides the following technical solution:

[0008] A dataset generation method, comprising:

[0009] The application programming interface (API) is decomposed at different granularities to obtain multiple API sub-interfaces;

[0010] Type statistics are performed on multiple application programming sub-interfaces, and the generation ratio is determined based on the statistical results;

[0011] Obtain the prompt words corresponding to each of the multiple application programming sub-interfaces;

[0012] The prompt words are input into a text language model for data generation processing to obtain instruction result pairs; wherein, the text language model has learned the technical documentation and usage guidelines of the application programming interface;

[0013] The instruction result pairs are stored in the dataset, and the instruction result pairs in the dataset are statistically analyzed to obtain distribution data;

[0014] If the distribution data matches the generation ratio, the dataset is determined as the target dataset.

[0015] Preferably, the application programming interface is decomposed at different granularities to obtain multiple application programming sub-interfaces, including:

[0016] Obtain the declaration information of the application programming interface;

[0017] Based on the aforementioned declaration information, different granularities are determined;

[0018] The program programming interface is split at different granularities to obtain multiple application programming sub-interfaces.

[0019] Preferably, if the distribution data does not match the generation ratio, the method further includes:

[0020] The dataset is supplemented by acquiring new prompt words so that the distribution data of the supplemented dataset matches the generation ratio.

[0021] Preferably, the dataset is supplemented by acquiring new prompt words, including:

[0022] By comparing the distribution data with the generation ratio, the target type of the missing instruction result pair is determined;

[0023] The application programming sub-interface that matches the target type is identified as the target application programming sub-interface;

[0024] Retrieve the prompt word corresponding to the target application programming sub-interface;

[0025] The newly acquired prompt words are input into the text language model for data generation and processing to obtain supplementary instruction result pairs;

[0026] The supplementary instruction results are written into the dataset.

[0027] Preferably, the generation ratio is determined based on statistical results, including:

[0028] The statistical results are output in the visualization interface;

[0029] Obtain the input configuration information;

[0030] The generation ratio is determined using the configuration information.

[0031] Preferably, determining the generation ratio using the configuration information includes:

[0032] The category ratios of each category are read from the configuration information, and the generation ratio is determined based on the category ratios.

[0033] Preferably, determining whether the distribution data matches the generation ratio includes:

[0034] The subclass ratios of each category are obtained using the aforementioned category ratios;

[0035] Read the minimum number of extended records for each subclass from the configuration information;

[0036] Using the subclass ratio and the minimum number of extended records, determine the minimum number of records corresponding to the subclass;

[0037] Calculate the minimum total number of records that satisfy the category matching based on the minimum number of records;

[0038] Determine whether the minimum number of records, the minimum total number of records, and the distribution data match.

[0039] If so, then the distribution data is determined to match the generation ratio;

[0040] If not, then it is determined that the distribution data does not match the generation ratio.

[0041] A data generation apparatus, comprising:

[0042] The splitting module is used to decompose the application programming interface at different granularities, resulting in multiple application programming sub-interfaces;

[0043] A generation ratio statistics module is used to perform type statistics on multiple application programming sub-interfaces and determine the generation ratio based on the statistical results;

[0044] The prompt processing module is used to obtain the prompt words corresponding to the multiple application programming sub-interfaces respectively;

[0045] The instruction result pair generation module is used to input the prompt words into the text language model for data generation processing to obtain instruction result pairs; wherein, the text language model has learned the technical documentation and usage guidelines of the application programming interface;

[0046] The analysis module is used to store the instruction result pairs into a dataset and perform statistics on the instruction result pairs in the dataset to obtain distribution data;

[0047] The data generation module is used to determine the dataset as the target dataset when the distributed data matches the generation ratio.

[0048] An electronic device, comprising:

[0049] memory for storing computer programs;

[0050] A processor is used to implement the steps of the above-described data generation method when executing the computer program.

[0051] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described data generation method.

[0052] Applying the method provided in the embodiments of this application, the application programming interface (API) is decomposed at different granularities to obtain multiple API sub-interfaces; type statistics are performed on the multiple API sub-interfaces, and the generation ratio is determined based on the statistical results; prompt words corresponding to each of the multiple API sub-interfaces are obtained; the prompt words are input into a text language model for data generation processing to obtain instruction-result pairs; wherein, the text language model has learned the technical documents and usage guidelines of the API; the instruction-result pairs are stored in a dataset, and the instruction-result pairs in the dataset are statistically analyzed to obtain distribution data; if the distribution data matches the generation ratio, the dataset is determined as the target dataset.

[0053] Application programming interfaces (APIs) are broken down into sub-APIs of varying granularity. These sub-APIs are then statistically analyzed to determine the generation ratio. To ensure a more comprehensive dataset containing instruction-result pairs related to the APIs, prompts corresponding to each sub-API are retrieved during prompt generation. Then, a textual language model based on the API's technical documentation and user guides is used for data generation to produce instruction-result pairs. These pairs are then added to the dataset, and statistical analysis is performed. Once the distribution of the data matches the production ratio, the dataset generation is considered complete, and the target dataset is obtained.

[0054] The technical advantages of this application are as follows: When generating instruction-result pairs in the dataset, by splitting the application programming interface (API), the instruction-result pairs can cover instructions of different granularities of the API. By statistically analyzing the API sub-interfaces, the generation ratio can be determined. After obtaining the distribution data of instruction-result pairs in the dataset, when the generation ratio matches the distribution data, the target dataset can be determined. This makes the data distribution of the final generated target dataset controllable, providing comprehensive and customizable training samples for further training of generative models.

[0055] Accordingly, embodiments of this application also provide a data generation apparatus, device, and readable storage medium corresponding to the above-described data generation method, which have the aforementioned technical effects, and will not be elaborated further here. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating the implementation of a data generation method in an embodiment of this application.

[0058] Figure 2 This is a schematic diagram illustrating an implementation of a data generation method in an embodiment of this application;

[0059] Figure 3 This is a schematic diagram of the structure of a data generation device according to an embodiment of this application;

[0060] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0061] Figure 5 This is a schematic diagram of the specific structure of an electronic device in an embodiment of this application. Detailed Implementation

[0062] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] Please refer to Figure 1 , Figure 1 This is a flowchart of a data generation method according to an embodiment of this application. The method includes the following steps:

[0064] S101. The application programming interface is split at different granularities to obtain multiple application programming sub-interfaces.

[0065] For ease of description, we will use the abbreviation API for Application Programming Interface, and refer to the sub-APIs as sub-APIs. Sub-APIs are the sub-functional modules corresponding to APIs.

[0066] In this application, different granularities can be pre-set, and then the API can be split based on the different granularities to obtain multiple sub-APIs.

[0067] For example, if the internal functions of an API are visualized as a tree structure according to parent-child relationships and logical architecture, then the path from the root node to each leaf node can correspond to a sub-API.

[0068] In one specific embodiment of this application, the application programming interface is decomposed at different granularities to obtain multiple application programming sub-interfaces, including:

[0069] Step 1: Obtain the declaration information of the application programming interface;

[0070] Step 2: Determine different granularities based on the declared information;

[0071] Step 3: Decompose the program programming interface at different granularities to obtain multiple application programming sub-interfaces.

[0072] For ease of description, the above steps will be combined below.

[0073] In this embodiment, API declaration information can be obtained by reading from a storage device or receiving input.

[0074] Figure 2 As shown, this declaration information can be explanatory information about the API declaration provided by technical personnel (developers). The declaration information (i.e., belonging to root requirements) can specifically include the following characteristics:

[0075] Description: Provide a detailed description of the overall functionality of the API, including use cases and applicable scope.

[0076] Type: such as color, enumeration value, string, function, etc., specifying the type of the return value.

[0077] Hierarchy: Determines the hierarchy to which the current API belongs, i.e., which module's child node it is.

[0078] Dependencies: Indicate whether this API needs to work with other APIs. For example, enabling the footer feature may be necessary before adjusting the footer.

[0079] Enumerable items: List the enumerable values ​​supported by this API, such as font size (small four, large four, etc.) and position (top left, bottom right, etc.).

[0080] Expanded terms: List key instructions or keywords related to the current API, such as words related to "high," such as "highest," "largest," and "top."

[0081] For example, the following is a specific API declaration:

[0082] {"Level 1 API": "footnote","Level 1 Description": "Footnote, used for data generation","Level 1 Data Generation": "Data Generation","Level 2 API_name": "footnote_position","Level 2 Description": "Adjust the position of footnotes in the document","Level 2 Type": "Enumeration Value","Level 2 Dependency": ["Enable Footnotes"],"Level 2 Enumerable Items": [{"value":"Bottom Left","Extension": ["leftbottom","Left Side"]},{"value": "Bottom Right","Extension": ["rightbottom","Right Side"]},{"value": "Top","Extension": ["top","top","highest"]},{"value": "bottom","extension": ["bottom","lowest"]}],"Level 2 Extension": ["Footnote","Page Footnote", "Text Below"],...Other Content}.

[0083] Based on the characteristics in the API declaration information from the previous step, the API can be broken down into sub-APIs of different granularities. The breakdown process specifically includes:

[0084] Type Combination: Based on the API's type, hierarchy, dependencies, enumerables, and extended terms, combine it into multiple sub-APIs for more granular data processing and generation. For example, a complex API can be broken down into multiple sub-functional modules, each generating data independently.

[0085] Hierarchical analysis: Consider the hierarchical relationship of APIs, and separate the top-level APIs from their sub-APIs to ensure that the APIs at each level are fully covered.

[0086] This step of splitting the API allows for more precise operation and control, facilitating subsequent data processing.

[0087] For example, the declaration information of the APIs listed above can be broken down into the following three sub-APIs:

[0088] Sub-API1: {"Level 1 API":"footnote","Level 1 Description":"Footnote, used for data generation","Level 1 Data Generation":"Data Generation","Level 2 API_name":"footnote_position","Level 2 Description":"Adjust the position of footnotes in the document","Level 2 Type":"Enumeration Value","Level 2 Dependency":["Enable Footnotes"],"Level 2 Enumerable Item":{"value":"bottom left","Extension":{"leftbottm"},"Level 2 Extension":["Footnote"],...Other content}.

[0089] Sub-API2: {"Level 1 API":"footnote","Level 1 Description":"Footnote, used for data generation","Level 1 Data Generation":"Data Generation","Level 2 API_name":"footnote_position","Level 2 Description":"Adjust the position of footnotes in the document","Level 2 Type":"Enumeration Value","Level 2 Dependency":["Enable Footnotes"],"Level 2 Enumerable Item":{"Value":"Bottom Left","Extension":{"Left Side"},"Level 2 Extension":["Page Footnote"],...Other content}.

[0090] Sub-API3: {"Level 1 API":"footnote","Level 1 Description":"Footnote, used for data generation","Level 1 Data Generation":"Data Generation","Level 2 API_name":"footnote_position","Level 2 Description":"Adjust the position of footnotes in the document","Level 2 Type":"Enumeration Value","Level 2 Dependency":["Enable Footnotes"],"Level 2 Enumerable Item":{"Value":"Bottom Left","Extension Word":{"Left Side"},"Level 2 Extension Word":["Below"],...Other Content}.

[0091] In practical applications, when generating a dataset for a specific API, the splitting and subsequent processing can be performed only on that API. When generating datasets for multiple APIs, all APIs can be split, or only a subset of them can be split. The actual splitting method depends on the needs of training and generating the model, and is not limited here.

[0092] S102. Perform type statistics on multiple application programming sub-interfaces and determine the generation ratio based on the statistical results.

[0093] After breaking down the APIs into sub-APIs, type statistics can be performed on these sub-APIs to obtain statistical results. Based on these statistical results, the generation ratio can be determined.

[0094] After the sub-APIs are separated, the quantity distribution information of the sub-APIs at different granularities and types can be statistically analyzed. The statistical results can include the distribution of different types of sub-APIs, such as the total number of sub-APIs and the number of sub-APIs under a certain type.

[0095] After the APIs are split, the next step is to statistically analyze the APIs at different granularities and of different types to obtain their quantity distribution. This will clarify the data distribution and provide a basis for determining the subsequent data proportions.

[0096] Once the statistical information is obtained, the generation ratio can be directly specified based on that information.

[0097] In one specific embodiment of this application, determining the generation ratio based on statistical results includes:

[0098] Step 1: Output the statistical results in the visualization interface;

[0099] Step 2: Obtain the input configuration information;

[0100] Step 3: Use the configuration information to determine the generation ratio.

[0101] For ease of description, the three steps mentioned above will be explained in combination below.

[0102] The statistical results can be displayed in a visualization interface, either in a list format or in a tree topology.

[0103] Then, based on the statistical results and their actual needs, users can input configuration information. This configuration information can directly include the generation ratio or the number of generated records for each category, thereby calculating the generation ratio based on the generated records.

[0104] In one specific embodiment of this application, determining the generation ratio using configuration information includes:

[0105] Read the category ratios of each category from the configuration information, and determine the generation ratio based on the category ratios.

[0106] In other words, the generation ratio is determined directly based on the category ratios of each category. For example, if there are only three categories, A, B, and C, and their ratio is 1:2:1, then the generation ratio of A, B, and C can be directly set to 1:2:1.

[0107] S103. Obtain the prompt words corresponding to each of the multiple application programming sub-interfaces.

[0108] In order for the text language model to generate instruction result pairs for each sub-API, the prompt words corresponding to each sub-API can be obtained.

[0109] Specifically, developers can customize suggestion words based on the characteristics of APIs at different granularities. These suggestion words will be combined with the API type, level, dependencies, etc., to ensure that the generated data is more accurate. The design of suggestion words should be refined to specific API categories to avoid interference with other API categories.

[0110] For example, a specific prompt word is as follows:

[0111] Task: Users use natural language to control the fields, properties, events, and parameters of components.

[0112] Limitations: 1. The query must be an imperative statement of the user's actual needs, requiring richness and variety, and closely aligned with user habits; 2. Generate corresponding APIs for the current options and extended terms;

[0113] Note: Output in JSON format. No apology, explanation, or comments are required. To reiterate, no apology, explanation, or comments are required.

[0114] Result: The result is a JSON list containing two fields, query and api, for each element. The event parameters are generated based on the content of the corresponding fields. The query is the user's natural language description, and the api is the corresponding JSON format.

[0115] Example: Input is {example_input}, output is {example_output};

[0116] Current input is {d}, please output {k} results.

[0117] S104. Input the prompt words into the text language model for data generation and processing to obtain the instruction result pair.

[0118] The text language model has learned the technical documentation and usage guidelines of the application programming interface.

[0119] It should be noted that, in this embodiment, in order for the text language model to generate instruction result pairs based on prompt words, the text language model can be pre-trained to learn the API's technical documentation and usage guidelines. For details on how to enable the text language model to learn the API's technical documentation and usage guidelines, please refer to the specific training scheme for the text language model; these details will not be elaborated upon here.

[0120] The text language model can be specifically a generative model such as GPT (Generative Pretrained Transformer, a language model based on the Transformer architecture).

[0121] The following section uses a text language model as the GPT generation model to explain in detail the generation of instruction-result pairs:

[0122] Customized prompts are input into the GPT generative model, leveraging its powerful generation capabilities to produce a large number of structured API instruction-result pairs (i.e., instruction-result pairs). This step generates the necessary diverse data, providing a foundation for subsequent data processing and analysis.

[0123] The instruction result includes the input description instruction and the coded result of the description instruction. For example, if the user inputs the instruction "set the title color to red", the coded result is {title:{color:red}}.

[0124] For example, some examples of instruction result pairs generated by the GPT generative model are as follows:

[0125] "[{\"query\":\"Set the tab bar component to a line style\",\"api\":{\"atom_tabs\":{\"type\":\"line\"}}},{\"query\":\"I want the tab bar style to be line style\",\"api\":{\"atom_tabs\":{\"type\":\"line\"}}},{\"query\":\"Change the tab bar component to a line style\",\"api\":{\"atom_tabs\":{\"type\":\"line\"}}},{\"query\":\"Set the tab bar component to a line style { "query": "How to make the tab bar component display as a line?", "api": { "atom_tabs": { "type": "line"}}}, { "query": "Make the tab bar component use a line design?", "api": { "atom_tabs": { "type": "line"}}}, { "query": "Please change the tab bar to a line style?", "api": { "atom_tabs": { " type": "line"}}},{"query": "I need the tab bar component to display as a line", "api":{"atom_tabs":{"type": "line"}}},{"query": "Change the style of the tab bar component to line", "api":{"atom_tabs":{"type": "line"}}},{"query": "Switch the type of the tab bar component to line", "api":{"atom_tabs":{"type": "line"}}},{"query": ""I need the tab bar component to display as a line", "api":{"atom_tabs":{"type": "line"}}},{"query": ""Change the style of the tab bar component to ...Change the style of the tab bar component to line", "api":{"atom_tabs":{"type": "line"}}},{"query": "Change the style of the tab bar component to line", "api":{"atom_tabs":{"type": "line"}}},{"query": "Change the style of the tab bar component to line", "api":{"a y":\"Set the tab bar to block type\",\"api\":{\"atom_tabs\":{\"type\":\"block\"}}},{\"query\":\"I want the tab bar style to be block type\",\"api\":{\"atom_tabs\":{\"type\":\"block\"}}},{\"query\":\"I want to change the tab bar component to block type\",\"api\":{\"atom_tabs\":{\"type\":\"block\"}}},{\"query\":\"How to make the tab bar component display as block type\","api":{"atom_tabs":{"type":"block"}}},{"query":"Sets the tab bar component's style to block type","api":{"atom_tabs":{"type":"block"}}},{"query":"Changes the tab bar component's type to block type","api":{"atom_tabs":{"type":"block"}}},{"query":"Makes the tab bar component display as block type","api":{"atom_tabs":{"type":"block"}}} _tabs":{"type":"block"}}},{"query":"I want the tab bar component to be card type","api":{"atom_tabs":{"type":"card"}}},{"query":"Set the tab bar component to card type","api":{"atom_tabs":{"type":"card"}}},{"query":"How to make the tab bar component a card type","api":{"atom_tabs":{"type":"card"}}},{"query":"How to make the tab bar component a card type","api":{"atom_tabs":{"type":"card"}}} ard\"}}},{\"query\":\"Change the style of the tab bar component to card type\",\"api\":{\"atom_tabs\":{\"type\":\"card\"}}},{\"query\":\"I need the tab bar component to display as card type\",\"api\":{\"atom_tabs\":{\"type\":\"card\"}}},{\"query\":\"Please make the tab bar component adopt card type layout\",\"api\":{\"atom_tabs\":{\"type\":\"card\"}}},{\"query\"}} ":"Switch the tab bar component type to card type\","api\":{\"atom_tabs\":{\"type\":\"card\"}}},{\"query\":\"Set the tab bar component to text type\",\"api\":{\"atom_tabs\":{\"type\":\"text\"}}},{\"query\":\"I want the tab bar component to be text type\",\"api\":{\"atom_tabs\":{\"type\":\"text\"}}},{\"query\":\"How to make the tab bar component display as text type\","api":{"atom_tabs":{"type":"text"}}},{"query":"Change the style of the tab bar component to text type","api":{"atom_tabs":{"type":"text"}}},{"query":"I need the tab bar component to display as text type","api":{"atom_tabs":{"type":"text"}}},{"query":"Make the tab bar component use text type design","api":{"atom_tabs":{"type":"text"}}} {"query":"Please set the tab bar component to text type","api":{"atom_tabs":{"type":"text"}}},{"query":"I want the tab bar component to be text type","api":{"atom_tabs":{"type":"text"}}},{"query":"I switch the tab bar component to text type","api":{"atom_tabs":{"type":"text"}}}]",

[0126] S105. Store the instruction result pairs into the dataset, and perform statistics on the instruction result pairs in the dataset to obtain the distribution data.

[0127] In this embodiment, a dataset can be created in advance to store the generated instruction result pairs.

[0128] During the generation of instruction result pairs, or after all instruction result pairs corresponding to prompt words have been generated, the instruction result pairs in the dataset can be statistically analyzed to obtain the distribution data of instruction result pairs.

[0129] S106. If the distribution data matches the generation ratio, the dataset is determined as the target dataset.

[0130] If the distribution data and the generation ratio are clearly matched, it can be determined that the current dataset meets the requirements, and the current dataset can be identified as the target dataset.

[0131] Specifically, this distribution data can be calculated by statistically analyzing the number of instruction results for different categories within the dataset, and then determining the distribution ratio based on these numbers. If this distribution ratio matches the production ratio, the distribution data is considered to match the production ratio. Alternatively, the production ratio can be calculated against the minimum number of expanded records to determine the minimum total number of records for each type. The distribution data can then be specifically the total number of records for each category; if the total number of records is greater than or equal to the minimum total number of records, the distribution data is considered to match the production ratio.

[0132] Applying the method provided in the embodiments of this application, the application programming interface (API) is decomposed at different granularities to obtain multiple API sub-interfaces; type statistics are performed on the multiple API sub-interfaces, and the generation ratio is determined based on the statistical results; prompt words corresponding to each of the multiple API sub-interfaces are obtained; the prompt words are input into a text language model for data generation processing to obtain instruction-result pairs; wherein, the text language model has learned the technical documents and usage guidelines of the API; the instruction-result pairs are stored in a dataset, and the instruction-result pairs in the dataset are statistically analyzed to obtain distribution data; if the distribution data matches the generation ratio, the dataset is determined as the target dataset.

[0133] Application programming interfaces (APIs) are broken down into sub-APIs of varying granularity. These sub-APIs are then statistically analyzed to determine the generation ratio. To ensure a more comprehensive dataset containing instruction-result pairs related to the APIs, prompts corresponding to each sub-API are retrieved during prompt generation. Then, a textual language model based on the API's technical documentation and user guides is used for data generation to produce instruction-result pairs. These pairs are then added to the dataset, and statistical analysis is performed. Once the distribution of the data matches the production ratio, the dataset generation is considered complete, and the target dataset is obtained.

[0134] The technical advantages of this application are as follows: When generating instruction-result pairs in the dataset, by splitting the application programming interface (API), the instruction-result pairs can cover instructions of different granularities of the API. By statistically analyzing the API sub-interfaces, the generation ratio can be determined. After obtaining the distribution data of instruction-result pairs in the dataset, when the generation ratio matches the distribution data, the target dataset can be determined. This makes the data distribution of the final generated target dataset controllable, providing comprehensive and customizable training samples for further training of generative models.

[0135] It should be noted that, based on the above embodiments, the embodiments of this application also provide corresponding improvement schemes. In the preferred / improved embodiments, the same or corresponding steps as in the above embodiments can be referred to each other, and the corresponding beneficial effects can also be referred to each other; however, these will not be elaborated upon in the preferred / improved embodiments herein.

[0136] In one specific embodiment of this application, when the distribution data and the generation ratio do not match, the method further includes:

[0137] The dataset is supplemented by acquiring new prompt words to ensure that the distribution data of the supplemented dataset matches the generation ratio.

[0138] In other words, when it is found that the distribution of data does not match the generation ratio, it means that the data distribution in the current dataset does not meet the required generation ratio. At this time, the data result pair generation process can continue, and the dataset can be supplemented so that the data distribution of the supplemented dataset matches the production ratio.

[0139] Specifically, the dataset can be supplemented by acquiring new prompt words and then supplementing the dataset based on the new prompt words.

[0140] In one specific embodiment of this application, the dataset is supplemented by re-acquiring new prompt words, including:

[0141] Step 1: Compare the distribution data with the generation ratio to determine the target type of the missing instruction result pairs;

[0142] Step 2: Identify the application programming sub-interface that matches the target type as the target application programming sub-interface;

[0143] Step 3: Retrieve the prompt words corresponding to the target application's programming sub-interface;

[0144] Step 4: Input the newly acquired prompt words into the text language model for data generation processing to obtain supplementary instruction result pairs;

[0145] Step 5: Write the supplementary instruction results into the dataset.

[0146] For ease of description, the above five steps will be explained in combination below.

[0147] In this embodiment, the target type of the missing instruction result pairs can be determined by comparing the distribution data with the generation ratio. For example, assuming there are three types: A, B, and C, and the current data distribution corresponds to a ratio of 1:2:3, while the generation ratio is 2:2:3, it is clear that instruction result pairs of type A are missing. Therefore, type A can be identified as the target application sub-interface. Then, the prompt words for type A are re-acquired, and instruction result pairs corresponding to these prompt words are generated using a text language model. For clarity, in this embodiment, the instruction result pairs generated based on the re-acquired prompt words are referred to as supplementary instruction result pairs. The supplementary instruction result pairs are then written into the dataset, thereby supplementing the dataset.

[0148] After supplementing the dataset, it can be determined again whether the distribution of the current dataset is consistent with the generation ratio. If they are consistent, the target dataset can be obtained; if they are inconsistent, the data in the dataset can be supplemented again.

[0149] It is important to note that once the number of records in the dataset meets the minimum expansion factor, if the proportion of distributed data does not match the generation proportion, the proportion of distributed data in the dataset can be made to match the generation proportion by deleting redundant data result pairs.

[0150] In one specific embodiment of this application, determining whether the distribution data matches the generation ratio includes:

[0151] Step 1: Obtain the subclass ratio of each subclass under each category using the category ratio;

[0152] Step 2: Read the minimum number of extended records for each subclass from the configuration information;

[0153] Step 3: Determine the minimum number of records corresponding to each subcategory by using the subcategory allocation and the minimum number of extended records;

[0154] Step 4: Calculate the minimum total number of records that satisfy the category matching based on the minimum number of records;

[0155] Step 5: Determine if the minimum number of records, minimum total number of records, and distribution data match;

[0156] Step 6: If so, determine if the distribution data matches the generation ratio;

[0157] Step 7: If not, then determine that the distribution data does not match the generation ratio.

[0158] For ease of description, the above steps will be combined below.

[0159] For a category set A, which includes multiple subcategories a, b, c, d, if the category ratio is P, the ratio of each subcategory is Pa, Pb, Pc, Pd.

[0160] The minimum number of extended records required for each sub-API of each sub-category is lima, limb, limc, limd. That is, the minimum number of instruction-result pairs the model needs to generate for each sub-API of the current sub-category.

[0161] The total number of sub-APIs is N, and the number of sub-APIs in each sub-category is Na, Nb, Nc, Nd.

[0162] The matching can be determined by performing the following steps:

[0163] Step 1: Calculate the number of records that meet the minimum number requirement.

[0164] For each subcategory i in category set A, calculate the minimum number of records Mi required to satisfy each subcategory, multiplying the minimum number of expanded records limi for each sub-API of the current subcategory i by the number of sub-APIs Ni of the current subcategory i. .

[0165] Step 2: Calculate the minimum total number of records that satisfy the matching requirement P. Calculate the minimum total number of records S for each subcategory i while satisfying its own matching requirement. .

[0166] Step 3: Calculate the final number of records required for each subcategory.

[0167] For each category Calculate the final number of records Si required for subcategory i, i.e. .

[0168] After obtaining the final number of records required for each subcategory, and given the distribution data of the dataset, we can determine whether the current statistical count for each subcategory matches the final number of records. If they match, it means the distribution data matches the generation ratio. If the count exceeds the limit, records can be deleted; if there is a discrepancy, data can be supplemented by adding new subcategories later.

[0169] For example: If there is an attribute set (i.e., a category set, where in this example, the attribute corresponds to the category) And the number of records for each attribute in the original dataset: ;

[0170] Proportioning requirements ;

[0171] The minimum number of extended records for each sub-API is ;

[0172] Step 1: Calculate the number of records that satisfy the minimum expansion requirements for each subcategory a, b, c, d.

[0173] , , , .

[0174] Step 2: Calculate the minimum total number of records S required to meet the mixing ratio requirements: .

[0175] Step 3: Calculate the final number of records required for each subcategory:

[0176] , , , .

[0177] Final result:

[0178] Attribute a requires 1800 records, attribute b requires 1800 records, attribute c requires 1200 records, and attribute d requires 1200 records.

[0179] Verification: The final number of records for each attribute is at least a specified multiple of the original number of records: a: 1800 > 2 * 100, b: 1800 > 3 * 200, c: 1200 = 4 * 300, d: 1200 > 3 * 400.

[0180] Final proportions: a: 1800 / 6000 = 0.3, b: 1800 / 6000 = 0.3, c: 1200 / 6000 = 0.2, d: 1200 / 6000 = 0.2.

[0181] Verification has shown that the method for determining the final number of records for each attribute ensures that each attribute reaches at least its specified minimum expansion multiple, while also satisfying the given matching requirements.

[0182] In practical applications, the generated instruction results can be rigorously filtered to remove erroneous or non-compliant data. This ensures that the final output data is of high quality and consistent.

[0183] After the data is generated, the data proportions for each API can be recalculated and compared with the previously set matching requirements. APIs with insufficient numbers can be filtered out for further supplementary processing.

[0184] For APIs with insufficient sample size, customized prompts can be used to guide the model to generate new data that differs from the existing data, using existing API results as input. This helps to further enrich the diversity of the data and avoid generating similar or duplicate data.

[0185] Furthermore, it can continuously generate and optimize data until all API quantity and diversity requirements are met. This iterative process ensures that the final dataset is of high quality, diversity, and coverage.

[0186] To facilitate those skilled in the art to better understand and implement the data generation method provided in the embodiments of this application, the data generation method will be described in detail below with reference to relevant technical solutions and specific application scenarios.

[0187] API-oriented data composition methods can be mainly divided into the following three categories:

[0188] 1. Manual data construction: Data is created manually or obtained by scraping historical data.

[0189] 2. Automated Generation: By using automated methods, strategies are set for each API to automatically generate possible API results, and then the corresponding input instructions are annotated in reverse.

[0190] 3. API documentation-based generation: By constructing API documentation and inputting each API description, possible input and output cases are generated using existing open-source large models, thereby constructing training data.

[0191] Manually constructing data or generating data by scraping historical data has the following drawbacks:

[0192] (1) High verification cost: Each piece of data needs to be manually verified to ensure its correctness, which is time-consuming and labor-intensive.

[0193] (2) Low data generation efficiency: It relies on a lot of manual operation, resulting in low data generation efficiency and difficulty in meeting the needs of large-scale data.

[0194] (3) Difficulty in controlling data distribution and quantity: In terms of data scale and diversity, it is difficult to carry out fine control through manual means, which can easily lead to data deviation or insufficiency.

[0195] For automated data generation, a program is written to set strategies for each API, automatically enumerate and generate the API's output results, and then use these output results to annotate the input commands. However, this approach has drawbacks.

[0196] (1) High development cost: It depends on the detailed generation strategy of each API, such as color, font, preset value, etc., which requires a lot of development time and resources.

[0197] (2) Poor scalability: When adding new APIs, data types or return values, the generation strategy needs to be redesigned and implemented, resulting in high maintenance costs.

[0198] (3) Limited flexibility: Automatically generated data is often constrained by rules and cannot cover all possible use cases of the API.

[0199] For automatic generation based on API documentation and large models, sample input and output data for the API are generated using open-source large models by providing API documentation. However, this approach has drawbacks:

[0200] (1) Low instruction coverage: The input and output cases generated by large models often cannot fully cover all the use cases of the API, and some important use cases are easily missed.

[0201] (2) Uneven data distribution: The generated data may have a bias in distribution, making it impossible to achieve full data coverage.

[0202] (3) Insufficient handling of extreme values: There is insufficient consideration for boundary values ​​and abnormal situations, making it difficult to ensure the comprehensiveness of the data.

[0203] (4) Poor generalization: The generated data instructions lack flexibility and are difficult to adapt to a wide range of applications in various API scenarios.

[0204] As can be seen from the above embodiments, the technical solution provided in this application aims to generate dedicated business data for training large models by introducing features to define APIs and automatically splitting multiple small sub-APIs based on these features. This method not only helps control the distribution ratio of data but also significantly increases the diversity and coverage of generation instructions. It enables the automated, batch-controllable generation of training data for multiple business parties using a single, universal solution.

[0205] Specifically, this application defines a standard API specification, including types, hierarchy, dependencies, enumerables, and a vocabulary. This not only provides a clear framework but also lays the foundation for subsequent automated splitting and data generation.

[0206] Based on the defined characteristics, the API is broken down into different granularities, and sub-functional modules are further refined and restructured. This process allows for a more flexible API processing approach.

[0207] The distribution of APIs of different types and granularities is statistically analyzed, and the data generation ratio is calculated accordingly to control the balance and quality of the dataset.

[0208] A distributed, controllable, general solution framework for APIs: By defining API features and automating their splitting, this approach enables customized and diversified generation of training data for large models. This method allows for the automatic generation of business data with specific parameters and configurations for different business scenarios using a unified framework, while ensuring data balance and coverage, thus improving the efficiency and quality of data generation.

[0209] Based on the target dataset generated in this application, the following results can be achieved after training the generative model:

[0210] Users need to specify the corresponding component (i.e., API name) and set various parameters of the component (API parameters) through natural language to achieve quick and convenient use and increase creation efficiency.

[0211] In front-end business development scenarios, for product managers, it allows for quick page building and comparison of different solutions: for example, I need the tab bar component to be displayed as text, and I want to make the third tab in the tab bar unavailable.

[0212] In component adjustment scenarios, user-facing adjustments can quickly specify effects without having to search through numerous APIs for the required adjustments. Users need to set various parameters of the component using natural language, such as adjusting the font size to four, enabling footnotes, and using seasonal charts.

[0213] In the configuration of market charts in the financial sector, product managers and designers can quickly compare the differences between different solutions and quickly build styles and functions for different markets: for example, the outer frame of the candlestick chart is dark red, and the crosshair cursor is blue.

[0214] Please refer to Figure 2 The specific implementation steps of this application include:

[0215] 1. Establishing a Standard API Specification: First, a standard API specification was defined, including types, hierarchy, dependencies, enumerable items, and an extensible vocabulary. This step provides a clear framework and basis for subsequent splitting and data generation.

[0216] 2. API Decomposition and Restructuring: Based on defined characteristics, the API is decomposed into different levels of granularity. This means not only considering the overall functionality of the API but also further refining and restructuring its sub-modules. This facilitates more flexible API handling and lays the foundation for subsequent data generation.

[0217] 3. Statistical Distribution: Perform statistical analysis on the quantity and distribution of APIs at different granularities and of different types to create a comprehensive data distribution map. The purpose of this step is to ensure a clear understanding of the quantity and distribution of each type of API.

[0218] 4. Calculate Data Proportions: Based on the distribution data above, calculate the required proportions for each API during data generation. This step helps to more precisely control the amount of data generated, ensuring the balance of the dataset.

[0219] 5. Customized prompt word construction: Design customized prompt words for APIs at different granularities. These prompt words will be combined with specific quantity requirements to ensure that the generated data can achieve the expected results.

[0220] 6. Data Generation: Using the GPT generative model, data is generated based on the designed prompts. This step leverages the powerful capabilities of generative AI to generate diverse data on a large scale.

[0221] 7. Data Processing and Error Removal: Post-processing is performed on the generated data to filter and remove erroneous data. This step ensures the quality and accuracy of the final data.

[0222] 8. Data Proportion Calculation and API Filtering: Recalculate the data proportions and filter out APIs with insufficient quantity. The purpose of this step is to ensure that each API reaches the required quantity and coverage.

[0223] 9. Continued Data Generation: For the APIs selected in step 8, the customized prompts built in step 5 are combined with the existing API generation results to guide the model in generating new data that differs from the existing data. This step helps to improve the data mix and ensure the diversity of rich data, avoiding duplication and homogenization.

[0224] 10. Iterative optimization: Repeat steps 6 to 9 to continuously optimize and generate data until the quantity and diversity requirements of all APIs are met.

[0225] Corresponding to the above method embodiments, this application also provides a data generation apparatus, and the data generation apparatus described below can be referred to in correspondence with the data generation method described above.

[0226] See Figure 3 As shown, the device includes the following modules:

[0227] The splitting module 101 is used to split the application programming interface at different granularities to obtain multiple application programming sub-interfaces;

[0228] The generation ratio statistics module 102 is used to perform type statistics on multiple application programming sub-interfaces and determine the generation ratio based on the statistical results.

[0229] The prompt processing module 103 is used to obtain the prompt words corresponding to multiple application programming sub-interfaces respectively;

[0230] The instruction result pair generation module 104 is used to input prompt words into the text language model for data generation processing to obtain instruction result pairs; wherein, the text language model has learned the technical documentation and usage guidelines of the application programming interface;

[0231] The analysis module 105 is used to store instruction result pairs into a dataset and perform statistics on the instruction result pairs in the dataset to obtain distribution data.

[0232] The data generation module 106 is used to determine the dataset as the target dataset when the distribution data matches the generation ratio.

[0233] Using the apparatus provided in this application embodiment, the application programming interface (API) is decomposed at different granularities to obtain multiple API sub-interfaces; type statistics are performed on the multiple API sub-interfaces, and the generation ratio is determined based on the statistical results; prompt words corresponding to each of the multiple API sub-interfaces are obtained; the prompt words are input into a text language model for data generation processing to obtain instruction-result pairs; wherein, the text language model has learned the technical documents and usage guidelines of the API; the instruction-result pairs are stored in a dataset, and the instruction-result pairs in the dataset are statistically analyzed to obtain distribution data; if the distribution data matches the generation ratio, the dataset is determined as the target dataset.

[0234] Application programming interfaces (APIs) are broken down into sub-APIs of varying granularity. These sub-APIs are then statistically analyzed to determine the generation ratio. To ensure a more comprehensive dataset containing instruction-result pairs related to the APIs, prompts corresponding to each sub-API are retrieved during prompt generation. Then, a textual language model based on the API's technical documentation and user guides is used for data generation to produce instruction-result pairs. These pairs are then added to the dataset, and statistical analysis is performed. Once the distribution of the data matches the production ratio, the dataset generation is considered complete, and the target dataset is obtained.

[0235] The technical advantages of this application are as follows: When generating instruction-result pairs in the dataset, by splitting the application programming interface (API), the instruction-result pairs can cover instructions of different granularities of the API. By statistically analyzing the API sub-interfaces, the generation ratio can be determined. After obtaining the distribution data of instruction-result pairs in the dataset, when the generation ratio matches the distribution data, the target dataset can be determined. This makes the data distribution of the final generated target dataset controllable, providing comprehensive and customizable training samples for further training of generative models.

[0236] In one specific embodiment of this application, the module is split up, specifically for obtaining the declaration information of the application programming interface;

[0237] Different granularities are determined based on the declared information;

[0238] By breaking down the program programming interface at different granularities, multiple application programming sub-interfaces are obtained.

[0239] In one specific embodiment of this application, it further includes:

[0240] The supplementary module is used to supplement the dataset by acquiring new prompt words when the distribution data and the generation ratio do not match, so that the distribution data of the supplemented dataset matches the generation ratio.

[0241] In one specific embodiment of this application, the supplementary module is specifically used to compare the distribution data with the generation ratio to determine the target type of the missing instruction result pair;

[0242] The application programming sub-interface that matches the target type is identified as the target application programming sub-interface;

[0243] Retrieve the prompt word corresponding to the target application's programming sub-interface;

[0244] The newly acquired prompt words are input into the text language model for data generation and processing to obtain supplementary instruction result pairs;

[0245] Write the supplementary instruction results into the dataset.

[0246] In one specific embodiment of this application, a proportional statistics generation module is used to output statistical results on a visual interface;

[0247] Obtain the input configuration information;

[0248] Use the configuration information to determine the generation ratio.

[0249] In one specific embodiment of this application, the generation ratio statistics module is specifically used to read the category ratios of various categories from the configuration information and determine the generation ratio based on the category ratios.

[0250] In one specific embodiment of this application, the matching judgment module is used to obtain the subclass matching ratio of subclasses under each category using the category matching ratio;

[0251] Read the minimum number of extended records for each subclass from the configuration information;

[0252] Determine the minimum number of records corresponding to each subcategory by using the subcategory allocation and the minimum number of extended records;

[0253] Calculate the minimum total number of records that satisfy the category matching based on the minimum number of records;

[0254] Determine if the minimum number of records, the minimum total number of records, and the distribution data match.

[0255] If so, then determine that the distribution data matches the generation ratio;

[0256] If not, then it is determined that the distribution data does not match the generation ratio.

[0257] Corresponding to the above method embodiments, this application also provides an electronic device. The electronic device described below and the data generation method described above can be referred to in correspondence.

[0258] See Figure 4 As shown, the electronic device includes:

[0259] Memory 332 is used to store computer programs;

[0260] The processor 322 is used to implement the steps of the data generation method in the above method embodiments when executing a computer program.

[0261] For details, please refer to Figure 5 , Figure 5This is a schematic diagram of the specific structure of an electronic device provided in this embodiment. The electronic device can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) (e.g., one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. The memory 332 can be temporary or permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 and execute the series of instruction operations stored in the memory 332 on the electronic device 301.

[0262] Electronic device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0263] The steps in the data generation method described above can be implemented by the structure of an electronic device.

[0264] Corresponding to the above method embodiments, this application also provides a readable storage medium. The readable storage medium described below can be referred to in conjunction with the data generation method described above.

[0265] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data generation method described in the above method embodiments.

[0266] The readable storage medium can specifically be a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or any other readable storage medium capable of storing program code.

[0267] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0268] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0269] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0270] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0271] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data generation method, characterized in that, include: The application programming interface (API) is decomposed at different granularities to obtain multiple API sub-interfaces; Type statistics are performed on multiple application programming sub-interfaces, and the generation ratio is determined based on the statistical results; Obtain the prompt words corresponding to each of the multiple application programming sub-interfaces; The prompt words are input into a text language model for data generation processing to obtain instruction result pairs; wherein, the text language model has learned the technical documentation and usage guidelines of the application programming interface; The instruction result pairs are stored in the dataset, and the instruction result pairs in the dataset are statistically analyzed to obtain distribution data; If the distribution data matches the generation ratio, the dataset is determined as the target dataset; Among these, determining the generation ratio based on statistical results includes: The statistical results are output in the visualization interface; Obtain the input configuration information; Determining the generation ratio using the configuration information includes: reading the category ratios of each category from the configuration information, and determining the generation ratio based on the category ratios; The determination of whether the distribution data matches the generation ratio includes: The subclass ratios of each category are obtained using the aforementioned category ratios; Read the minimum number of extended records for each subclass from the configuration information; Using the subclass ratio and the minimum number of extended records, determine the minimum number of records corresponding to the subclass; Calculate the minimum total number of records that satisfy the category matching based on the minimum number of records; Determine whether the minimum number of records, the minimum total number of records, and the distribution data match. If so, then the distribution data is determined to match the generation ratio; If not, then it is determined that the distribution data does not match the generation ratio.

2. The method according to claim 1, characterized in that, By breaking down the application programming interface (API) at different granularities, multiple API sub-interfaces are obtained, including: Obtain the declaration information of the application programming interface; Based on the aforementioned declaration information, different granularities are determined; The application programming interface is split at different granularities to obtain multiple application programming sub-interfaces.

3. The method according to claim 1, characterized in that, In the case where the distribution data does not match the generation ratio, the method further includes: The dataset is supplemented by acquiring new prompt words so that the distribution data of the supplemented dataset matches the generation ratio.

4. The method according to claim 3, characterized in that, The dataset is supplemented by re-acquiring new prompt words, including: By comparing the distribution data with the generation ratio, the target type of the missing instruction result pair is determined; The application programming sub-interface that matches the target type is identified as the target application programming sub-interface; Retrieve the prompt word corresponding to the target application programming sub-interface; The newly acquired prompt words are input into the text language model for data generation and processing to obtain supplementary instruction result pairs; The supplementary instruction results are written into the dataset.

5. A data generation apparatus, characterized in that, include: The splitting module is used to decompose the application programming interface at different granularities, resulting in multiple application programming sub-interfaces; A generation ratio statistics module is used to perform type statistics on multiple application programming sub-interfaces and determine the generation ratio based on the statistical results; wherein, determining the generation ratio based on the statistical results includes: outputting the statistical results on a visual interface; obtaining input configuration information; and determining the generation ratio using the configuration information; wherein, determining the generation ratio using the configuration information includes: reading the category ratios of each category from the configuration information and determining the generation ratio based on the category ratios; The prompt processing module is used to obtain the prompt words corresponding to the multiple application programming sub-interfaces respectively; The instruction result pair generation module is used to input the prompt words into a text language model for data generation processing to obtain instruction result pairs; wherein, the text language model has learned the technical documentation and usage guidelines of the application programming interface; The analysis module is used to store the instruction result pairs into a dataset and perform statistics on the instruction result pairs in the dataset to obtain distribution data; A data generation module is used to determine the dataset as the target dataset when the distributed data matches the generation ratio; The step of determining whether the distribution data matches the generation ratio includes: obtaining the subclass ratio of subclasses under each category using the category ratio; reading the minimum number of extended records for each subclass from the configuration information; determining the minimum number of records corresponding to each subclass using the subclass ratio and the minimum number of extended records; calculating the minimum total number of records that satisfies the category ratio based on the minimum number of records; determining whether the minimum number of records, the minimum total number of records, and the distribution data match; if yes, determining that the distribution data matches the generation ratio; if no, determining that the distribution data does not match the generation ratio.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data generation method as described in any one of claims 1 to 4.

7. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data generation method as described in any one of claims 1 to 4.

Images