Test data generation method and device

Abstract

The invention discloses a test data generation method and device, and the method comprises the steps: carrying out the vectorization processing of a test target description, and obtaining a vector representation; calculating the similarity between the vector representation and each rule in the test data rule base; screening the similarity greater than a preset similarity from all the similarities, and marking the rule corresponding to the similarity as a target rule; generating a rule description set based on the target rule and the test target description; generating a basic test data set based on the rule description set and the test target description; performing boundary condition analysis on the rule description set to obtain a parameter boundary set; generating boundary test data based on the parameter boundary set; and determining a final test data set based on the basic test data set and the boundary test data. By accurately matching the rule and the test target and combining boundary analysis, manual intervention is reduced, and the automation level of test data generation is improved, so that the generation efficiency of the test data is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for generating test data. Background Technology

[0002] With the rapid development of artificial intelligence technology, large-scale pre-trained language models have become a core technology in the field of natural language processing. However, in practical applications, especially in industries with high compliance requirements such as finance, obtaining test data faces many challenges.

[0003] Traditional test data generation methods rely on rule-based approaches that use manually defined rule sets. Since these rule sets need to be manually updated to meet constantly changing business needs, each update requires manual intervention. Furthermore, maintaining these rule sets is a significant undertaking, especially given the evolving business scenarios and compliance requirements. The manual maintenance of rule sets demands substantial time and resources, thus reducing the efficiency of test data generation.

[0004] Therefore, improving the efficiency of test data generation has become an urgent problem to be solved in this field. Summary of the Invention

[0005] This application provides a test data generation method and apparatus, with the aim of improving the efficiency of test data generation.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] A method for generating test data, comprising:

[0008] The description of the test target is vectorized to obtain a vector representation;

[0009] Calculate the similarity between the vector representation and each rule in the test data rule base;

[0010] Filter out similarities greater than a preset similarity from all similarities, and mark the rules corresponding to the similarities as target rules;

[0011] Based on the target rules and the test target description, a set of rule descriptions is generated;

[0012] Based on the rule description set and the test target description, a basic test data set is generated;

[0013] Boundary condition analysis is performed on the rule description set to obtain the parameter boundary set;

[0014] Based on the aforementioned parameter boundary set, boundary test data is generated;

[0015] Based on the basic test data set and the boundary test data, the final test data set is determined.

[0016] Optionally, generating a rule description set based on the target rule and the test target description includes:

[0017] Each rule description in the rule description set is structured to obtain a set of constraints;

[0018] A test point matrix is ​​generated based on the set of constraints.

[0019] Based on the test point matrix and the test target description, an initial test data set is generated;

[0020] Based on the initial test data set and the constraint set, a coverage metric is generated;

[0021] A basic test data set is generated based on the coverage metric and the initial test data.

[0022] Optionally, generating a basic test data set based on the coverage metric and the initial test data includes:

[0023] When the diversity index of the initial test data is not less than the diversity threshold and the coverage index is not less than the coverage threshold, the initial test data set is determined as the basic test data set.

[0024] When the diversity index of the initial test data is less than the diversity threshold, and / or the coverage index is less than the coverage threshold, the initial test data set is subjected to iterative processing.

[0025] When the number of iterations is the maximum number of iterations, the initial test data set after iteration is determined as the basic test data set.

[0026] Optionally, after generating the basic test data set based on the rule description set and the test target description, the process further includes:

[0027] Calculate the overall compliance rate of the aforementioned basic test dataset;

[0028] A basic compliance report is generated based on the overall compliance rate.

[0029] Optionally, after generating boundary test data based on the parameter boundary set, the method further includes:

[0030] Calculate the boundary coverage index of the boundary test data;

[0031] A boundary compliance report is generated based on the boundary coverage metric.

[0032] Optionally, determining the final test data set based on the basic test data set and the boundary test data includes:

[0033] Merge the basic test dataset and the boundary test dataset to obtain the integrated test dataset.

[0034] Calculate the overall quality score of the integrated test data set;

[0035] When the overall quality score is less than the quality threshold, a state vector is obtained;

[0036] Generate supplementary data requests based on the state vector;

[0037] Based on the aforementioned supplementary data, request the generation of supplementary data;

[0038] After merging the supplementary data and the integrated test data set to obtain the optimized test dataset, and determining the optimized test dataset as the integrated test data set, the process returns to the step of calculating the comprehensive quality score of the integrated test data set.

[0039] When the overall quality score is not less than the quality threshold, the optimized test dataset is determined as the final test data set.

[0040] Optional, also includes:

[0041] Filter out a set of candidate rules from the rule base that correspond to the final test data set and the test target description;

[0042] Calculate the rule vector representation of each candidate rule in the candidate rule set;

[0043] For each rule vector representation, calculate the similarity between the rule vector representation and each rule in the test data rule base;

[0044] Filter out similarities below a preset threshold from all similarities and mark them as target similarities;

[0045] The candidate rules corresponding to the target similarity are added to the test data rule base.

[0046] A test data generation device, comprising:

[0047] The processing unit is used to vectorize the description of the test target to obtain a vector representation.

[0048] A calculation unit is used to calculate the similarity between the vector representation and each rule in the test data rule base;

[0049] The filtering unit is used to filter out similarities greater than a preset similarity from all similarities, and to identify the rule corresponding to the similarity as the target rule;

[0050] The first generation unit is used to generate a set of rule descriptions based on the target rule and the test target description;

[0051] The second generation unit is used to generate a basic test data set based on the rule description set and the test target description;

[0052] The analysis unit is used to perform boundary condition analysis on the rule description set to obtain the parameter boundary set;

[0053] The third generation unit is used to generate boundary test data based on the parameter boundary set;

[0054] The determining unit is used to determine the final test data set based on the basic test data set and the boundary test data.

[0055] Optionally, the first generation unit includes:

[0056] The data processing unit is used to perform structured processing on each rule description in the rule description set to obtain a set of constraints.

[0057] The fourth generation unit is used to generate a test point matrix based on the set of constraints.

[0058] The fifth generation unit is used to generate an initial test data set based on the test point matrix and the test target description;

[0059] The sixth generation unit is used to generate a coverage index based on the initial test data set and the constraint set;

[0060] The sixth generation unit is used to generate a basic test data set based on the coverage index and the initial test data.

[0061] Optionally, the sixth generation unit is specifically used for:

[0062] When the diversity index of the initial test data is not less than the diversity threshold and the coverage index is not less than the coverage threshold, the initial test data set is determined as the basic test data set.

[0063] When the diversity index of the initial test data is less than the diversity threshold, and / or the coverage index is less than the coverage threshold, the initial test data set is subjected to iterative processing.

[0064] When the number of iterations is the maximum number of iterations, the initial test data set after iteration is determined as the basic test data set.

[0065] The technical solution provided in this application vectorizes the test target description to obtain a vector representation; calculates the similarity between the vector representation and each rule in the test data rule base; selects similarities greater than a preset similarity from all similarities and identifies the rules corresponding to these similarities as target rules; generates a set of rule descriptions based on the target rules and the test target description; generates a basic test data set based on the set of rule descriptions and the test target description; performs boundary condition analysis on the set of rule descriptions to obtain a set of parameter boundaries; generates boundary test data based on the set of parameter boundaries; and determines the final test data set based on the basic test data set and the boundary test data. By accurately matching rules and test targets, combined with boundary analysis, manual intervention is reduced and the automation level of test data generation is improved, thereby increasing the efficiency of test data generation. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 A flowchart illustrating a test data generation method provided in this application embodiment;

[0068] Figure 2 A flowchart illustrating a method for generating a basic test dataset as provided in this application embodiment;

[0069] Figure 3 A flowchart illustrating a method for determining a final test data set provided in an embodiment of this application;

[0070] Figure 4 A flowchart illustrating a rule generation method provided in an embodiment of this application;

[0071] Figure 5 This is a schematic diagram of a test data synthesis system provided in an embodiment of this application;

[0072] Figure 6 This is a schematic diagram of the architecture of a test data generation device provided in an embodiment of this application. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0074] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] like Figure 1 The diagram shown is a flowchart of a test data generation method provided in an embodiment of this application, including the following steps:

[0076] S101: Vectorize the description of the test target to obtain a vector representation.

[0077] The test target description is a textual description of the test target.

[0078] Understandably, converting the textual description of the test target into a mathematical vector can be used for various calculations, comparisons, classifications, and other tasks.

[0079] S102: Calculate the similarity between the vector representation and each rule in the test data rule base.

[0080] Specifically, the cosine similarity between the vector representation and the rule vector corresponding to each rule in the test data rule base can be calculated, as shown in formula (1).

[0081] (1)

[0082] In formula (1), Let be a vector representing the cosine similarity to the rule. Represented as a vector, Let be the vector representation of the i-th rule.

[0083] It should be noted that the test data rule base records rules for different types of data and establishes a vector retrieval library. (Rule text) After vectorization, the rule vector is obtained: .in The vector dimension is typically 768 or 1536. Following the above method, in the subsequent process, keywords will be extracted from the text of the test data requirements to construct a vector library for data rule retrieval, denoted as the test data rule library K.

[0084] S103: Filter out the similarities that are greater than the preset similarity from all similarities, and mark the rules corresponding to the similarities as the target rules.

[0085] Optionally, the preset similarity can be set according to the actual situation, and no specific restrictions are imposed here.

[0086] Specifically, the similarity scores are selected from all similarity scores that are greater than a preset similarity score. This means obtaining the top-k rules from all cosine similarity scores. The specific format is as follows: ,in, The vector representation of the test target. Let be the vector representation of the i-th rule in the rule base. The function returns the k rules with the highest similarity, where S is the target rule.

[0087] S104: Generate a set of rule descriptions based on the target rules and test target descriptions.

[0088] Specifically, based on the target rules and test target descriptions, a set of rule descriptions is generated. Specifically, the target rules and test target descriptions are input into the rule parsing agent to obtain the set of rule descriptions, which is represented in the following form: , Let T represent the set of rules generated by the rule-parsing agent after processing the input, and let T be the description of the test target. R represents the number of rules generated, and R is the set of rule descriptions.

[0089] S105: Generate a basic test data set based on the rule description set and the test target description.

[0090] In step S105, the specific implementation process of generating the basic test data set based on the rule description set and the test target description is as follows: generating the initial test data set based on the rule description set and the test target description; generating the basic test data set based on the initial test data set.

[0091] Optionally, in another embodiment of this application, the specific implementation of step S104 is as follows: Figure 2 As shown, it includes the following steps:

[0092] S201: Perform structured processing on each rule description in the rule description set to obtain the constraint condition set.

[0093] The specific form of structuring each rule description in the rule description set is as follows: , , For rule description The corresponding set of constraints, For the first Specific constraints.

[0094] S202: Generate a test point matrix based on the set of constraints.

[0095] In step S202, the specific implementation process of generating the test point matrix based on the constraint set is as follows: the constraint set is input into the rule parsing agent to generate the test point matrix, which is specifically represented as follows: Where P is the test point matrix, Representing constraints The priority is inherited from the corresponding rule description. Importance weights.

[0096] S203: Generate an initial test data set based on the test point matrix and test target description.

[0097] The specific form of generating the initial test data set based on the test point matrix and test target description is as follows: D is the initial test data set, d m This is the m-th initial test data.

[0098] S204: Generate coverage metrics based on the initial test data set and constraint set.

[0099] Understandably, in order to ensure that the generated test data can meet all the constraints in the test point matrix, it is necessary to calculate the coverage index. Specifically, based on the initial test data set and the constraint set, the coverage index is generated in the form shown in formula (2).

[0100] (2)

[0101] In formula (2), Coverage(D,P) is the coverage rate indicator. Represents test data Do the constraints meet? .

[0102] S205: Generate a basic test data set based on the coverage metric and initial test data.

[0103] In step S205, the diversity index of the initial test data is first defined, and the basic test data is determined based on the diversity index and the initial test data.

[0104] The specific manifestation of the diversity index of the initial test data is shown in formula (3).

[0105] (3)

[0106] In formula (3), The embedding representation of the two test data can be obtained through semantic vector model encoding. Let be the vector similarity function, and Diversity(D) be the diversity index of the initial test data.

[0107] Optionally, in another embodiment of this application, the specific implementation of step S205 includes processes A1 to A3.

[0108] A1: When the diversity index of the initial test data is not less than the diversity threshold and the coverage index is not less than the coverage threshold, the initial test data set is determined as the basic test data set.

[0109] A2: When the diversity index of the initial test data is less than the diversity threshold and / or the coverage index is less than the coverage threshold, the initial test data set is iteratively processed.

[0110] The specific manifestation of iterative processing of the initial test data set is shown in formula (4).

[0111] (4)

[0112] In formula (4), D (t+1) The initial test data set generated in the (t+1)th iteration. Indicates the The initial test data set generated in the next iteration The function takes the diversity of the current dataset as additional input when generating a new dataset, guiding the model to generate more diverse samples.

[0113] In addition, a coverage report will also be generated when the coverage metric is below the coverage threshold.

[0114] A3: When the number of iterations is the maximum number of iterations, the initial test data set after iteration is determined as the basic test data set.

[0115] As can be seen from the above, when the diversity index and coverage index of the initial test data are not less than the diversity threshold and the coverage index is not less than the coverage threshold, the initial test data set is determined as the basic test data set. When the diversity index and / or coverage index of the initial test data are less than the diversity threshold and / or the coverage index is less than the coverage threshold, the initial test data set is iteratively processed until the diversity index and coverage index of the initial test data are not less than the diversity threshold and the coverage index is not less than the coverage threshold, or the maximum number of iterations is reached, and the basic test data set is output. Specifically, the above content can be found in formula (5).

[0116] (5)

[0117] In formula (5), The preset coverage threshold, The maximum number of iterations, D is the diversity threshold. * This is the basic test data set.

[0118] Optionally, after step S105, the generated test data also needs to be subject to compliance checks to ensure compliance with industry rules and compliance risks. Therefore, in another embodiment of this application, a data compliance check method is provided, including processes B1 to B2.

[0119] B1: Calculate the overall compliance rate of the basic test data set.

[0120] Specifically, the overall compliance rate of the basic test data set is expressed in the form of formula (6).

[0121] (6)

[0122] In formula (6), ComplianceRate(D*) is the overall compliance rate. The compliance threshold is typically set to 0.8 or 0.9.

[0123] B2: Generate a basic compliance report based on the overall compliance rate.

[0124] In process B2, when the overall compliance rate is lower than expected, a basic compliance report is generated based on the overall compliance rate. The specific format of the basic compliance report is as follows: Among them, ComplianceReport (D*) is the basic compliance report.

[0125] It should be noted that by calling function M i,j The basic compliance report is transmitted to the rule parsing agent, and the test data is iterated and optimized again for subsequent data optimization and correction.

[0126] S106: Perform boundary condition analysis on the rule description set to obtain the parameter boundary set.

[0127] This involves performing boundary condition analysis on the rule description set to identify the parameter boundaries of each rule description, thus obtaining the parameter boundary set. Its specific form is as follows: .in, It is a rule description The parameters in and These are the minimum and maximum values ​​of the parameter, B. r For the parameter boundary set.

[0128] S107: Generate boundary test data based on the parameter boundary set.

[0129] In step S107, the specific form of generating boundary test data based on the parameter boundary set is as follows: .in, It is a small offset used to generate test samples that slightly exceed the boundary.

[0130] Optionally, after step S107, it is also necessary to verify the validity of the boundary test data to avoid potential risks and improve the comprehensiveness and robustness of the test data. Therefore, in another embodiment of this application, a boundary data testing method is provided, including processes C1 to C2.

[0131] C1: Calculate the boundary coverage index of the boundary test data.

[0132] The specific form of the boundary coverage index in the boundary test data is as follows: Among them, BoundaryCoverage(Dboundary,R) is the boundary coverage index. Represents boundary test data Was the rule description tested? Medium parameters Value .

[0133] C2: Generates a boundary compliance report based on the boundary coverage metric.

[0134] Specifically, similar basic test data will be iterated based on the boundary compliance report.

[0135] Understandably, boundary test data can uncover potential compliance risks that conventional testing methods may overlook, thereby improving the comprehensiveness and robustness of test data.

[0136] In addition, compliance checks will be performed on the boundary test data, mainly including consistency checks, format compliance checks, and data sensitivity checks. A combination of knowledge base-based rule matching and semantic understanding from a large-scale pre-trained language model is typically used to achieve a comprehensive compliance assessment of the test data. Specifically, the compliance scoring formula for the knowledge base is shown in formula (7) (i.e., calculating the overall compliance rate of the basic test data set).

[0137] (7)

[0138] In formula (7), Compliance(d) is the overall compliance rate. For the applicable rule set, For rules The weight, It is a binary function; it returns 1 if the condition is met, and 0 otherwise.

[0139] Next, the compliance score of the knowledge base is fused with the compliance score of the large-scale pre-trained language model. Here, the score of the large-scale pre-trained language model is a comprehensive score of three checks: rule consistency check, format compliance check, and data sensitivity check. Each score is divided into four levels, from 0 to 3. Finally, the compliance verification agent scores the test data compliance as shown in formula (8).

[0140] (8)

[0141] in, and Let be the weighting coefficient, satisfying ,generally , . The rule consistency check score is used to evaluate large-scale pre-trained language models. The format compliance check score for evaluating large-scale pre-trained language models. This is the data sensitivity check score for evaluating large-scale pre-trained language models. The denominator 9 is the sum of the full scores for the three checks, used to normalize the scores of large-scale pre-trained language models to the range of 0-1. FinalCompliance(d) is the score for compliance with the test data.

[0142] S108: Determine the final test data set based on the basic test data set and the boundary test data.

[0143] Specifically, the final test data set is determined based on the basic test data set and the boundary test data. The basic test data set and the boundary test data are merged to obtain the integrated test data set. The integrated test data set is scored to obtain the scoring results. The final test data set is determined based on the scoring results.

[0144] Optionally, in another embodiment of this application, the specific implementation of step S108 is as follows: Figure 3 As shown, it includes the following steps:

[0145] S301: Merge the basic test data set and the boundary test data to obtain the integrated test data set.

[0146] Specifically, the data integration agent merges the basic test data set and the boundary test data to obtain the integrated test data set, which is: Dtotal is the integrated test data set.

[0147] Specifically, the data integration agent acts as the central coordinator in the system, responsible for managing interactions between different agents, coordinating workflows, and optimizing overall performance. The data integration agent maintains a system state vector: Among them, S coverage S represents the coverage of the current data. compiance For compliance rate, S diversity For diversity, S boundary S represents the boundary coverage rate. iterations This represents the number of iterations already performed.

[0148] S302: Calculate the overall quality score of the integrated test dataset.

[0149] Specifically, the comprehensive quality score of the integrated test data set is expressed in the form of formula (9).

[0150] (9)

[0151] In formula (9), Q(D) total (This is the overall quality score.) These are weighting coefficients, which can be dynamically adjusted according to the test scenario to meet certain requirements. .

[0152] S303: When the overall quality score is less than the quality threshold, obtain the state vector.

[0153] For example, when insufficient test coverage results in an overall quality score below the quality threshold, the data integration AI can identify the uncovered constraints and obtain a state vector. The specific form of the state vector is as follows: Among them, UncoveredConstraints(D total ,P) is the state vector.

[0154] S304: Generate supplementary data request based on state vector.

[0155] Specifically, the generation of targeted supplementary data requests based on the state vector takes the following form: SupplementRequest is a request for supplementary data.

[0156] S305: Generate supplementary data based on the supplementary data request.

[0157] Specifically, supplementary data requests are sent to the basic test data generation agent and the boundary test data generation agent respectively, triggering targeted supplementary data generation, the specific form of which is shown in the formula.

[0158] (10)

[0159] In formula (10), D Supplement To supplement the data.

[0160] S306: Merge the supplementary data and the integrated test data set to obtain the optimized test dataset. After determining the optimized test dataset as the integrated test data set, return to step S302.

[0161] Specifically, by merging the supplementary data and the integrated test data set, the optimized test dataset takes the following form: .

[0162] In addition, the supplementary data and the integrated test data set are merged to obtain the optimized test dataset. After the optimized test dataset is determined as the integrated test data set, the process returns to step S302 for iteration until the maximum number of iterations is reached and the overall quality score is not less than the quality threshold.

[0163] S307: When the overall quality score is not less than the quality threshold, the optimized test dataset will be determined as the final test dataset.

[0164] When the overall quality score is not less than the quality threshold, the iterative test data set is output, and its specific form is shown in formula (11).

[0165] (11)

[0166] In formula (11), It is a quality threshold. It represents the maximum number of iterations.

[0167] After outputting the iterative test data set, the final test data set is represented as shown in formula (12).

[0168] (12)

[0169] In formula (12), This is the maximum size limit of the output dataset. This is the final test data set.

[0170] As can be seen, in this way, the data integration agent not only achieves efficient integration of test data, but also ensures that the generated test dataset achieves the best balance in terms of coverage, compliance, diversity and boundary testing, thereby providing comprehensive and high-quality compliance test data.

[0171] Optionally, after step S108, new rules are generated based on test requirements and test data, and these new rules are added to the rule base to fill the gaps in the original rule base, making the rule base more comprehensive and covering more test scenarios. Therefore, another embodiment of this application provides a rule generation method, such as... Figure 4 As shown, it includes the following steps:

[0172] S401: Select a set of candidate rules from the rule base that correspond to the final test data set and the test target description.

[0173] Specifically, a large-scale pre-trained language model can be used to filter out a set of candidate rules from the rule base that correspond to the final test data set and the test target description. Its specific form is shown in formula (13).

[0174] (13)

[0175] In formula (13), It is a set of candidate new rules. For each candidate rule.

[0176] S402: Calculate the rule vector representation of each candidate rule in the candidate rule set.

[0177] The specific representation of the rule vector for each candidate rule in the candidate rule set is as follows: .in, This is the rule vector representation of the candidate rule.

[0178] S403: For each rule vector representation, calculate the similarity between the rule vector representation and each rule in the test data rule base.

[0179] The calculation rule vector represents the specific form of similarity with each rule in the test data rule base, as shown in formula (14).

[0180] (14)

[0181] In formula (14), r i For each rule in the test data rule base, Let the rule vector represent the similarity to each rule in the test data rule base.

[0182] S404: Filter out similarities below a preset threshold from all similarities and mark them as target similarities.

[0183] It is understandable that selecting similarities below a preset threshold from all similarities satisfies this condition: When the similarity of the target is such that the candidate rule is the new rule.

[0184] S405: Add the candidate rules corresponding to the target similarity to the test data rule base.

[0185] This involves adding candidate rules corresponding to the target similarity, i.e., new rules, to the test data rule base. .

[0186] It should be noted that the performance evaluation of test data is mainly conducted from the following dimensions: data coverage, data compliance rate, boundary coverage, and diversity metrics.

[0187] Data coverage: The degree to which the generated test data covers the rule constraints, and its specific form is shown in formula (15).

[0188] (15)

[0189] In formula (15), TotalCoverage is the data coverage rate, and C r It is a set of rules.

[0190] Data compliance rate: The proportion of generated data that meets compliance requirements, and its specific form is shown in formula (16).

[0191] (16)

[0192] In formula (16), ComplianceRate is the data compliance rate.

[0193] Boundary coverage: The degree to which the system covers the test of the boundary conditions of the rules. Its specific form is shown in formula (17).

[0194] (17)

[0195] In formula (17), BoundaryCoverage(D Boundary R) represents the boundary coverage rate.

[0196] Diversity index: the diversity and representativeness of the generated data, and its specific manifestation is shown in formula (18).

[0197] (18)

[0198] In formula (18), Diversity(D total () is a diversity indicator.

[0199] In addition, to better illustrate Figure 1 For the content shown, please refer to [link / reference]. Figure 5 The diagram illustrates a test data synthesis system. The user inputs a test objective description T. A rule parsing agent generates rule descriptions R and test constraints P based on the test objective description T. A basic test data generation agent generates a basic test data set, and a boundary test data generation agent generates a boundary test data set. A compliance verification agent verifies both the basic and boundary test data sets separately. After successful verification, a data integration agent merges the basic and boundary test data sets to obtain an integrated test data set. The integrated test data set undergoes fallback optimization to obtain the final test data set.

[0200] It should be noted that, based on the processes shown in S101-S108 above, this embodiment can achieve the following beneficial effects:

[0201] 1. A multi-agent collaborative system based on the synthesis of test data from a large-scale pre-trained language model was designed, comprising five specialized agents (rule parsing, basic data generation, boundary test data generation, compliance verification, and data integration agents). Each agent undertakes a specific function, collaborating to complete the test data generation task. This architecture design overcomes the limitations of traditional single models, significantly improving the system's ability to handle complex compliance requirements through division of labor and collaboration.

[0202] 2. By defining data diversity and coverage metrics, the system can quantitatively evaluate the quality of the generated test data and continuously iterate to generate more diverse and comprehensive data, achieving adaptive optimization in the test data generation process, which helps to improve data quality and coverage.

[0203] 3. In addition to focusing on routine testing scenarios, the system also places special emphasis on test coverage for boundary conditions and abnormal situations. By identifying the boundary values ​​of rule parameters, the system can generate test data for these boundaries, including samples that slightly exceed the boundaries, thereby comprehensively verifying the system's behavior under extreme conditions.

[0204] like Figure 6 The diagram shown is a schematic of the architecture of a test data generation device provided in an embodiment of this application. The generation device includes: a processing unit 100, a calculation unit 200, a filtering unit 300, a first generation unit 400, a second generation unit 500, an analysis unit 600, a third generation unit 700, and a determination unit 800.

[0205] The processing unit 100 is used to vectorize the description of the test target to obtain a vector representation.

[0206] The calculation unit 200 is used to calculate the similarity between the vector representation and each rule in the test data rule base.

[0207] The filtering unit 300 is used to filter out similarities greater than a preset similarity from all similarities and mark the rule corresponding to the similarity as the target rule.

[0208] The first generation unit 400 is used to generate a set of rule descriptions based on the target rules and the test target description.

[0209] The first generation unit 400 includes:

[0210] The data processing unit is used to perform structured processing on each rule description in the rule description set to obtain the constraint set.

[0211] The fourth generation unit is used to generate a test point matrix based on the set of constraints.

[0212] The fifth generation unit is used to generate an initial test data set based on the test point matrix and the test target description.

[0213] The sixth generation unit is used to generate coverage metrics based on the initial test data set and constraint set.

[0214] The sixth generation unit is used to generate a basic test data set based on the coverage metric and the initial test data.

[0215] The sixth generation unit is specifically used for: determining the initial test data set as the basic test data set when the diversity index of the initial test data is not less than the diversity threshold and the coverage index is not less than the coverage threshold; performing iterative processing on the initial test data set when the diversity index of the initial test data is less than the diversity threshold and / or the coverage index is less than the coverage threshold; and determining the iterated initial test data set as the basic test data set when the number of iterations is the maximum number of iterations.

[0216] The second generation unit 500 is used to generate a basic test data set based on the rule description set and the test target description.

[0217] Analysis unit 600 is used to perform boundary condition analysis on the rule description set to obtain the parameter boundary set.

[0218] The third generation unit 700 is used to generate boundary test data based on the parameter boundary set.

[0219] Unit 800 is used to determine the final test data set based on the basic test data set and the boundary test data.

[0220] The determination unit 800 is specifically used for: merging the basic test data set and the boundary test data to obtain the integrated test data set; calculating the comprehensive quality score of the integrated test data set; when the comprehensive quality score is less than the quality threshold, obtaining the state vector; generating a supplementary data request based on the state vector; generating supplementary data based on the supplementary data request; merging the supplementary data and the integrated test data set to obtain the optimized test dataset, and after determining the optimized test dataset as the integrated test data set, returning to the step of calculating the comprehensive quality score of the integrated test data set; when the comprehensive quality score is not less than the quality threshold, determining the optimized test dataset as the final test data set.

[0221] In summary, by selecting relevant target rules, a set of rule descriptions and a set of basic test data are generated. Based on this, boundary condition analysis is performed to generate boundary test data, ultimately resulting in a complete set of test data. This method, through precise matching of rules and test objectives combined with boundary analysis, reduces manual intervention and improves the automation level of test data generation, thereby significantly improving the efficiency of test data generation.

[0222] Combination Figure 6 The generating apparatus, as shown, also includes:

[0223] The compliance rate calculation unit is used to calculate the overall compliance rate of the basic test data set.

[0224] The Basic Compliance Report Generation Unit is used to generate a basic compliance report based on the overall compliance rate.

[0225] Combination Figure 6 The generating apparatus, as shown, also includes:

[0226] The indicator generation unit is used to calculate the boundary coverage indicator of the boundary test data.

[0227] The boundary compliance report generation unit is used to generate boundary compliance reports based on the boundary coverage metric.

[0228] Combination Figure 6 The generating apparatus, as shown, also includes:

[0229] The rule filtering unit is used to filter out a set of candidate rules from the rule base that correspond to the final test data set and the test target description.

[0230] The vector computation unit is used to compute the rule vector representation of each candidate rule in the candidate rule set.

[0231] The similarity unit is used to calculate the similarity between each rule vector representation and each rule in the test data rule base.

[0232] The identification unit is used to filter out similarities below a preset threshold from all similarities and identify them as target similarities.

[0233] The Add Unit is used to add candidate rules corresponding to target similarity to the test data rule base.

[0234] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. In particular, for system or system embodiments, since they are fundamentally similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. Components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0235] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0236] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating test data, characterized in that, include: The description of the test target is vectorized to obtain a vector representation; Calculate the similarity between the vector representation and each rule in the test data rule base; Filter out similarities greater than a preset similarity from all similarities, and mark the rules corresponding to the similarities as target rules; Based on the target rules and the test target description, a set of rule descriptions is generated; Based on the rule description set and the test target description, a basic test data set is generated; Boundary condition analysis is performed on the rule description set to obtain the parameter boundary set; Based on the aforementioned parameter boundary set, boundary test data is generated; Based on the basic test data set and the boundary test data, the final test data set is determined.

2. The method according to claim 1, characterized in that, The step of generating a rule description set based on the target rule and the test target description includes: Each rule description in the rule description set is structured to obtain a set of constraints; A test point matrix is ​​generated based on the set of constraints. Based on the test point matrix and the test target description, an initial test data set is generated; Based on the initial test data set and the constraint set, a coverage metric is generated; A basic test data set is generated based on the coverage metric and the initial test data.

3. The method according to claim 2, characterized in that, The step of generating a basic test data set based on the coverage metric and the initial test data includes: When the diversity index of the initial test data is not less than the diversity threshold and the coverage index is not less than the coverage threshold, the initial test data set is determined as the basic test data set. When the diversity index of the initial test data is less than the diversity threshold, and / or the coverage index is less than the coverage threshold, the initial test data set is subjected to iterative processing. When the number of iterations is the maximum number of iterations, the initial test data set after iteration is determined as the basic test data set.

4. The method according to claim 1, characterized in that, After generating the basic test data set based on the rule description set and the test target description, the process further includes: Calculate the overall compliance rate of the aforementioned basic test dataset; A basic compliance report is generated based on the overall compliance rate.

5. The method according to claim 1, characterized in that, After generating boundary test data based on the parameter boundary set, the process further includes: Calculate the boundary coverage index of the boundary test data; A boundary compliance report is generated based on the boundary coverage metric.

6. The method according to claim 1, characterized in that, The process of determining the final test data set based on the basic test data set and the boundary test data includes: Merge the basic test dataset and the boundary test dataset to obtain the integrated test dataset. Calculate the overall quality score of the integrated test data set; When the overall quality score is less than the quality threshold, a state vector is obtained; Generate supplementary data requests based on the state vector; Based on the aforementioned supplementary data, request the generation of supplementary data; After merging the supplementary data and the integrated test data set to obtain the optimized test dataset, and determining the optimized test dataset as the integrated test data set, the process returns to the step of calculating the comprehensive quality score of the integrated test data set. When the overall quality score is not less than the quality threshold, the optimized test dataset is determined as the final test data set.

7. The method according to claim 1, characterized in that, Also includes: Filter out a set of candidate rules from the rule base that correspond to the final test data set and the test target description; Calculate the rule vector representation of each candidate rule in the candidate rule set; For each rule vector representation, calculate the similarity between the rule vector representation and each rule in the test data rule base; Filter out similarities below a preset threshold from all similarities and mark them as target similarities; The candidate rules corresponding to the target similarity are added to the test data rule base.

8. A test data generation device, characterized in that, include: The processing unit is used to vectorize the description of the test target to obtain a vector representation. A calculation unit is used to calculate the similarity between the vector representation and each rule in the test data rule base; The filtering unit is used to filter out similarities greater than a preset similarity from all similarities, and to identify the rule corresponding to the similarity as the target rule; The first generation unit is used to generate a set of rule descriptions based on the target rule and the test target description; The second generation unit is used to generate a basic test data set based on the rule description set and the test target description; The analysis unit is used to perform boundary condition analysis on the rule description set to obtain the parameter boundary set; The third generation unit is used to generate boundary test data based on the parameter boundary set; The determining unit is used to determine the final test data set based on the basic test data set and the boundary test data.

9. The apparatus according to claim 8, characterized in that, The first generation unit includes: The data processing unit is used to perform structured processing on each rule description in the rule description set to obtain a set of constraints. The fourth generation unit is used to generate a test point matrix based on the set of constraints. The fifth generation unit is used to generate an initial test data set based on the test point matrix and the test target description; The sixth generation unit is used to generate a coverage index based on the initial test data set and the constraint set; The sixth generation unit is used to generate a basic test data set based on the coverage index and the initial test data.

10. The apparatus according to claim 8, characterized in that, The sixth generation unit is specifically used for: When the diversity index of the initial test data is not less than the diversity threshold and the coverage index is not less than the coverage threshold, the initial test data set is determined as the basic test data set. When the diversity index of the initial test data is less than the diversity threshold, and / or the coverage index is less than the coverage threshold, the initial test data set is subjected to iterative processing. When the number of iterations is the maximum number of iterations, the initial test data set after iteration is determined as the basic test data set.

Images