Product test scheme design method and device and storage medium

Abstract

The invention provides a product test scheme design method and device and a storage medium. Establishing a standard test case library and a definable test case library; when a test scheme is constructed, an information file of a to-be-tested product is provided for the large language model, and the large language model is restrained through a first system cue word to analyze the information file to obtain a model and a version of the to-be-tested product; on the basis of test function instructions of the test cases in the library, standard to-be-tested functions are classified and summarized from the information file, and the to-be-tested functions, newly-added to-be-tested functions and respective sources can be defined; the large language model matches a target standard test case and a target definable test case suitable for the to-be-tested product from a standard test case library and a definable test case library based on the standard to-be-tested function, the definable to-be-tested function, the version and the model under the constraint of a second system cue word; and creating a test scheme task by combining a definable to-be-tested function source, a newly added to-be-tested function and the source of the newly added to-be-tested function of the matching target definable test case.

Description

Technical Field

[0001] The present invention relates to the technical field of product test scheme design, and in particular to a product test scheme design method, device and storage medium. Background Art

[0002] With the rapid development of high-tech in the world today, intelligent driving has become one of the main innovative development directions of the automotive industry. The safety and practicality of the driving assistance system are undoubtedly the most concerned issues for major OEMs and users. Therefore, pre-production testing is essential, and the actual vehicle test is very limited in the test scenarios and functions due to the limitations of test scenarios, safety and economy. Therefore, hardware-in-the-loop simulation testing has become an important means to ensure the quality of the software system.

[0003] Test case design is the most basic and important part of the entire testing work. The design of test cases directly affects the accuracy and reliability of the test results. Therefore, how to achieve the most complete test case coverage in a short period of time has become the most important research topic in the field of hardware-in-the-loop simulation testing. All test cases are designed to cover the functions and performance involved in the product as comprehensively as possible. Designing a large number of test cases is an extremely time-consuming task and is highly dependent on the experience and energy of the testers, which can easily lead to test point errors and waste of manpower. Summary of the invention

[0004] In order to solve the above technical problem or at least partially solve the above technical problem, the present invention provides a product test scheme design method, device and storage medium.

[0005] In a first aspect, the present invention provides a product test scheme design method, comprising:

[0006] Divide the existing test cases into standard test cases and definable test cases in advance, and establish a standard test case library and a definable test case library for storage respectively;

[0007] When constructing a test plan for any product to be tested, obtain an information file of the product to be tested, wherein the information file includes the model, version and functional specifications;

[0008] Providing the information file to the large language model, analyzing the information file by constraining the large language model with the first system prompt word to obtain the model and version of the product to be tested, and summarizing the standard functions to be tested and their sources, the definable functions to be tested and their sources, and the newly added functions to be tested and their sources from the functional specifications based on the test function descriptions corresponding to the test cases in the standard test case library and the definable test case library;

[0009] Under the constraints of the second system prompt words, the large language model matches the target standard test cases and target definable test cases suitable for the product to be tested from the standard test case library and the definable test case library based on the standard test function to be tested, the definable test function to be tested, the version and the model after verification;

[0010] Create a test plan task using target standard test cases, target definable test cases, definable test function sources matching the target definable test cases, newly added test functions and their sources, and formulate a test plan using the test plan task.

[0011] Furthermore, standard test cases are established based on the basic functional requirements and test procedures for products in the standard, and for all basic tests that the product must meet; test cases can be defined to establish requirements for functions to be tested in addition to basic functions in the functional specifications of the product.

[0012] Furthermore, the large language model summarizes various functions of the product to be tested based on the functional description in the functional specification, and the large language model gives the content of the functional specification summarizing the functions to be tested as part of the source;

[0013] The large language model performs correlation matching on the various functions of the product to be tested, and matches the TopK most relevant test function descriptions from the test function descriptions corresponding to the test cases;

[0014] The relevance of the top K most relevant test function descriptions of any function to be tested given by the statistical large language model is calculated; if the proportion of test function descriptions belonging to the same test case in the test function descriptions that meet the relevance criteria exceeds the set ratio threshold, the function to be tested is a function to be tested of the corresponding category;

[0015] If the relevance of any test function description content of any function to be tested does not reach the relevance threshold, or if there is a test function description content whose relevance reaches the relevance threshold but the proportion of the test function description content of the test function description belonging to the same test case does not reach the proportion threshold, then the function to be tested is determined to be a newly added function to be tested;

[0016] The large language model provides the TopK most relevant test function description contents based on which the function to be tested is classified as another part of the source.

[0017] Furthermore, various types of functions to be tested and their sources given by the large language model analysis information file are saved, and the sources are further used to check whether the functions to be tested extracted and classified by the large language model are reasonable. If not, the errors are corrected.

[0018] Furthermore, through the second system prompt word constraint large language model, based on the standard functions to be tested, definable functions to be tested, versions and models after verification, target standard test cases and target definable test cases are matched from the standard test case library and the definable test case library. The corresponding versions and models of the target standard test cases and target definable test cases are similar to those of the products to be tested, and the target standard test cases and target definable test cases match the summarized standard functions to be tested and definable functions to be tested.

[0019] Furthermore, the test case design methods such as equivalence class partitioning, boundary value analysis, orthogonal experiment and scenario design are used to map the test requirements of the new functions into the test cases and construct new test cases.

[0020] Furthermore, new test cases and their test function descriptions for new functions to be tested for any model version of the product are added to the definable test case library.

[0021] Furthermore, monitor the changes in the basic functional requirements and test procedures for the product in the test standards, and update the standard test cases and the test function descriptions of the standard test cases according to the changes.

[0022] In a second aspect, the present invention provides a product test scheme design device, comprising: at least one processing unit, the processing unit is connected to a storage unit via a bus unit, the storage unit stores a computer program, and when the computer program is executed by the processing unit, the product test scheme design method is implemented.

[0023] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein when the computer program is executed by a processor, the product test scheme design method is implemented.

[0024] The above technical solution provided by the embodiment of the present invention has the following advantages compared with the prior art:

[0025] When building a test plan for this application, the information file of the product to be tested is provided to the large language model, and the large language model is analyzed through the first system prompt word constraint to obtain the model and version of the product to be tested. Based on the test function description of the test case in the standard test case library and the definable test case library, the standard test function, the definable test function, the newly added test function and their respective sources are classified and summarized from the information file; under the constraint of the second system prompt word, the large language model matches the target standard test case and target definable test case suitable for the product to be tested from the standard test case library and the definable test case library based on the standard test function, the definable test function, the version and the model; and creates a test plan task in combination with the definable test function source, the newly added test function and its source of the matching target definable test case. With the help of the analysis ability of the language model, the standard test function, the definable test function and the newly added test function are directly classified and summarized from the information file, and with the help of the large language model, the target standard test case and the target definable test case suitable for the product to be tested are matched from the standard test case library and the definable test case library based on the standard test function, the definable test function, the version and the model. The summary efficiency and coverage of the functions to be tested are effectively improved; and the test plan of the product to be tested is constructed by directly reusing standard test cases, modifying and reusing target definable test cases, and constructing new test cases for newly added functions to be tested. The reuse rate of existing test cases is increased to the greatest extent, the waste of manpower and material resources is avoided, and the efficiency and accuracy of test case design are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0028] Figure 1 A flowchart of a product testing solution design method provided in an embodiment of the present application;

[0029] Figure 2 An architectural diagram of a product testing solution design method provided in an embodiment of the present application;

[0030] Figure 3 A flowchart of the large language model provided in the embodiment of the present application working in the first stage;

[0031] Figure 4A schematic diagram of a product testing solution design device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0034] Example 1

[0035] like Figure 1 and Figure 2 As shown, the present invention provides a product test scheme design method, comprising:

[0036] Existing test cases are divided into standard test cases and definable test cases in advance. Standard test cases are established based on the basic functional requirements and test procedures for products in the standard, and for all basic tests that the product must meet; definable test cases are established for the functional requirements that need to be tested in addition to the basic functions in the product's functional specifications.

[0037] Standard test cases are completely universal, and corresponding standard test cases need to be configured in the test plan for any model and version of the product; definable test cases are partially universal. For example, as product models and versions develop, some functions are maintained but performance is improved, and some new functions are added; by modifying the parameters of the definable test cases of old versions and models of products and adding indicators, they can be applied to new versions and models of products.

[0038] A standard test case library and a definable test case library are established to store standard test cases and definable test cases respectively; in the standard test case library and the definable test case library, each standard test case and definable test case is associated with a test function description corresponding to the test case. This application formats the test function description corresponding to the test case and segments it using a text segmentation tool; each segmented content is encoded into a vector using a text encoding model.

[0039] In the process of formulating test plans for the products to be tested, in order to automatically, accurately and efficiently screen out reusable standard test cases and definable test cases, it is more convenient to formulate test plans, such as Figure 2 As shown, the present application introduces a large language model, and configures a first system prompt word and a second system prompt word that can be optionally enabled for the large language model. The system prompt word refers to a fixed instruction text configured for the large language model, which is used to guide the large language model to generate the required output.

[0040] When constructing a test plan for any product to be tested, obtain the information file of the product to be tested, wherein the information file includes the model, version and functional specifications.

[0041] In the first stage: enable the first system prompt word, provide the information file to the large language model, and analyze the information file through the first system prompt word constraint to obtain the model and version of the product to be tested; constrain the large language model through the first system prompt word based on the test function descriptions corresponding to the test cases in the standard test case library and the definable test case library to summarize the standard functions to be tested and their sources, the definable functions to be tested and their sources, and the newly added functions to be tested and their sources from the functional specifications in the information file.

[0042] Specifically, Figure 3 As shown, the first system prompt word constrains the large language model to summarize various functions to be tested of the product to be tested according to the functional description in the functional specification;

[0043] The first system prompt word constrains the large language model to perform correlation matching on various functions of the product to be tested, and matches the TopK most relevant test function description contents from the test function description corresponding to the test case;

[0044] The relevance of the TopK most relevant test function description contents of any function to be tested given by the statistical large language model is calculated; among the test function description contents that meet the relevance standard, if the proportion of the test function descriptions belonging to the same test case exceeds the set proportion threshold, the function to be tested is a function to be tested of the corresponding category; in the specific implementation process, a relevance threshold is preset to measure whether the relevance standard is met.

[0045] For example: count the relevance of the top K most relevant test function description contents of the function to be tested A; among the test function description contents that meet the relevance standard, the number of test function descriptions belonging to the standard test case A is N, and N / K is greater than the ratio threshold, that is, the proportion of the test function description contents that meet the relevance standard of the test function descriptions belonging to the same test case exceeds the set ratio threshold, then the function to be tested A is a standard function to be tested;

[0046] If the relevance of any test function description content of any function to be tested does not reach the relevance threshold, or if there is a test function description content whose relevance reaches the relevance threshold but the proportion of the test function description content of the test function description belonging to the same test case does not reach the proportion threshold, then the function to be tested is determined to be a newly added function to be tested;

[0047] During the specific implementation process, the large language model summarizes the content in the functional specification of the function to be tested as part of the source; the large language model gives the TopK most relevant test function description contents based on the classification of the function to be tested as another part of the source.

[0048] The various functions to be tested and their sources given by the large language model analysis information file are saved, and the sources are further used to check whether the functions to be tested extracted and classified by the large language model are reasonable. If not, correct the errors and execute the second stage. If reasonable, proceed directly to the second stage.

[0049] In the second stage, under the constraints of the second system prompt words, the large language model matches the target standard test cases and target definable test cases suitable for the product to be tested from the standard test case library and the definable test case library based on the standard functions to be tested, definable functions to be tested, versions and models after verification.

[0050] Specifically, the second system prompt words are enabled, and the large language model is constrained by the second system prompt words to match target standard test cases and target definable test cases from the standard test case library and the definable test case library based on the standard functions to be tested, definable functions to be tested, versions and models after verification. The corresponding versions and models of the target standard test cases and target definable test cases are similar to those of the product to be tested, and the target standard test cases and target definable test cases match the summarized standard functions to be tested and definable functions to be tested.

[0051] Create a test plan task using target standard test cases, target definable test cases, definable test function sources matching the target definable test cases, newly added test functions and their sources, and formulate a test plan using the test plan task.

[0052] The target standard test case is universal and can be applied to the product to be tested without modification. The test plan task directly reuses the target standard test case to target the standard function to be tested. The target definable test case is partially universal and can be used for the product to be tested after modifying the parameters or indicators. Modifying the parameters or indicators needs to be based on the functional scope of the definable function to be tested source that matches the target definable test case. The test plan task provides the target definable test case and the definable function to be tested source that matches the target definable test case. New functions to be tested need to be developed specifically by the test case developers, and the development needs to be based on the source of the new functions to be tested. The test plan task provides the new functions to be tested and their sources.

[0053] The test plan task assigns the modification of definable test cases and the construction of new test cases to test case developers; test case developers use equivalence class partitioning, boundary value analysis, orthogonal experiment and scenario design test case design methods to map the test requirements of new functions into test cases and construct new test cases.

[0054] Add new test cases and their test function descriptions for any model version of the product to the definable test case library. Monitor changes in the basic functional requirements and test procedures for products in the test standards, and update standard test cases and their test function descriptions according to the changes.

[0055] Example 2

[0056] like Figure 4 The embodiment of the present invention provides a product test scheme design device, comprising: at least one processing unit, the processing unit is connected to a storage unit via a bus unit, the storage unit stores a computer program, and the processing unit executes the computer program to implement the product test scheme design method, including:

[0057] Divide the existing test cases into standard test cases and definable test cases in advance, and establish a standard test case library and a definable test case library for storage respectively;

[0058] When constructing a test plan for any product to be tested, obtain an information file of the product to be tested, wherein the information file includes the model, version and functional specifications;

[0059] Providing the information file to the large language model, analyzing the information file by constraining the large language model with the first system prompt word to obtain the model and version of the product to be tested, and summarizing the standard functions to be tested and their sources, the definable functions to be tested and their sources, and the newly added functions to be tested and their sources from the functional specifications based on the test function descriptions corresponding to the test cases in the standard test case library and the definable test case library;

[0060] Under the constraints of the second system prompt words, the large language model matches the target standard test cases and target definable test cases suitable for the product to be tested from the standard test case library and the definable test case library based on the standard test function to be tested, the definable test function to be tested, the version and the model after verification;

[0061] Create a test plan task using target standard test cases, target definable test cases, definable test function sources matching the target definable test cases, newly added test functions and their sources, and formulate a test plan using the test plan task.

[0062] Of course, the computer program stored in the storage unit of a product test scheme design device provided in an embodiment of the present invention is not limited to the method operations described above, and can also execute related operations in a product test scheme design method provided in any embodiment of the present invention.

[0063] Example 3

[0064] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed, the product test scheme design method is implemented, including:

[0065] Divide the existing test cases into standard test cases and definable test cases in advance, and establish a standard test case library and a definable test case library for storage respectively;

[0066] When constructing a test plan for any product to be tested, obtain an information file of the product to be tested, wherein the information file includes the model, version and functional specifications;

[0067] Providing the information file to the large language model, analyzing the information file by constraining the large language model with the first system prompt word to obtain the model and version of the product to be tested, and summarizing the standard functions to be tested and their sources, the definable functions to be tested and their sources, and the newly added functions to be tested and their sources from the functional specifications based on the test function descriptions corresponding to the test cases in the standard test case library and the definable test case library;

[0068] Under the constraints of the second system prompt words, the large language model matches the target standard test cases and target definable test cases suitable for the product to be tested from the standard test case library and the definable test case library based on the standard test function to be tested, the definable test function to be tested, the version and the model after verification;

[0069] Create a test plan task using target standard test cases, target definable test cases, definable test function sources matching the target definable test cases, newly added test functions and their sources, and formulate a test plan using the test plan task.

[0070] A computer-readable storage medium provided in an embodiment of the present invention stores a computer program which is not limited to the method operations described above, but can also execute related operations in a product test scheme design method provided in any embodiment of the present invention.

[0071] In the embodiments provided by the present invention, it should be understood that the disclosed methods, devices and storage media can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the description division of the circuit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling shown or discussed can be indirect coupling through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0072] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A product test scheme design method, characterized in that: include: Divide the existing test cases into standard test cases and definable test cases in advance, and establish a standard test case library and a definable test case library for storage respectively; When constructing a test plan for any product to be tested, obtain an information file of the product to be tested, wherein the information file includes the model, version and functional specifications; Providing the information file to the large language model, analyzing the information file by constraining the large language model with the first system prompt word to obtain the model and version of the product to be tested, and classifying and summarizing the standard functions to be tested and their sources, the definable functions to be tested and their sources, and the newly added functions to be tested and their sources from the functional specifications based on the test function descriptions corresponding to the test cases in the standard test case library and the definable test case library; Under the constraints of the second system prompt words, the large language model matches the target standard test cases and target definable test cases suitable for the product to be tested from the standard test case library and the definable test case library based on the standard test function to be tested, the definable test function to be tested, the version and the model after verification; Create a test plan task using target standard test cases, target definable test cases, definable test function sources matching the target definable test cases, newly added test functions and their sources, and formulate a test plan using the test plan task.

2. The product test scheme design method according to claim 1, characterized in that: Standard test cases are established based on the basic functional requirements and test procedures for products in the standard, and for all basic tests that the product must meet; definable test cases can be established for the functional requirements that need to be tested in addition to the basic functions in the functional specifications of the product.

3. The product test scheme design method according to claim 1, characterized in that: The large language model summarizes various functions of the product to be tested based on the functional description in the functional specification, and the large language model gives the content of the functional specification summarizing the functions to be tested as part of the source; The large language model performs correlation matching on the various functions of the product to be tested, and matches the TopK most relevant test function descriptions from the test function descriptions corresponding to the test cases; The relevance of the top K most relevant test function descriptions of any function to be tested given by the statistical large language model is calculated; if the proportion of test function descriptions belonging to the same test case in the test function descriptions that meet the relevance criteria exceeds the set ratio threshold, the function to be tested is a function to be tested of the corresponding category; If the relevance of any test function description content of any function to be tested does not reach the relevance threshold, or if there is a test function description content whose relevance reaches the relevance threshold but the proportion of the test function description content of the test function description belonging to the same test case does not reach the proportion threshold, then the function to be tested is determined to be a newly added function to be tested; The large language model provides the TopK most relevant test function description contents based on which the function to be tested is classified as another part of the source.

4. The product test scheme design method according to claim 1, characterized in that: The various functions to be tested and their sources given by the large language model analysis information file are saved, and the sources are further used to check whether the functions to be tested extracted and classified by the large language model are reasonable. If not, the errors are corrected.

5. The product test scheme design method according to claim 1, characterized in that: Through the second system prompt word constraint large language model, based on the standard functions to be tested, definable functions to be tested, versions and models after verification, target standard test cases and target definable test cases are matched from the standard test case library and the definable test case library. The corresponding versions and models of the target standard test cases and target definable test cases are similar to those of the products to be tested, and the target standard test cases and target definable test cases match the summarized standard functions to be tested and definable functions to be tested.

6. The product test scheme design method according to claim 1, characterized in that: By using the test case design method of equivalence class partitioning, boundary value analysis, orthogonal experiment and scenario design, the test requirements of the new functions are mapped to the test cases to construct new test cases.

7. The product test scheme design method according to claim 6, characterized in that: Add new test cases and their test function descriptions for the new functions to be tested for any model version of the product to the definable test case library.

8. The product test scheme design method according to claim 1, characterized in that: Monitor changes in basic functional requirements and test procedures for products in test standards, and update standard test cases and their test function descriptions based on the changes.

9. A product test scheme design device, characterized in that: include: At least one processing unit, the processing unit is connected to a storage unit via a bus unit, the storage unit stores a computer program, and when the computer program is executed by the processing unit, a product test scheme design method as described in any one of claims 1-8 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the product test scheme design method as described in any one of claims 1 to 8 is implemented.

Images