Test information processing method, integrated test device, medium and controller
By converting the requirement information into logical requirements and combining semantic retrieval and logical retrieval, the problems of low efficiency of test case generation and poor retrieval flexibility in the prior art are solved, and efficient and accurate automatic generation of test cases is achieved.
Patent Information
- Application Number
- CN202411098449.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-12-06
AI Technical Summary
In the prior art, the generation of test cases depends on manual compilation, and the database retrieval function is poor, and it is impossible to effectively distinguish between logic and domain knowledge, resulting in inaccurate search results for variable test scenarios, and secondary labor is required after finding similar test cases. Corrected, low processing efficiency.
The first requirement mapping step is used to convert the requirement information into logical requirements, and the composite search step is combined with semantic search and logical search. BGE asymmetric encoding and Faiss similarity search library is used, and the BM25 text matching algorithm and few-sample learning Few-Shot is used. -Learn process, generate natural language test cases.
It significantly improves the generation efficiency and quality of test cases, reduces the dependence on domain knowledge, and realizes fast and efficient test case retrieval and automatic generation.
Smart Images

Figure CN119088857A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of intelligent manufacturing, and in particular relates to a test information processing method, an integrated test device, a medium and a controller. Background Art
[0002] Integration testing is an important part of product development and manufacturing, involving the integration and testing and verification of many components. It is used to ensure smooth interaction between different components and meet technical specifications and functional safety requirements, so that the system operation meets established standards and ensures safe and reliable operation of the product.
[0003] In the related art, test cases are generated manually. In the absence of auxiliary tools, the implementation process is highly dependent on the operator, and the efficiency of processing relevant information needs to be improved.
[0004] Among them, database retrieval also faces three technical problems: first, the search function of the database often only supports precise matching of natural language text, and has poor flexibility; second, the relevant search methods cannot effectively distinguish between the logical part and the domain knowledge part of the test requirements, and have a strong dependence on the knowledge field, resulting in difficulty in capturing valuable search results for changing test scenarios (such as new demand function points); third, when similar test cases (historical cases) are found, secondary labor is still required to correct them, and processing efficiency needs to be improved.
[0005] In summary, how to quickly and efficiently retrieve the historical test database and obtain relevant test requirement-test case pairs based on the natural language requirements proposed by users, and automatically generate new test cases based on current test requirements to effectively improve the efficiency of integrated testing and ensure the quality of test cases is a technical problem that needs to be solved urgently. Summary of the invention
[0006] An embodiment of the present invention discloses a test information processing method, the core of which is to set a first requirement mapping step, a second composite retrieval step, and a third coding recovery step; the first requirement mapping step can convert current requirement information into logical requirement information, and convert historical requirement description information in a first database into logical description information.
[0007] Furthermore, the second composite retrieval step can enhance the data filtering capability through the semantic retrieval step and the logical retrieval step; its semantic retrieval step can retrieve the first requirement sample set RS from the historical requirement description information; the logical retrieval step can retrieve the second requirement sample set φ from the logical description information; then, through the third encoding recovery step, the first requirement sample set RS and the second requirement sample set φ are converted into natural language, and by comparing the elements of the first requirement sample set RS and the second requirement sample set φ, alternative test cases for the current requirement information are obtained.
[0008] Among them, the first requirement mapping step can use a semantic model to encode the current requirement information and the historical requirement description information and use it in the semantic retrieval step; while the logical requirement information and the logical description information are described in a logical language that removes text information but retains logical symbols.
[0009] Specifically, the first requirement sample set RS is composed of RSi=<Ri,Ti> Ri is the i-th information of the first requirement, Ti is the i-th information of the first use case; the second requirement sample set φ is composed of φi=<r i,ti> Composition: ri is the i-th information of the second requirement, ti is the i-th information of the second use case, i=1 to N, and N is a positive integer.
[0010] Among them, the i-th information of the first requirement corresponds one-to-one to the i-th information of the first use case, and the i-th information of the second requirement corresponds one-to-one to the i-th information of the second use case; in addition, the third coding recovery step can use a preset signal matrix to perform natural language conversion, and the signal matrix records the correspondence between historical requirement description information and preset test signals.
[0011] Furthermore, the BGE asymmetric encoding and decoding structure may be used in the first requirement mapping step, and the Faiss similarity search library may be used in the semantic retrieval step, and the first N elements of the first requirement sample set RS may be obtained according to the similarity, where N is a positive integer.
[0012] Furthermore, the semantic encoding model adopted in its semantic retrieval step can be the M3E model; the logical retrieval step can adopt the BM25 text matching algorithm, and the query information Q and the document to be matched d can be obtained through a weighted summation process to obtain the corresponding similarity score, namely Score (Q, d), and the first N elements of the second demand sample set φ can be obtained by comparing the similarity scores, where N is a positive integer.
[0013] Furthermore, the test information processing method may also be provided with a fourth sample learning step; the first requirement sample set RS and the second requirement sample set φ may be combined using a few-shot learning process to obtain a natural language instance, and combined with preset prompt words and current requirement information, and input into a fourth data processing model to improve the test case.
[0014] Specifically, N=1 can be selected, and its fourth data processing model can adopt a language model, which includes a natural language processing model ChatGPT; and its first database includes a historical database of preset product tests.
[0015] Correspondingly, an embodiment of the present invention also discloses an integrated testing device, whose core components include a first requirement mapping unit, a second compound retrieval unit, and a third coding recovery unit; the first requirement mapping unit can convert current requirement information into logical requirement information, and convert historical requirement description information in the first database into logical description information; the second compound retrieval unit retrieves a first requirement sample set RS from the historical requirement description information, and retrieves a second requirement sample set (012) φ from the logical description information; the third coding recovery unit converts the first requirement sample set RS and the second requirement sample set φ into natural language, and compares the elements of the first requirement sample set RS and the second requirement sample set φ to obtain alternative test cases for the current requirement information.
[0016] Accordingly, its first demand mapping unit uses a semantic model to encode current demand information and historical demand description information and uses it for a semantic retrieval step; its logical demand information and logical description information are described using a logical language that removes text information but retains logical symbols.
[0017] Specifically, the first requirement sample set RS is composed of RSi=<Ri,Ti> Ri is the i-th information of the first requirement, Ti is the i-th information of the first use case; the second requirement sample set φ is composed of φi=<r i,ti> Composition: ri is the i-th information of the second requirement, ti is the i-th information of the second use case, i=1 to N, and N is a positive integer.
[0018] Among them, the i-th information of the first requirement corresponds one-to-one to the i-th information of the first use case, and the i-th information of the second requirement corresponds one-to-one to the i-th information of the second use case; the third coding recovery unit uses a preset signal matrix to perform natural language conversion, and the signal matrix records the correspondence between historical requirement description information and preset test signals.
[0019] Specifically, the first demand mapping unit may adopt a BGE asymmetric encoding and decoding structure, the semantic retrieval step may adopt a Faiss similarity search library, and the first N elements of the first demand sample set RS may be obtained according to the similarity, where N is a positive integer.
[0020] Similarly, the semantic encoding models that can be used in its semantic retrieval step include the M3E model; the logical retrieval step can use the BM25 text matching algorithm, and obtain the corresponding similarity score, namely Score (Q, d), for the query information Q and the document to be matched d through a weighted sum process, and the first N elements of the second demand sample set φ can be obtained by comparing the similarity scores, where N is a positive integer.
[0021] Furthermore, the integrated testing device may also be provided with a fourth sample learning unit; the first requirement sample set RS and the second requirement sample set φ may be combined by using the few-shot learning process to obtain a natural language instance, and combined with the preset prompt words and current requirement information, input into the fourth data processing model to improve the test case; wherein N=1 may be selected, and the fourth data processing model may adopt a language model, which may be a natural language processing model ChatGPT; its first database is a historical database of preset product tests.
[0022] Correspondingly, an embodiment of the present invention also discloses a computer storage medium and a controller; the computer storage medium includes a storage medium body for storing a computer program; when the computer program is executed by a microprocessor, any of the above test information processing methods can be implemented; the controller includes any of the above integrated test devices and / or computer storage media, and adopts the same technical means to solve the same technical problems.
[0023] In summary, the first requirement mapping step / unit of the present invention converts the current requirement information into logical requirement information, and converts the historical requirement description information into logical description information; the second composite retrieval step / unit obtains the first requirement sample set RS and the second requirement sample set φ through semantic retrieval and logical retrieval; the third encoding recovery step / unit compares the retrieval results to obtain test cases; the fourth sample learning step / unit improves the test data by few-shot learning FSL (Few-Shot-Learn); its products and methods bring together the respective advantages of semantic retrieval and logical retrieval, and can use language models to generate test cases that conform to natural language specifications in scenarios such as product integration testing, and use the logical matching process of test information to reduce the retrieval process's dependence on specific domain knowledge.
[0024] It should be noted that the terms "first", "second" and similar terms used in this article are only for describing the various components in the technical solution, and do not constitute a limitation of the technical solution, nor can they be understood as an indication or suggestion of the importance of the corresponding elements; elements with "first", "second" and similar terms indicate that the corresponding technical solution contains at least one of the element. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solution of the present invention and facilitate further understanding of the technical effects, technical features and purposes of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings. The accompanying drawings constitute an essential part of the specification and are used together with the embodiments of the present invention to illustrate the technical solution of the present invention, but do not constitute a limitation to the present invention.
[0026] The same reference numerals in the accompanying drawings represent the same components, and the details of each drawing are as follows.
[0027] Figure 1 It is a schematic diagram of the information processing flow of the method and product embodiment of the present invention.
[0028] Figure 2 This is the similarity score formula of the BM25 (Best Matching 25) algorithm.
[0029] Figure 3 The figure is a schematic diagram of a flow chart of an embodiment of the method of the present invention.
[0030] Figure 4 It is a schematic diagram of the process flow of the product embodiment of the present invention.
[0031] Figure 5 This is a schematic diagram of the product layout structure of the present invention Figure 1 .
[0032] Figure 6 This is a schematic diagram of the product layout structure of the present invention Figure 2 .
[0033] Figure 7 This is a schematic diagram of the product layout structure of the present invention Figure 3 .
[0034] The meanings of the various reference numerals / numbers are as follows.
[0035] 001-a first database, such as a resource library storing historical test data or a historical database of integrated testing; 011-the first requirement sample set, namely RS; 012-the second requirement sample set, i.e. φ; 030-Similarity score, i.e. Score (Q, d); 031-Query object, i.e. Q (Query); 032-The document to be matched, i.e. d (document); 033-positive integer serial number, i.e., i; 034- upper limit of the sequence number, i.e. n; 035-the weight of the i-th word of the query object, i.e. wi (weight-i); 036-The similarity between the i-th word of the query object and the document to be matched, that is, R(qi, d); 037-the i-th word of the query object, i.e. qi; 099-test case; 100-first requirement mapping step; 110-Current demand information; 111-Logical requirement information; 120-historical demand description information; 121-Logical description information; 200-second composite search step; 210-semantic retrieval step; 220-Logical search step; 300-third encoding recovery step; 333-Signal Matrix; 400-fourth sample learning step; 404-a fourth data processing model, such as a large language model LLM (Large Language Model); 410-prompt word; 420-natural language example, consisting of a first requirement sample set and a second requirement sample set; 430 —re-drawing the current demand information, that is, the current demand information is used as input in the fourth data processing model; 600-Integrated test device; 610-first demand mapping unit; 620-second composite search unit; 630-third encoding recovery unit; 640-Fourth Sample Learning Unit; 900 - Vehicles; 901-controller; 903-Computer storage media. DETAILED DESCRIPTION
[0036] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described below are only for explaining the technical solutions of the present invention, rather than for limiting the present invention. In addition, the parts described in the embodiments or the accompanying drawings are only examples of the relevant parts of the present invention, rather than the whole of the present invention.
[0037] like Figure 1 , Figure 3 The test information processing method shown includes a first requirement mapping step 100, a second compound retrieval step 200, and a third coding recovery step 300; the first requirement mapping step 100 converts the current requirement information 110 into logical requirement information 111, and converts the historical requirement description information 120 in the first database 001 into logical description information 121; in the second compound retrieval step 200, its retrieval step 210 retrieves the first requirement sample set 011, namely RS, from the historical requirement description information 120; the logical retrieval step 220 retrieves the second requirement sample set 012, namely φ, from the logical description information 121.
[0038] Furthermore, the third encoding recovery step 300 can convert the first requirement sample set 011 and the second requirement sample set 012 into natural language, and obtain the alternative test case 099 corresponding to the current requirement information 110 by comparing the elements of the first requirement sample set 011 and the second requirement sample set 012.
[0039] Among them, the first requirement mapping step 100 uses a semantic model to encode the current requirement information 110 and the historical requirement description information 120 and uses them for the semantic retrieval step 210; the logical requirement information 111 and the logical description information 121 are described in a logical language, which removes text information but retains logical symbols.
[0040] Specifically, the first requirement sample set 011, namely RS, is composed of RSi=<Ri,Ti> Ri is the i-th information of the first requirement, Ti is the i-th information of the first use case; the second requirement sample set 012, i.e., φ, is composed of φi=<r i,ti> Composition: ri is the i-th information of the second requirement, ti is the i-th information of the second use case, i=1 to N, and N is a positive integer.
[0041] Among them, the i-th information of the first requirement corresponds one-to-one with the i-th information of the first use case, and the i-th information of the second requirement corresponds one-to-one with the i-th information of the second use case; its third encoding recovery step 300 adopts a preset signal matrix 333 for natural language conversion, and the signal matrix 333 records the correspondence between the historical requirement description information 120 and the preset test signal.
[0042] Furthermore, the BGE asymmetric encoding and decoding structure may be used in the first requirement mapping step 100, and the semantic retrieval step 210 may use the Faiss similarity search library, and the first N elements of the first requirement sample set 011 may be obtained according to their similarity, where N is a positive integer.
[0043] Correspondingly, the semantic encoding model that can be adopted in its semantic retrieval step 210 can be the M3E model; its logical retrieval step 220 adopts the BM25 text matching algorithm, which can obtain the corresponding similarity score 030, i.e., Score (Q, d), of the query information Q and the document to be matched d through a weighted summation process, and obtain the first N elements of the second demand sample set 012 by comparing the similarity score 030, where N is a positive integer.
[0044] Furthermore, the test information processing method may also be provided with a fourth sample learning step 400; the first requirement sample set 011 and the second requirement sample set 012 may be combined using a few-shot learning process to obtain a natural language instance 420, and combined with a preset prompt word 410 and current requirement information 110, input into the fourth data processing model 404 to improve the test case 099.
[0045] Specifically, N=1 can be selected, and the fourth data processing model can adopt a language model, which can be a natural language processing model ChatGPT; its first database 001 can adopt a historical database of preset product tests.
[0046] like Figure 4 , Figure 1 The integrated testing device 600 shown adopts the same inventive concept, and its core components are provided with a first requirement mapping unit 610, a second compound retrieval unit 620, and a third coding recovery unit 630; the first requirement mapping unit 610 converts the current requirement information 110 into logical requirement information 111, and converts the historical requirement description information 120 in the first database 001 into logical description information 121; its second compound retrieval unit 620 retrieves the first requirement sample set 011 from the historical requirement description information 120, and retrieves the second requirement sample set 012 from the logical description information 121; the third coding recovery unit 630 converts the first requirement sample set 011 and the second requirement sample set 012 into natural language, and compares the elements of the first requirement sample set 011 and the second requirement sample set 012 to obtain the alternative test case 099 corresponding to the current requirement information 110.
[0047] The first requirement mapping unit 610 uses a semantic model to encode the current requirement information 110 and the historical requirement description information 120 for semantic retrieval; the logical requirement information 111 and the logical description information 121 are described in a logical language that removes text information but retains logical symbols.
[0048] Specifically, the first requirement sample set 011, namely RS, is composed of RSi=<Ri,Ti> Ri is the i-th information of the first requirement, Ti is the i-th information of the first use case; the second requirement sample set 012, i.e., φ, is composed of φi=<r i,ti> Composition: ri is the i-th information of the second requirement, ti is the i-th information of the second use case, i=1 to N, and N is a positive integer.
[0049] Among them, the i-th information of the first requirement corresponds one-to-one with the i-th information of the first use case, and the i-th information of the second requirement corresponds one-to-one with the i-th information of the second use case; the third coding recovery unit 630 uses the preset signal matrix 333 to perform natural language conversion, and the signal matrix 333 records the correspondence between the historical requirement description information 120 and the preset test signal.
[0050] Specifically, the first requirement mapping unit 610 may adopt a BGE asymmetric encoding and decoding structure, and the semantic retrieval may adopt the Faiss similarity search library, and the first N elements of the first requirement sample set 011 may be obtained according to the similarity, where N is a positive integer.
[0051] Similarly, the semantic encoding model that can be used for semantic retrieval can be the M3E model; the logical retrieval can use the BM25 text matching algorithm, and the query information Q and the document to be matched d are subjected to a weighted summation process to obtain the corresponding similarity score 030, that is, Score (Q, d), and the similarity score 030 is compared to obtain the first N elements of the second demand sample set 012, where N is a positive integer.
[0052] Furthermore, the integrated testing device 600 may also be provided with a fourth sample learning unit 640; the first requirement sample set 011 and the second requirement sample set 012 may be combined by a few-shot learning process to obtain a natural language instance 420, and combined with a preset prompt word 410 and current requirement information 110, input into a fourth data processing model 404 to improve the test case 099; wherein N=1, and the fourth data processing model may adopt a language model, which includes a natural language processing model ChatGPT; and its first database 001 is a historical database of preset product tests.
[0053] like Figure 5 , Figure 7 The computer storage medium 903 shown includes a storage medium body for storing a computer program; when the computer program is executed by a microprocessor, any of the above test information processing methods can be implemented; Figure 5 , Figure 6 The controller 901 includes any of the above integrated test devices 600 and / or computer storage media 903, which can be used to solve the same technical problem.
[0054] In practical applications, for example, when conducting integration testing, it is usually necessary to write corresponding test cases according to the test requirements. Research has found that if there is a lack of auxiliary tools, engineers or staff with less testing experience may spend more time to familiarize themselves with the specific process; they usually face three major challenges: First, database searches often only support precise matching of natural language texts, and their flexibility is limited; second, technologies based on natural language semantic search cannot distinguish between the logical part and the domain knowledge part of the input requirements, and the search results may be overly dependent on domain knowledge, resulting in failure to meet test requirements in common cold start scenarios (i.e. new functional requirements); third, even if similar historical test cases are found, they still need to be manually rewritten, which will also reduce work efficiency.
[0055] The present invention can quickly and efficiently select the most relevant test requirements and test case pairs from the historical test database, i.e., the first database 001, based on the natural language requirements proposed by the user, and automatically generate new test cases 099 in combination with the current test requirements, i.e., the current requirement information 110; based on this, it can not only significantly improve the efficiency of test engineers in writing test cases, but also improve the quality of test cases, further accelerate the progress of product development, such as vehicle 900, and ensure the stability and safety of various related systems of the product.
[0056] Specifically, the present invention can combine and give full play to the respective advantages of natural language NL (Natural Language) semantic retrieval and first order logic FOL (First Order Logic) matching, and discloses an information processing method that can be used for integrated test case generation.
[0057] Among them, in the natural language retrieval stage of requirements, the semantic model can be used to encode the natural language description input by the user (i.e., the current requirement information 110) and the historical requirement description in the database (historical requirement description information 120), and retrieve the first N requirement-test case sample pairs related to the user's problem (i.e., the first requirement sample set 011, namely RS).
[0058] Specifically, the retrieval process can return a set of requirement-test case sample pairs; denoted as RS, each RSi =<Ri,Ti> ÎRS; at the same time, according to the project-related signal matrix 333, the signal names in Ti can be converted into corresponding natural language descriptions; for example: "UMM_UsageModeSt" is replaced with "power status"; thereby preparing for the subsequent generation of test cases 099 or for the Few-Shot-Learn process.
[0059] Furthermore, in the FOL logic matching stage of the requirements, the present invention can convert the natural language description input by the user and the historical test requirement description in the database into a logical language, that is, obtain the logical requirement information 111; then, by removing the text part in the logical language and retaining only the logical symbols in the sentence, the pertinence of the information is improved; finally, the BM25 algorithm can be used to match the first N requirement-test case sample pairs in the database that are logically completely consistent with the user's question (that is, the second requirement sample set 012, that is, φ).
[0060] Specifically, the set of requirement-test case sample pairs can be retrieved and recorded as φ; each φi =<Ri, Ti> Îφ.
[0061] Furthermore, a large language model LLM (Large Language Model) can be used to generate test cases, and the 2N (i.e., the sum of the number of elements in the sets RS and φ) pairs of requirements and test cases in the database related to user requirements retrieved simultaneously by the above two methods are combined into examples D (Demonstrations); it is worth noting that: at this time, the signal names of the test cases in the database have been replaced (through the third encoding recovery step 300) with natural language that is easy for LLM to understand according to the signal matrix 333.
[0062] Specifically, different prompt words P (Prompt) and new requirement description Qu (Question) can be combined as input information of the large language model, and finally a test case 099 for the new requirement (ie, the current requirement information 110) is generated.
[0063] Among them, it can be achieved by combining semantic vector models, such as BAAI General Embedding and Facebook's Faiss natural language semantic retrieval: Among them, BGE is an Embedding model with powerful semantic capabilities, which adopts the RetroMAE model architecture, that is, the asymmetric Encoder-decoder structure.
[0064] Specifically, its Encoder part consists of Bert (12-layer Transformer Encoder), while the Decoder part only consists of one-layer Transformer Decoder.
[0065] During the training process, BGE first uses the RetroMAE pre-training algorithm on two large-scale corpora to enhance the semantic representation ability of the model; then, it enhances the judgment of semantic vectors through negative sampling and hard negative example mining; finally, it draws on the strategy of Instruction Tunning to enhance its general ability in multi-task scenarios.
[0066] Among them, Faiss is an open source similarity search library that can provide efficient similarity search for dense vectors and support searches of billions of vectors. When using Faiss, you need to create an index file based on the original vector and then query the index file. When creating an index file for the first time, it needs to go through two processes: training and adding. Subsequently, if new vectors need to be added to the index file, only one adding operation is required to achieve incremental index creation.
[0067] Specifically, in this embodiment, the BGE semantic coding model is first used to encode the historical demand descriptions in the database and store them offline; then, Faiss is used to create an index for the semantic coding vector so that it can be retrieved quickly and efficiently later; finally, when the user enters a natural language demand query, the semantically encoded user question can be used to recall the first N demand descriptions related to the user demand in the large-scale coding vector database with the index created.
[0068] At the same time, the signal names of the test cases in the database, i.e., the first database 001 (e.g., UMM_UsageModeSt), can be replaced with natural language descriptions that are easy for the large language model LLM to understand according to the signal matrix 333; for example, "UMM_UsageModeSt" can be replaced with "power status"; thereby preparing for the subsequent generation of test case 099 or for the Few-Shot-Learn process.
[0069] Among them, since the requirement description and the test case in the database are one-to-one corresponding, therefore, when the requirement description is retrieved, the first N test cases related to the user problem can also be obtained.
[0070] Furthermore, BM25 can be used for demand logic matching; unlike natural language, the first-order logic language FOL is built around objects and relations; the basic syntactic elements of FOL are symbols representing objects, relations and functions.
[0071] Specifically, FOL includes the constant C (Constant) symbol representing the object, the predicate Pr (Predicate) symbol representing the relationship, and the function F (Function) symbol representing the function.
[0072] Among them, considering that the results obtained from semantic matching often tend to ignore the potential logic in the requirements; by using FOL to perform a logical transformation on the requirements before searching, the search process will increase its ability to capture potential logic, and this process will also have a positive effect on the generation of test cases.
[0073] Taking BM25 as an example, this text matching algorithm mainly calculates the similarity score 030 between the query object 31, namely Q (Query) and the document collection; Figure 2 As shown, the similarity score 030 consists of three parts: first, the importance of each word in the Query; second, the relevance between each word in the Query and the document; and third, the relevance between each word in the Query and the Query itself.
[0074] Among them, Q (Query) represents a query object 031; qi represents the i-th word in Q; d represents the document to be matched; wi represents the weight of qi; R(qi,d) represents the similarity score between qi and d.
[0075] In this embodiment, the natural language requirement description input by the user, namely the current requirement information 110 and the historical test requirement description information 120 in the database, namely the first database 001 , are converted into a logical language using FOL.
[0076] Specifically, if the natural language of the demand description, i.e., the current demand information 110, is: "If the vehicle's current gear is not in the parking gear, and the vehicle speed is less than or equal to a certain value (10km / h), then when it is determined that the driver has left, the vehicle 900 should enter the parking state"; then the logical language after conversion using FOL is: "(current gear ≠ parking gear) ∧ (vehicle speed ≤ 10km / h) → (the vehicle enters the parking state)".
[0077] Among them, in order to ensure the coverage of test cases generated by subsequent processing, the result after the requirement is negated can also be supplemented, that is, "(current gear = parking gear) ∨ (vehicle speed>10km / h)→(vehicle has not entered the parking state)".
[0078] Furthermore, the text part of the logic language after FOL conversion can be removed, and only the logical symbols in the sentence are retained; as mentioned above, "(current gear position ≠ parking gear) ∧ (vehicle speed ≤ 10km / h) → (vehicle enters the parking state)" and "(current gear position = parking gear) ∨ (vehicle speed > 10km / h) → (vehicle does not enter the parking state)" are obtained; and they can be converted into "(≠) ∧ (≤) -> ()" and "(=) ∨ (>) -> ()" respectively.
[0079] Furthermore, the BM25 algorithm can be used to match the first N historical requirement descriptions related to user problems in the database; since the test requirements and test cases in the database correspond one-to-one, when the requirement description is retrieved, the corresponding test case description information is also obtained.
[0080] Furthermore, the large language model LLM can be used to perform Few-Shot-Learn to generate test cases; combined with the above search results, optimization is continued based on 2N pairs of requirements and test cases related to user problems; it is worth noting that at this time, the signal names of the test cases in the database have been replaced with natural language that is easy for LLM to understand according to the signal matrix 333; next, the above results can be put into the large language model LLM for Few-Shot-Learn; its large language model uses qwen-1.5-72b.
[0081] Specifically, the final content input to the large language model includes three parts; the first is the prompt word 410, namely Prompt; since the present embodiment aims to automatically generate test cases, the prompt can be set to: "You are an automotive test engineer, and now you are required to write the corresponding test case according to the requirement description according to the following example"; the second is the natural language instance 420, namely Demonstrations, which is the example provided to the large model.
[0082] Demonstrations consists of 2N pairs of requirements and test cases related to user problems retrieved according to N and FOL in the previous two sections. In this embodiment, N=1, so only two pairs of requirements and test case samples are input into the large model. They can be input in the format of "requirement description: xxx \n test case: xxx".
[0083] As mentioned above, the third is to reiterate 430 the current requirement information, namely Question, or "new requirement description"; this description is the user question (current requirement information 110) used for retrieval in the previous two sections, that is, the requirement description for ultimately generating a test case.
[0084] Specifically, in this embodiment, the Question part is written in the format of "new requirement description: xxx"; the above three parts are three character strings, which are combined together and input into the large language model together to eventually generate test case 099 for "new requirement".
[0085] Among them, when matching queries, the optional semantic encoding model can also be M3E (Moke Massive Mixed Embedding); the large language model LLM used to generate test cases can also be the natural language processing model ChatGPT or other models.
[0086] In summary, the inventive concept and beneficial effects of this embodiment include: combining NL semantic retrieval and FOL logical matching to realize requirement mapping and compound retrieval of test cases; and then combining the semantic retrieval method of BGE and Faiss and the requirement logic matching process of BM25 to provide data to the Few-Shot-Learn process of the large language model and optimize the processing process of test case 099.
[0087] It should be noted that the above embodiments are only for more clearly illustrating the technical solutions of the present invention. Those skilled in the art will understand that the implementation methods of the present invention are not limited to the above contents, and obvious changes, replacements or substitutions based on the above contents do not exceed the scope of the technical solutions of the present invention; other implementation methods will also fall within the scope of the present invention without departing from the concept of the present invention.
Claims
1. A test information processing method, characterized in that The invention comprises a first requirement mapping step (100), a second composite retrieval step (200), and a third coding recovery step (300); wherein: the first requirement mapping step (100) converts the current requirement information (110) into logical requirement information (111), and converts the historical requirement description information (120) in the first database (001) into logical description information (121); the second composite retrieval step (200) comprises a semantic retrieval step (210) and a logical retrieval step (220), wherein the semantic retrieval step (210) converts the historical requirement description information (120) into logical description information (121); The first requirement sample set (011) RS is retrieved from the information (120); the logic retrieval step (220) retrieves the second requirement sample set (012) φ from the logic description information (121); the third encoding recovery step (300) converts the first requirement sample set (011) RS and the second requirement sample set (012) φ into natural language, and compares the elements of the first requirement sample set (011) RS and the second requirement sample set (012) φ to obtain the candidate test cases (099) of the current requirement information (110).
2. The test information processing method according to claim 1, wherein: The first requirement mapping step (100) uses a semantic model to encode the current requirement information (110) and the historical requirement description information (120) and use them in the semantic retrieval step (210); the logical requirement information (111) and the logical description information (121) are described using a logical language, which removes text information but retains logical symbols.
3. The test information processing method according to claim 1 or 2, wherein: The first requirement sample set (011) RS is composed of RSi=<Ri,Ti> Ri is the i-th information of the first requirement, Ti is the i-th information of the first use case; the second requirement sample set (012) φ is composed of φi=<r i,ti> Composition: ri is the i-th information of the second requirement, ti is the i-th information of the second use case, i=1 to N, and N is a positive integer.
4. The test information processing method according to claim 3, wherein: The i-th information of the first requirement corresponds one-to-one to the i-th information of the first use case, and the i-th information of the second requirement corresponds one-to-one to the i-th information of the second use case; the third encoding recovery step (300) uses a preset signal matrix (333) to convert the natural language, and the signal matrix (333) records the correspondence between the historical requirement description information (120) and the preset test signal.
5. The test information processing method according to any one of claims 1, 2 or 4, wherein: The first requirement mapping step (100) uses a BGE asymmetric encoding and decoding structure, the semantic retrieval step (210) uses a Faiss similarity search library, and obtains the first N elements of the first requirement sample set (011) RS based on the similarity, where N is a positive integer.
6. The test information processing method according to claim 5, wherein: The semantic encoding model adopted in the semantic retrieval step (210) includes the M3E model; the logical retrieval step (220) adopts the BM25 text matching algorithm, obtains the corresponding similarity score (030), i.e., Score (Q, d), of the query information Q and the document to be matched d through a weighted sum process, and compares the similarity score (030) to obtain the first N elements of the second demand sample set (012) φ, where N is a positive integer.
7. The test information processing method as described in any one of claims 1, 2, 4 or 6, further comprising a fourth sample learning step (400); the fourth sample learning step (400) uses a few-shot learning process to combine the first requirement sample set (011) RS and the second requirement sample set (012) φ to obtain a natural language instance (420), and combines it with a preset prompt word (410) and the current requirement information (110), and inputs it into a fourth data processing model (404) to improve the test case (099).
8. The test information processing method according to claim 7, wherein: N=1, and the fourth data processing model includes a language model, and the language model includes a natural language processing model; the first database (001) is a historical database of preset product tests.
9. An integrated test device (600), comprising a first requirement mapping unit (610), a second composite retrieval unit (620), and a third code recovery unit (630); wherein: The first requirement mapping unit (610) converts the current requirement information (110) into logical requirement information (111), and converts the historical requirement description information (120) in the first database (001) into logical description information (121); the second composite retrieval unit (620) retrieves a first requirement sample set (011) RS from the historical requirement description information (120), and retrieves a second requirement sample set (012) φ from the logical description information (121); the third encoding recovery unit (630) converts the first requirement sample set (011) RS and the second requirement sample set (012) φ into natural language, and compares the elements of the first requirement sample set (011) RS and the second requirement sample set (012) φ to obtain a test case (099).
10. The integrated test device (600) as claimed in claim 9, wherein: The first requirement mapping unit (610) uses a semantic model to encode the current requirement information (110) and the historical requirement description information (120) for semantic retrieval; the logical requirement information (111) and the logical description information (121) are described using a logical language, which removes text information but retains logical symbols.
11. The integrated test device (600) according to claim 9 or 10, wherein: The first requirement sample set (011) RS is composed of RSi=<Ri,Ti> Ri is the i-th information of the first requirement, Ti is the i-th information of the first use case; the second requirement sample set (012) φ is composed of φi=<r i,ti> Composition: ri is the i-th information of the second requirement, ti is the i-th information of the second use case, i=1 to N, and N is a positive integer.
12. The integrated test device (600) according to claim 11, wherein: The i-th information of the first requirement corresponds one-to-one to the i-th information of the first use case, and the i-th information of the second requirement corresponds one-to-one to the i-th information of the second use case; the third coding recovery unit (630) uses a preset signal matrix (333) to convert the natural language, and the signal matrix (333) records the correspondence between the historical requirement description information (120) and the preset test signal.
13. The integrated test device (600) according to any one of claims 9, 10 or 12, wherein: The first requirement mapping unit (610) adopts a BGE asymmetric encoding and decoding structure, the semantic retrieval step (210) adopts a Faiss similarity search library, and obtains the first N elements of the first requirement sample set (011) RS according to the similarity, where N is a positive integer.
14. The integrated test device (600) according to claim 13, wherein: The semantic encoding model used in the semantic retrieval includes the M3E model; the logical retrieval uses the BM25 text matching algorithm, and obtains the corresponding similarity score (030), i.e., Score (Q, d), from the query information Q and the document to be matched d through a weighted sum process, and compares the similarity score (030) to obtain the first N elements of the second demand sample set (012) φ, where N is a positive integer.
15. The integrated test device (600) according to any one of claims 9, 10, 12 or 14, further comprising a fourth sample learning unit (640); the fourth sample learning unit (640) uses a few-shot learning process to combine the first requirement sample set (011) RS and the second requirement sample set (012) φ to obtain a natural language instance (420), and combines the preset prompt word (410) and the current requirement information (110) to input into a fourth data processing model (404) to improve the test case (099); wherein, N=1, and the fourth data processing model includes a language model, and the language model includes a natural language processing model ChatGPT, qwen-1.5-72b; the first database (001) is a historical database of preset product tests.
16. A computer storage medium (903), comprising a storage medium body for storing a computer program; when the computer program is executed by a microprocessor, the test information processing method according to any one of claims 1 to 8 is implemented.
17. A controller (901), comprising the integrated test device (600) according to any one of claims 9 to 15 and / or the computer storage medium (903) according to claim 16.
