Complex test task process planning method based on scene model

Abstract

The invention discloses a complex test task process planning method based on a scene model, and belongs to the technical field of testing. According to the technology provided by the invention, a method for manually constructing an instrument test process, establishing a traditional structured database, autonomously planning the instrument test process based on knowledge reasoning and the like in the previous technical scheme are abandoned, a complex test task process planning technology based on a scene model is designed, and the advantages of a large model are fully utilized; and the disadvantage of insufficient knowledge depth in the professional field is made up, so that compared with the prior art, the method is more favorable for coping with instrument test scheme planning under complex test tasks, and has higher reliability.

Description

Technical Field

[0001] The present invention belongs to the field of testing technology, and in particular relates to a complex testing task process planning method based on a scenario model. Background Art

[0002] Instrument testing technology plays an important role in the development of modern science and technology industry. With the rapid improvement of industrial technology level, the architecture of test system is also developing in the direction of complexity. However, complex architecture design also makes system testing very complicated. A test may assign different test tasks to the same or different execution instruments. Multiple different test tasks work together to complete a common test goal according to a certain timing or even more complex conditions. In such a complex test background, it is difficult to complete the test task with high quality by relying solely on manual operation or simple automatic test planning system. Therefore, it is urgent to study a complex test task process planning technology based on scenario model. By combining domain expert knowledge with large language model, a special scenario model is generated to complete the analysis and planning of complex test tasks and realize autonomous and reliable testing of instruments under complex tasks.

[0003] The method described in the patent "A method and platform for autonomous planning of instrument test processes based on knowledge reasoning" (application number: CN202311857365.1) includes the following steps: S1-acquisition of instrument test process knowledge, S2-construction of instrument test knowledge graph, and S3-intelligent judgment of test processes based on autonomous rules. The platform is used for steps S1 to S3 in the method, and finally outputs the instrument test judgment result. The method for autonomous planning of instrument test processes based on knowledge reasoning and the platform described in the present invention are applied to instrument performance testing, and provide an instrument test process planning solution under highly explainable conditions.

[0004] The method involved in the patent "A system reliability testing method, device, computer equipment and storage medium" (application number: CN202210731921.X) mainly includes: the system obtains test information corresponding to the test instructions, including test data and test tasks; obtains the test environment, test process and preset test result reference files corresponding to the test tasks, and system data corresponding to the test environment; analyzes based on the test data and system data to determine the effective test elements that meet the system reliability testing requirements; tests the system based on the effective test elements, test environment and test process.

[0005] The shortcomings of the existing technology are mainly reflected in the following aspects: First, most of the current instrument testing methods are usually manually operated by staff according to test scripts, resulting in a low level of intelligent instrument testing and low detection efficiency. In addition, due to the uneven levels of instrument testing personnel, the instrument testing process is relatively complex and changeable, resulting in non-standard instrument testing procedures and poor reliability. Second, there is a lack of process autonomous planning technology for complex testing tasks. Complex testing tasks may cover multiple subtasks, and there are intricate logical relationships between subtasks. Improper analysis may lead to task conflicts during execution, which ultimately leads to the failure of test task execution. Summary of the invention

[0006] In view of the above-mentioned technical problems existing in the prior art, the present invention proposes a complex test task process planning method based on a scenario model, which has a reasonable design, overcomes the shortcomings of the prior art, and has good effects.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A method for planning a complex test task process based on a scenario model comprises the following steps:

[0009] Step 1: Build an expert experience database;

[0010] Step 2: Build a domain scenario model;

[0011] Use the domain expert experience library built in step 1 to empower the traditional baseline large model and fine-tune the large model to build the domain scenario model mentioned in step 2;

[0012] Step 3: Complex task decomposition and process planning;

[0013] Using the domain scenario model constructed in step 2, perform semantic understanding according to the assigned test tasks, complete the complex task planning and decomposition in step 3, decompose the complex tasks into a set of meta-tasks, analyze the relationship between the meta-tasks, and plan and sort the meta-tasks;

[0014] Step 4: Generate a visual test plan.

[0015] Preferably, step 1 specifically includes the following steps:

[0016] Step 1.1: Obtain manuals for various types and models of instruments and equipment;

[0017] Step 1.2: Knowledge extraction, constructing a basic knowledge base of instrument performance and index detection;

[0018] Step 1.3: Using prior knowledge and according to the characteristics of the test task, decompose the complex task into multiple subtasks, and mark the relationship between each subtask according to the three basic relationships of serial collaboration, parallel collaboration, and conditional collaboration. In this way, build an expert experience library in the test field, generate knowledge data, and provide a knowledge source for the scenario model.

[0019] Preferably, step 2 specifically includes the following steps:

[0020] Step 2.1: Use the traditional large model as the baseline model and use the natural semantic processing advantages of the large model to extract key information;

[0021] Step 2.2: Establish a data channel between the expert experience database and the baseline large model. Use the knowledge enhancement method to query and call knowledge in the expert experience database for specific test tasks, thereby obtaining knowledge feedback from the expert experience database to form supplementary knowledge. The test knowledge obtained by feedback can be used to fine-tune the baseline large model.

[0022] Step 2.3: Complete the construction of the domain scenario model.

[0023] Preferably, step 3 specifically includes the following steps:

[0024] Step 3.1: Complex test task input;

[0025] Through human-computer interaction, complex test tasks are input in the form of natural language scripts; a large language model is used as the baseline model to understand the intent of the test task;

[0026] Step 3.2: Complex task planning and decomposition;

[0027] According to the analysis of the test task, the test intention is obtained, and the knowledge sequence under the specified test task is output in combination with the scenario model of the expert experience library, as shown in formula (1):

[0028] S(i)={S1,S2,...,S i ,...,S n} (1);

[0029] Among them, S(i) represents the i-th test task; S i Represents the meta-task under the test task;

[0030] Step 3.3: After obtaining the meta-task sequence, intelligently analyze the relationship between each meta-task, namely the three basic relationships of serial coordination, parallel coordination, and conditional coordination, and use this to complete the sorting of each meta-task, generate a task planning list, realize complex test task planning, and ensure reliable execution of instrument testing.

[0031] Preferably, step 4 specifically includes the following contents:

[0032] Build an autonomous planning platform for complex instrument test tasks. The platform integrates functional modules including human-computer interaction and task assignment, task analysis and understanding, and test task process planning. It realizes autonomous planning of instrument test processes through steps including human-computer operation, task intent understanding, and meta-task sequence generation.

[0033] Beneficial technical effects brought by the present invention:

[0034] The technology proposed in the present invention abandons the previous technical solutions of manually constructing instrument test processes, establishing traditional structured databases, and autonomous planning methods of instrument test processes based on knowledge reasoning, and designs a complex test task process planning technology based on scenario models. It fully utilizes the advantages of large models and makes up for the disadvantage of insufficient depth of professional knowledge, making it better at coping with instrument test plan planning under complex test tasks than existing technologies, and has higher reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Planning a flow chart for the complex test task flow scheme of the present invention;

[0036] Figure 2 Construct method flow chart for instrument testing expert experience base;

[0037] Figure 3 This is a flow chart of the construction method of the domain scenario model;

[0038] Figure 4 This is a schematic diagram of the test process planning based on the domain scenario model. DETAILED DESCRIPTION

[0039] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0040] The process of the technical solution of the present invention is as follows Figure 1 As shown, the specific implementation is described as follows:

[0041] (1) Building an expert experience database

[0042] Constructing an expert experience database is a key technology in the present invention. The present invention obtains manuals for instruments and equipment of various types and models, and on this basis, performs knowledge extraction to construct a basic knowledge base for instrument performance and index detection. At the same time, using prior knowledge, according to the characteristics of the test task, the complex task is decomposed into multiple subtasks, and the relationship between each subtask is labeled according to the three basic relationships of serial collaboration, parallel collaboration, and conditional collaboration, so as to construct an expert experience database in the test field, and the generated knowledge data provides the knowledge source for the scenario model. The specific construction method is as follows: Figure 2 shown.

[0043] Depend on Figure 2 It can be seen that the knowledge sources of the expert experience base include instrument sample manuals (unstructured data, semi-structured data, structured data) and complex instrument test examples. The former is mainly used to build basic knowledge in the expert experience base, such as certain attributes and indicators of instruments, while the latter is mainly used to build advanced knowledge in the expert experience base, such as test task process composition, test task relationship, task type, complex task and meta-task inclusion relationship, etc.

[0044] Based on this, the expert experience base knowledge data constructed by the present invention should include two parts: one is the test domain knowledge: a detailed description of the instrument test domain knowledge, which covers the basic information of the type of instrument; the other is the test task knowledge: a description of the test objectives to be achieved by a task, and how to achieve the established objectives of the task by decomposing complex tasks into meta-tasks and through the orderly combination and execution of meta-tasks.

[0045] (2) Building a domain scenario model

[0046] In order to solve the technical problem that traditional large models show certain rigidity and limitations when facing multiple objectives, multiple constraints and multiple priorities, the present invention proposes to construct a domain scenario model in combination with the expert experience library, use the traditional open source large model as the baseline model, and use the expert experience library as the knowledge engine to perform targeted fine-tuning on the baseline model to generate a scenario model for the special field of instrument testing.

[0047] like Figure 3 As shown, the process of constructing the domain scene model proposed in the present invention is briefly described as follows:

[0048] First, the traditional large model is used as the baseline model. The natural semantic processing advantage of the large model can be used to effectively extract key information, such as event extraction, topic mining, and test intent identification. Secondly, a data channel between the expert experience library and the baseline large model is established. The knowledge enhancement method is used to query and call knowledge in the expert experience library for specific test tasks, thereby obtaining knowledge feedback from the expert experience library to form supplementary knowledge. The test knowledge obtained by feedback can be used to fine-tune the baseline large model. Finally, on the basis of the above, the construction of the domain scenario model is completed.

[0049] (3) Complex task decomposition and process planning

[0050] The purpose of test task planning is to decompose complex tasks into multiple logical meta-tasks that can be completed by calling existing tools in a single step. Complex tasks are tasks that the model cannot call the tool only once or generate results directly, and meta-tasks are tasks that can be completed by calling the tool once.

[0051] like Figure 4 As shown, the present invention proposes an autonomous instrument test process planning method based on a scenario model, which uses the scenario model to intelligently analyze and decompose the complex test tasks input by the system, plan and decompose them into a sequence of meta-tasks, and sort them according to the basic relationship between the meta-tasks, thereby realizing the process planning of complex test tasks. The specific process is summarized as follows:

[0052] 1) Complex test task input

[0053] Through human-computer interaction, complex test tasks are input in the form of natural language scripts. The present invention uses a large language model as a baseline model. It can quickly mine valuable information from it, analyze and interpret test tasks more comprehensively, and understand the intent of test tasks.

[0054] 2) Complex task planning and decomposition

[0055] According to the analysis of the test task, the test intent is obtained. The scenario model of the expert experience library is combined to output the knowledge sequence under the specified test task, as shown in formula (1):

[0056] S(i)={S1,S2,...,S i ,...,S n} (1);

[0057] Among them, S(i) represents the i-th test task; S i Represents the meta-task under the test task;

[0058] After obtaining the meta-task sequence (arranged in disorder), the relationship between each meta-task is intelligently analyzed, namely the three basic relationships of serial coordination, parallel coordination, and conditional coordination, and the order of each meta-task is completed to generate a task planning list. Complex test task planning is realized, thereby ensuring the reliable execution of instrument testing.

[0059] (4) Generate a visual test plan

[0060] On the basis of steps 1, 2, and 3, an autonomous planning platform for complex instrument test tasks is developed. The platform integrates functional modules such as human-computer interaction and task assignment, task analysis and understanding, and test task process planning. The domain expert experience library built in step 1 is used to empower the traditional baseline large model, and the large model is fine-tuned to construct the domain scenario model mentioned in step 2. The model performs semantic understanding based on the assigned test tasks and completes the complex task planning and decomposition described in step 3. After decomposing the complex task into a set of meta-tasks, the relationship between the meta-tasks is analyzed, and the meta-tasks are planned and sorted to generate the final executable test plan. Finally, an autonomous planning platform for complex instrument test tasks is constructed, which integrates various technical functional modules, realizes autonomous planning of instrument test processes through steps such as human-computer operation, task intent understanding, and meta-task sequence generation, and promotes the autonomous and intelligent development of instrument testing.

[0061] The key points and protection points of the present invention application include:

[0062] The expert experience database data applied for in the present invention is derived from user manuals of various types of instruments and work experience knowledge.

[0063] The present invention applies to design and construct an expert experience library to empower the traditional large language model and provide a source of model knowledge.

[0064] The present invention applies to design a technical solution for complex test task process planning based on scenario model, utilizes expert experience library to fine-tune large language model, thereby forming scenario model, realizing task analysis and understanding under complex test tasks, and autonomous planning of test tasks.

[0065] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for planning a complex test task process based on a scenario model, characterized in that: The steps include: Step 1: Build an expert experience database; Step 2: Build a domain scenario model; Use the domain expert experience library built in step 1 to empower the traditional baseline large model and fine-tune the large model to build the domain scenario model mentioned in step 2; Step 3: Complex task decomposition and process planning; Using the domain scenario model constructed in step 2, perform semantic understanding according to the assigned test tasks, complete the complex task planning and decomposition in step 3, decompose the complex tasks into a set of meta-tasks, analyze the relationship between the meta-tasks, and plan and sort the meta-tasks; Step 4: Generate a visual test plan.

2. The method for planning a complex test task process based on a scenario model according to claim 1, characterized in that: Step 1 specifically includes the following steps: Step 1.1: Obtain manuals for various types and models of instruments and equipment; Step 1.2: Knowledge extraction, constructing a basic knowledge base of instrument performance and index detection; Step 1.3: Using prior knowledge and according to the characteristics of the test task, decompose the complex task into multiple subtasks, and mark the relationship between each subtask according to the three basic relationships of serial collaboration, parallel collaboration, and conditional collaboration. In this way, build an expert experience library in the test field, generate knowledge data, and provide a knowledge source for the scenario model.

3. The method for planning a complex test task process based on a scenario model according to claim 1, characterized in that: Step 2 specifically includes the following steps: Step 2.1: Use the traditional large model as the baseline model and use the natural semantic processing advantages of the large model to extract key information; Step 2.2: Establish a data channel between the expert experience database and the baseline large model. Use the knowledge enhancement method to query and call knowledge in the expert experience database for specific test tasks, thereby obtaining knowledge feedback from the expert experience database to form supplementary knowledge. The test knowledge obtained by feedback can be used to fine-tune the baseline large model. Step 2.3: Complete the construction of the domain scenario model.

4. The method for planning a complex test task process based on a scenario model according to claim 1, characterized in that: Step 3 specifically includes the following steps: Step 3.1: Complex test task input; Through human-computer interaction, complex test tasks are input in the form of natural language scripts; a large language model is used as the baseline model to understand the intent of the test task; Step 3.2: Complex task planning and decomposition; According to the analysis of the test task, the test intention is obtained, and the knowledge sequence under the specified test task is output in combination with the scenario model of the expert experience library, as shown in formula (1): S(i)={s1,s2,...,s i ,...,s n } (1); Among them, S(i) represents the i-th test task; s i Represents the meta-task under the test task; Step 3.3: After obtaining the meta-task sequence, intelligently analyze the relationship between each meta-task, namely the three basic relationships of serial coordination, parallel coordination, and conditional coordination, and use this to complete the sorting of each meta-task, generate a task planning list, realize complex test task planning, and ensure reliable execution of instrument testing.

5. The method for planning a complex test task process based on a scenario model according to claim 1, characterized in that: Step 4 specifically includes the following: Build an autonomous planning platform for complex instrument test tasks. The platform integrates functional modules including human-computer interaction and task assignment, task analysis and understanding, and test task process planning. It realizes autonomous planning of instrument test processes through steps including human-computer operation, task intent understanding, and meta-task sequence generation.

Images