Test method and device, test equipment, network equipment, medium and program product

Abstract

The invention provides a testing method and device, testing equipment, network equipment, a medium and a program product, and is applied to the technical field of testing. The method comprises the following steps: obtaining test requirement information of a to-be-tested scheme, and obtaining test process information obtained by executing the to-be-tested scheme by test execution equipment; analyzing the test process information according to the test requirement information by adopting a multi-modal large model to obtain a test intermediate execution instruction; and sending the test intermediate execution instruction to the test execution equipment for further test execution of the to-be-tested scheme. By adopting the method, the test process analysis in the multi-modal test process is realized by utilizing the analysis and reasoning capability of the multi-modal large model on the multi-modal information, and the test intermediate execution instruction for guiding the test execution is generated according to the test requirement information and is used for executing the subsequent test steps, so that the multi-modal test efficiency is improved under the multi-modal test scene. Automation of the interaction process is tested.

Description

Technical Field

[0001] The present application relates to the field of testing technology, and in particular to a testing method, device, testing equipment, network equipment, medium and program product. Background Art

[0002] With the rapid development of AI capabilities, the demand for testing in multiple dimensions such as function, performance, and stability is increasing in the fields of software and hardware, AI, digital humans, robots, and intelligent agents. General testing tools mainly use robotic arms, automated testing software, or test scripts to carry out functional performance testing for specific processes and indicators. However, with the emergence of AI capabilities brought by large models, various automated testing frameworks and tools have become difficult to support the increasingly complex product interaction processes and multimodal application scenario testing due to their single functions. The intelligence of the test cannot match the intelligence of the product, and it is impossible to achieve automation of the test interaction process in various types of multimodal test scenarios such as terminal applications, robots, and intelligent agents. Summary of the invention

[0003] The purpose of the technical solution of this application is to provide a testing method, apparatus, testing equipment, network equipment, medium and program product, which are used to solve the problem that the existing testing methods cannot realize the automation of the test interaction process in various types of multimodal testing scenarios such as terminal applications, robots and intelligent agents.

[0004] One embodiment of the present application provides a testing method, which includes:

[0005] Acquire test requirement information of the scheme to be tested, and acquire test process information obtained by the test execution device executing the scheme to be tested;

[0006] Adopting a multi-modal large model, analyzing the test process information according to the test requirement information, and obtaining the test intermediate execution instructions;

[0007] The test intermediate execution instruction is sent to the test execution device for further test execution of the to-be-tested solution.

[0008] Optionally, in the test method, the test requirement information includes test rules and / or test indicators described in natural language and / or in the form of pictures, and the method further includes:

[0009] The multimodal large model is used to analyze the test rules and / or test indicators to obtain test guidance information, and the test guidance information is used to generate an execution script for the scheme to be tested, and the execution script is used by the test execution device to execute the scheme to be tested.

[0010] Optionally, in the test method, the test guidance information includes one or more of a test indicator reference value, a test networking mode, a test step and script-related information.

[0011] Optionally, the testing method further comprises:

[0012] Obtaining test result information of the test execution device executing the scheme to be tested;

[0013] The multimodal large model is used to analyze the test result information according to the test requirement information to obtain a test report for the to-be-tested solution.

[0014] Optionally, in the test method, the test process information includes one or more of the current state of the test execution tool, the current state of the product under test and test intermediate result information.

[0015] Optionally, in the test method, the test requirement information includes one or more of the following:

[0016] Test plan information;

[0017] Test interaction rules;

[0018] Test indicators;

[0019] Test judging criteria;

[0020] Data analysis rules;

[0021] Data analysis process;

[0022] Test execution uses prompt words.

[0023] Optionally, the testing method further comprises:

[0024] Obtain test data sets and model adjustment strategies for multiple test application scenarios;

[0025] The pre-trained initial model is trained using the test data set and the model adjustment strategy to obtain the multimodal large model.

[0026] Optionally, in the testing method, the model adjustment strategy includes one or more of instruction fine-tuning, loss function adjustment, transfer learning strategy and multimodal fusion strategy.

[0027] Optionally, the test method, wherein obtaining a test data set in a plurality of test application scenarios, comprises:

[0028] Acquire application data included in the to-be-tested solutions in multiple test application scenarios; the application data includes one or more of application components, icon images, and text data;

[0029] The data to be tested in the application data is marked to obtain the test data set.

[0030] One embodiment of the present application further provides a testing method, which includes:

[0031] During the test process of the scheme to be tested, the test process information of the scheme to be tested is sent to the test analysis device;

[0032] Obtaining a test intermediate execution instruction sent by the test analysis device, wherein the test intermediate execution instruction is obtained by analyzing the test process information according to the test requirement information of the scheme to be tested using a multimodal large model;

[0033] According to the test intermediate execution instruction, continue to execute the test of the to-be-tested solution.

[0034] Optionally, the testing method further comprises:

[0035] Obtaining test result information after testing the scheme to be tested;

[0036] The test result information is transmitted to the test analysis device.

[0037] Optionally, in the test method, the test process information includes one or more of the current state of the test execution tool, the current state of the product under test and test intermediate result information.

[0038] One embodiment of the present application further provides a test device, which includes a processor and a transceiver, wherein:

[0039] The processor is used to obtain test requirement information of the scheme to be tested, and to obtain test process information obtained by the test execution device executing the scheme to be tested; and, using a multi-modal large model, analyze the test process information according to the test requirement information to obtain a test intermediate execution instruction;

[0040] The transceiver is used to send the test intermediate execution instruction to the test execution device for further test execution of the to-be-tested solution.

[0041] One embodiment of the present application further provides a test device, which includes a transceiver and a processor, wherein:

[0042] The transceiver is used to send the test process information of the scheme to be tested to the test analysis device during the test process of the scheme to be tested; and obtain the test intermediate execution instruction sent by the test analysis device, wherein the test intermediate execution instruction is obtained by analyzing the test process information according to the test requirement information of the scheme to be tested using a multi-modal large model;

[0043] The processor is used to continue to execute the test of the to-be-tested solution according to the test intermediate execution instruction.

[0044] One embodiment of the present application further provides a testing device, which includes:

[0045] An information acquisition module, used to acquire test requirement information of the scheme to be tested, and to acquire test process information obtained by the test execution device executing the scheme to be tested;

[0046] An analysis module, used to analyze the test process information according to the test requirement information by using a multi-modal large model, and obtain a test intermediate execution instruction;

[0047] The first sending module is used to send the test intermediate execution instruction to the test execution device for further test execution of the to-be-tested solution.

[0048] One embodiment of the present application further provides a testing device, which includes:

[0049] A second sending module is used to send the test process information of the scheme to be tested to the test analysis device during the test process of the scheme to be tested;

[0050] An acquisition module, used to obtain the test intermediate execution instruction sent by the test analysis device, wherein the test intermediate execution instruction is obtained by analyzing the test process information according to the test requirement information of the scheme to be tested using a multimodal large model;

[0051] The testing module is used to continue to execute the test of the to-be-tested solution according to the test intermediate execution instruction.

[0052] One embodiment of the present application further provides a network device, which includes a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program implements any of the above test methods when executed by the processor.

[0053] One embodiment of the present application further provides a readable storage medium, wherein a program is stored on the readable storage medium, and when the program is executed by a processor, the steps in any of the above test methods are implemented.

[0054] One embodiment of the present application further provides a computer program product, which includes computer instructions, and when the computer instructions are executed by a processor, the steps in any of the above test methods are implemented.

[0055] At least one of the above technical solutions of the present application has the following beneficial effects:

[0056] The testing method described in the embodiment of the present application utilizes the analysis and reasoning capabilities of a multimodal large model for multimodal information, and can simultaneously analyze multimodal data such as text, images, and videos generated synchronously during the testing process, thereby realizing test process analysis during the multimodal testing process, and generating test intermediate execution instructions to guide test execution based on test requirement information, which are used for subsequent test step execution, so as to realize automation of the test interaction process in a multimodal testing scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a flow chart of the test method described in Example 1 of the present application;

[0058] Figure 2 A schematic diagram of a system architecture using the method described in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of a flow chart of one implementation of the test method described in the examples of the present application;

[0060] Figure 4 This is a flow chart of the testing method described in Example 2 of the present application;

[0061] Figure 5 This is a schematic diagram of the structure of the test equipment described in Example 1 of the present application;

[0062] Figure 6 This is a schematic diagram of the structure of the test equipment described in Example 2 of the present application;

[0063] Figure 7 This is a schematic diagram of the structure of the testing device described in Example 1 of the present application;

[0064] Figure 8 This is a schematic diagram of the structure of the testing device described in Example 2 of the present application. DETAILED DESCRIPTION

[0065] In order to make the technical problems, technical solutions and advantages to be solved by the present application clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0066] In order to solve the problem that the existing testing methods are unable to realize the automation of the test interaction process in various types of multimodal testing scenarios such as terminal applications, robots and intelligent agents, the embodiment of the present application provides a testing method, which utilizes the analysis and reasoning capabilities of a multimodal large model for multimodal information, and can simultaneously analyze multimodal data such as text, pictures, videos, etc. generated synchronously during the test process, realize the test process analysis during the multimodal test process, and generate test intermediate execution instructions to guide the test execution according to the test requirement information, which are used for the execution of subsequent test steps, so as to realize the automation of the test interaction process in the multimodal testing scenario.

[0067] One embodiment of the present application provides a testing method, such as Figure 1 As shown, the method includes:

[0068] S110, obtaining test requirement information of the scheme to be tested, and obtaining test process information obtained by the test execution device executing the scheme to be tested;

[0069] S120, using a multi-modal large model, analyzing the test process information according to the test requirement information to obtain a test intermediate execution instruction;

[0070] S130, sending the test intermediate execution instruction to the test execution device for further test execution of the to-be-tested solution.

[0071] In the embodiment of the present application, the multimodal large model is a model that jointly trains multimodal information such as text, image, video and audio, that is, the obtained multimodal large model can jointly analyze multimodal information such as text, image, video and audio. In the test method described in the embodiment of the present application, the multimodal large model can analyze multimodal information such as text, image, video and audio during the test process according to the test requirement information of the test scheme, obtain the test intermediate execution instruction, and use it for further test execution of the scheme to be tested.

[0072] Optionally, the test requirement information includes one or more of the following:

[0073] Test plan information;

[0074] Test interaction rules;

[0075] Test indicators;

[0076] Test judging criteria;

[0077] Data analysis rules;

[0078] Data analysis process;

[0079] Test execution uses prompt words.

[0080] In one embodiment, optionally, the test requirement information may include test interaction rules, which may be interaction rules described by the tester in natural language, or may include test plan information, which may include test execution steps. Using the method described in this embodiment, a multimodal large model may be used to analyze the test process information fed back by the test execution module based on the test interaction rules and / or test plan information described in natural language, and generate test intermediate execution instructions for further test execution of the plan to be tested.

[0081] In one implementation manner, optionally, the test process information includes one or more of a current state of a test execution tool, a current state of a tested product, and test intermediate result information.

[0082] Specifically, by adopting this implementation, a multimodal large model can be used to analyze one or more of the current state of the test execution tool, the current state of the product under test, and the intermediate test result information fed back by the test execution device based on the test interaction rules and / or test plan information described in natural language to obtain the intermediate test execution instructions.

[0083] Optionally, in this implementation, the test intermediate execution instruction is an instruction that can be parsed and executed by a test engine of a test execution device, and may be an operation instruction or a structured parameter, such as one or more of a shell command, an HTTP request, and a robot motion instruction.

[0084] The test method described in the embodiments of the present application, in one implementation mode, is applied to a data analysis device, or called a data analysis module. The data analysis device or the data analysis module is a device for setting a multimodal large model, which can use the multimodal large model to analyze the test process information according to the test requirement information to obtain the test intermediate execution instructions.

[0085] In one embodiment, optionally, the method further comprises:

[0086] Obtaining test result information of the test execution device executing the scheme to be tested;

[0087] The multimodal large model is used to analyze the test result information according to the test requirement information to obtain a test report for the to-be-tested solution.

[0088] Optionally, using the multimodal large model to analyze the test result information according to the test requirement information includes:

[0089] Optionally, in one embodiment, the test result information includes test process data and / or test product status information, etc., and a multimodal large model is used to analyze the test result information according to the test indicators and / or test evaluation criteria in the test requirement information to obtain a test report. Specifically, the test requirement information can be input into the multimodal large model in the form of natural language description. The multimodal large model has natural language data analysis and processing capabilities, and has multimodal data analysis capabilities, such as the ability to jointly process text, pictures, voice, video, symbols, etc., so that the data analysis method and process can be described by language. The multimodal large model can be used to analyze the test requirement information and the test result information, summarize and extract text information, locate and identify objects and text in the picture, extract and translate voice content, determine whether the key event of the video occurs, and identify whether the structured / symbolized data is consistent with the judgment criteria, etc., to obtain the test report of the scheme to be tested, realize the test in the multimodal test scenario, and solve the problems of complex and diverse data types in the multimodal test scenario, difficult to distinguish by traditional methods, and frequent iteration of test steps according to product updates.

[0090] In another implementation manner, optionally, the test requirement information may further include historical test results of the solution to be tested and / or test indicators of competitive products related to the product to be tested of the solution to be tested, and the multimodal large model is used to analyze the test result information according to the test requirement information, which may further include:

[0091] Using a multi-modal large model, the test result information is compared with the historical test results of the solution to be tested to obtain a test report. Specifically, the performance change timing characteristics of the tested product of the solution to be tested are analyzed and obtained; and / or

[0092] Using a multimodal large model, the test result information is compared with the test indicators of competitive products related to the tested product to obtain a test report. Specifically, the reasons for the performance differences of different products are analyzed.

[0093] With this implementation, the reasons for the performance differences between different products obtained through the above analysis and the timing characteristics of the performance changes of the tested products can be written into the test report to help testers analyze product problems and locate abnormal causes.

[0094] Optionally, if the actual test results are too far from the expected ones, the multimodal large model can also be used to analyze whether there are problems in the test plan or script, so as to iteratively optimize the test process, eliminate the situation where the test results are abnormal due to problems in the test steps, and realize the debugging feedback optimization loop.

[0095] In one implementation of the test method described in the embodiment of the present application, optionally, the test requirement information includes test rules and / or test indicators described in natural language and / or picture form, and the method further includes:

[0096] The multimodal large model is used to analyze the test rules and / or test indicators to obtain test guidance information, and the test guidance information is used to generate an execution script for the scheme to be tested, and the execution script is used by the test execution device to execute the scheme to be tested.

[0097] Optionally, the test guidance information includes one or more of a test indicator reference value, a test networking mode, a test step, and script-related information.

[0098] By adopting this implementation mode, testers can use natural language or pictures to describe test rules and / or test indicators according to the actual needs of the test scenario, and use multimodal large models to analyze based on relevant knowledge and documents to generate recommendations for test plans and indicator determination rules, such as test indicator reference values, test networking methods, test steps, test scripts, and other contents.

[0099] Adopt the test method described in the embodiment of the present application, first use various automated test execution frameworks and scripts to execute the product test process, and record various types of test data required for the test, and upload them to the model deployment device (test analysis device) as the input content of the multimodal large model, and the multimodal large model analyzes the test results according to the test plan and indicator evaluation rules. Optionally, for test scenarios that need to interact with testers during the test process, the test execution device needs to record the current test process information and wait, and upload the test process information (such as the status of the tested product and the output content, etc.) to the multimodal large model, and the multimodal large model infers and generates subsequent test steps according to the test process and discrimination rules (test requirement information) defined by the tester, and sends the corresponding execution instructions to the device executing the test, completes the interaction and continues to execute the test, thereby realizing the automation of the test interaction process in the multimodal test scenario.

[0100] Figure 2 Schematic diagram of the system architecture using the test method described in the embodiment of the present application. In the embodiment of the present application, optionally, the system using the test method may also include a test execution device (module) and a test control device (module).

[0101] In one implementation mode, optionally, the test analysis device (module), the test execution device (module) and the test control device (module) can be the same device, or can be different devices independent of each other.

[0102] In an embodiment of the present application, optionally, the test management and control device is a device that directly connects to the test personnel, and is used to coordinate key information such as test plans, test scripts, test data, and evaluation criteria required for various tests, form test projects and tasks, and complete test task scheduling and result collation.

[0103] Specifically, the test control device interacts with the tester, and the tester can use the test control device to complete the test plan writing, test task issuance, and test result collation. Testers can use this module to write test plans and scripts, start or terminate test tasks, view the execution status of the test engine, view test data and reports, etc. In addition, the test control device can also interact with the data analysis device. Testers can use the multimodal large model through the test control device to assist in writing test plans and scripts, or execute and use prompt words to describe and define the intermediate states that may appear during the test process, so that the multimodal large model can simulate the tester to generate various types of interactive instructions according to the state of the product under test during the test, such as generating test guidance information, to achieve automation and intelligence of interactive tests. Optionally, the test execution device is used to execute the plan to be tested. Specifically, the test execution device automatically executes the test script issued by the measurement and control device, and records the key node data during the test process. Specifically, the test execution device is usually directly combined with the automated testing tool or script to execute the automated testing script issued by the test management and control device, collect and record the key intermediate results and final result data in the test process, that is, obtain the test process information, and send the test process information to the data analysis device that deploys the multimodal large model.

[0104] In an embodiment of the present application, optionally, the test execution device includes a test engine constructed by one or more automated test tools or scripts, which can parse the execution logic of the script and execute the test control operation steps (such as playing voice, clicking UI interface controls in the APP, capturing the mobile phone screen, etc.) according to the test execution script issued by the test management and control device. For test scenarios where the test intermediate process needs to interact through the data analysis device, the test execution device uploads the current state of the test execution tool, the current state of the tested product and / or the intermediate process results to the data analysis device, and parses the feedback test intermediate execution instructions to execute subsequent test steps according to the test intermediate execution instructions.

[0105] By adopting the testing method described in this embodiment, by connecting the test execution device with the data analysis device deployed with the multimodal large model, and combining the traditional testing tools with the multimodal large model, it is possible to realize intelligent test execution, so as to avoid the problem that the existing automated testing framework and tools, due to their single function, are unable to support the increasingly complex product interaction processes and multimodal application scenario testing, and the intelligent testing cannot match the intelligence of the product.

[0106] Optionally, the data analysis device can screen and analyze the received test data according to the test requirement information issued by the test control device, form a test report based on the analysis results, and upload the test report to the test control device. For test scenarios that require multiple interactions, the data analysis device can analyze the intermediate results and product status (test process information) returned by the test execution module according to the interaction rules issued by the test control device, form interaction instructions or tool operation commands (test intermediate execution instructions) and return them, and the test execution device completes subsequent test actions according to the test intermediate execution instructions.

[0107] In the embodiment of the present application, the test method is applied to a data analysis device, which uses the multimodal large model's ability to understand and reason about modal data such as text, images, videos, and voices, combined with a plug-in knowledge base, so that the data analysis device has the ability to perform tasks such as analyzing test results, predicting test steps, and generating test instructions. Based on the above, the test method described in the embodiment of the present application is adopted, and the main functions of the data analysis device are as follows:

[0108] Test rule analysis: Testers can use natural language or pictures to describe test rules and indicators according to the actual needs of the test scenario. The data analysis equipment deployed with the multimodal large model can perform analysis based on relevant knowledge and documents to generate test guidance information, such as generating recommendations for test plans and indicator determination rules. The test guidance information may include one or more of the following contents: test indicator reference values, test networking methods, test steps, test scripts, etc.

[0109] Test data analysis: The data analysis device can receive the test data and product status information (test process information and / or test result information) uploaded by the test execution device through a multimodal large model, analyze the test process information and / or test result information according to the test requirement information issued by the test control device and the indicator calculation method, organize it into a test report and submit it to the test control device. Testers can describe the data analysis methods and processes in natural language to achieve automatic analysis of test results. In addition, the data analysis device has multimodal data analysis capabilities, such as text, pictures, voice, video, symbols, etc. The multimodal large model can be used to summarize and extract text information, locate and identify objects and text in pictures, extract and translate voice content, determine whether key events in the video have occurred, and identify whether structured / symbolized data is consistent with the judgment criteria, etc., according to analysis needs, to solve the problems of complex and diverse data types in multimodal test scenarios, difficult identification by traditional methods, and frequent iterations of test steps according to product updates.

[0110] Automatic analysis of interaction during the test process: When the product under test needs to interact with the tester, the multimodal large model can be used to analyze the status of the tested product and the intermediate output results fed back by the test execution device according to the test plan step requirements or the interaction rules described by the tester in natural language, and generate subsequent interaction instructions (test intermediate execution instructions), which are parsed and executed by the test engine to complete the subsequent test steps. After fine-tuning, the multimodal large model can generate operational instructions or structured parameters that can be executed by the test tool, such as shell commands, HTTP requests, robot motion instructions, etc., based on the test tools and scripts used by the test execution device.

[0111] Test result analysis: After the data analysis equipment completes the analysis of the test data, it can compare the test results with the historical test results recorded by the test control equipment vertically to analyze the timing characteristics of product performance changes. It can also compare the indicators of other related competing products horizontally to analyze the potential reasons for the performance differences between different products, and help testers analyze product problems and locate abnormal causes. If the actual test results are too far from the expected results, the data analysis equipment can also analyze whether there are problems in the test plan or script, iteratively optimize the test process, and eliminate the situation where abnormal test results are caused by problems in the test steps, thus realizing the adjustment feedback optimization cycle.

[0112] Report generation: After the data analysis device completes the analysis of multiple test data, it can generate a test report for the test based on the preset test report template or the tester's description, and upload it to the test management and control device for archiving.

[0113] In the embodiment of the present application, optionally, the method further includes:

[0114] Obtain test data sets and model adjustment strategies for multiple test application scenarios;

[0115] The pre-trained initial model is trained using the test data set and the model adjustment strategy to obtain the multimodal large model.

[0116] Optionally, obtain test data sets for multiple test application scenarios, including:

[0117] Acquire application data included in the to-be-tested solutions in multiple test application scenarios; the application data includes one or more of application components, icon images, and text data;

[0118] The data to be tested in the application data is marked to obtain the test data set.

[0119] When using the testing method described in this embodiment, before applying the multimodal large model for data analysis, it is necessary to train the pre-trained initial model using test data sets and model adjustment strategies under multiple test application scenarios to fine-tune the pre-trained initial model and obtain a multimodal large model that can analyze the multimodal testing process.

[0120] Optionally, the process of fine-tuning the pre-trained initial model includes four steps: dataset design and preparation, model selection and initialization, model fine-tuning strategy design, training and model optimization:

[0121] 1) Dataset design and preparation process

[0122] First, based on the main analysis tasks undertaken by the multimodal large model, data collection work is carried out from the test application scenario to construct a test data set.

[0123] A feasible method is to use test execution equipment or manual annotation to pre-annotate multiple test scenarios, terminal or web page UI pages, collect a large amount of application components, icon images and text data contained therein, and ensure that the data set covers different devices, operating systems and interface styles when collecting data. Then, the collected data is annotated, including identifying the category and location of interface elements (such as buttons, text boxes, icons, etc.), identifying interface text and image content, etc., to obtain a test data set.

[0124] In one implementation, after obtaining the test data set, it is optional to clean the obtained test data set, and remove the noise data of low quality or atypical test scenes through manual detection and other methods to ensure the accuracy and consistency of the data set. In addition, since the amount of UI-related test data is relatively small, some data (such as icons, buttons, etc.) may be too similar, and the data set can be expanded through data enhancement methods such as image rotation, scaling, and cropping.

[0125] For example, since it is necessary to fine-tune the pre-trained initial model on the UI-related data set, the method of manually collecting the test data set can be adopted. Specifically, the collected test data set can include thousands of manually collected screenshots from mobile phones and computers. Each screenshot is manually labeled with screen elements, potential tasks, and operation methods, and recorded in the form of questions and answers. The labeled information can be formed into a test data set. At the same time, the content of the test data set can also span multi-platform operating systems. In addition to focusing on web pages on the Windows and MacOS sides, it also includes data on Android and IOS user behavior. These data include tasks, operation sequences and corresponding screenshots. When annotating, in addition to manual annotation, GPT-4 can also be used to convert it into a natural language question-and-answer format to assist the annotation process.

[0126] 2) Model selection and initialization

[0127] In one embodiment, the model weights pre-trained on a large-scale general visual question answering (VQA) dataset are optionally used as the pre-trained initial model to take advantage of its powerful capabilities in image and text processing. When selecting the initial model, a multimodal pre-trained large model (such as CogAgent, Ferret-UI, etc.) whose training set includes a UI page dataset can be selected. Since the model has been adaptively fine-tuned on data related to the UI interface, it has better performance when fine-tuning than a multimodal large model trained with a natural image dataset, and the required dataset is usually smaller.

[0128] Specifically, the selected pre-trained initial model is a multimodal pre-trained large model that includes a UI page dataset. In this way, the pre-trained initial model already has strong capabilities in image and text processing related to UI tasks, and can effectively capture the semantic association between images and text. In the fine-tuning initialization stage, using its pre-trained model structure and parameters, it can directly utilize the rich visual and language features it has learned, reducing the resource requirements for training from scratch. This targeted model selection can improve the performance of the model in UI interface-related tasks.

[0129] 3) Model fine-tuning strategy design

[0130] In an embodiment of the present application, optionally, the adjustment strategy for pre-training the pre-trained initial model includes one or more of a test instruction, a loss function, a transfer learning strategy, and a multimodal fusion strategy.

[0131] For test instructions, it is necessary to design appropriate instruction prompts for multimodal large models based on the actual needs of multiple test application scenarios. For example, in the UI element detection task, you can use instructions such as "find the button on the screen" or "describe the navigation bar on this page", and fine-tune the instructions in combination with the collected test data set, so as to use this kind of highly targeted instructions to help the model understand and complete the task more accurately.

[0132] For the transfer learning strategy, in the design of fine-tuning strategies for UI task testing for multimodal large models, the pre-trained initial model can be used for transfer learning, by loading the pre-trained weights and further training on the data set of specific automated test-related tasks to shorten the training time and improve the model performance. For example, in the UI page recognition task, the multimodal large model may need to recognize various UI elements and classify and locate them, so when fine-tuning, it will focus on the data sets of these tasks.

[0133] For loss function design, a suitable loss function is designed for UI page recognition and detection tasks, such as combining cross entropy loss and Intersection over Union (IoU) loss to balance the classification and positioning accuracy in image tasks.

[0134] For multimodal fusion strategies, visual, auditory and text information can be effectively combined by designing fusion strategies. For example, when dealing with UI multi-round dialogue test tasks with visual, textual and audio information, an attention mechanism can be used to balance information from different modalities. This mechanism helps the model focus on the most important information for the current task when processing tasks.

[0135] 4) Training process and optimization

[0136] This process is used to fine-tune and optimize the pre-trained initial model through the training process and optimization, and to tune the hyperparameters, that is, to adjust the hyperparameters through methods such as grid search or Bayesian optimization. For example, the hyperparameters include learning rate, batch size and / or regularization parameters to find the best training configuration. At the same time, the problem of data balance should also be considered, and the data should be processed to solve the imbalance problem, ensuring that different categories of UI elements are evenly distributed in the training data set to avoid the model being biased towards certain categories. After the fine-tuning is completed, the data generated in the actual test task can be used to evaluate the model performance and evaluate its actual application performance in the test field.

[0137] In the actual process of fine-tuning the selected multimodal large model, many hyperparameter settings need to be continuously optimized and explored. The learning rate, batch size, regularization parameters, etc. of the model can be adjusted through grid search or Bayesian optimization methods to find the best training configuration. For the learning rate, it is necessary to find the most suitable learning rate so that the model converges quickly and stably. For the batch size, it is necessary to adjust the number of samples used in each training iteration to balance the training speed and memory usage. For the regularization parameters, it is necessary to set the appropriate regularization strength to prevent the model from overfitting.

[0138] In dealing with data imbalance problems, in addition to increasing the number of certain samples through data augmentation or manually reducing the number of samples to solve the problem of imbalanced training data categories, you can also use a weighted loss function to give higher loss weights to categories with fewer samples to ensure that the model attaches importance to these categories.

[0139] After fine-tuning is completed, the fine-tuned multimodal large model is evaluated using data generated in actual test tasks. The model is applied to real UI automation test scenarios to observe its performance in tasks such as identifying and locating UI elements and generating descriptions; optionally, evaluation indicators can include accuracy, recall, F1-score, IoU, etc.

[0140] like Figure 3 This is a flow chart of one implementation method of the test method described in the embodiments of the present application, taking the use of a multimodal large model to test the product interaction process as an example, where the test indicators are: when a user issues a command, the smart home elderly assistance robot accurately understands the user's intention at the semantic level and makes corresponding feedback to the user's command.

[0141] In this implementation, since the evaluation task is related to the robot screen display, voice feedback and interaction with other intelligent hardware, it is necessary to judge the task execution status through video, picture and voice data. The main evaluation process is:

[0142] S301, start;

[0143] S302, the tester writes the test interaction rules through the test control device: the tester formulates the interaction rules between the multimodal large model and the test equipment and the matching equipment according to the specific process in the test process, and uses natural language prompts to make the multimodal large model understand the possible situations and coping methods in the test process. For example, when playing the wake-up voice or waking up the intelligent robot with the multimodal large model voice capability, if the robot performs the wake-up action or plays the answer voice, the subsequent steps are executed, otherwise the multimodal large model controls the wake-up process again.

[0144] S303, this step is an optional step. The test management and control device sends the test rules and / or test indicators to the data analysis device, and the data analysis device analyzes the test rules and / or test indicators to obtain test guidance information.

[0145] S304, the test control device sends the test script and data to the test execution device: after the tester completes the plan writing and environment setting up, the test process is started; specifically, the test control module sends the test script and data to the test execution device, sends the test plan, large model interaction prompt words and test indicator evaluation criteria to the data analysis device, and starts the automated execution of the test process.

[0146] S305, the test engine of the test execution device executes the test, such as the test engine executes the test script and plays the test instructions.

[0147] S306, the test execution device determines whether it needs to interact with the data analysis device; if the judgment result is yes, execute S307, if the judgment result is no, execute step S309.

[0148] S307, the test execution module sends the test process information to the data analysis device, and the data analysis device analyzes the test process information according to the test requirement information.

[0149] S308, the data analysis device uses a multimodal large model to analyze the test process information according to the test requirement information, obtains the test intermediate execution instruction, and sends the test intermediate execution instruction to the test execution device, and returns to execute step S305.

[0150] For example, the test instruction of the test execution device is "pick up the cup on the table". The camera and microphone carried by the test execution device record the entire process of the robot's movement and picking up, and upload the key step images or sounds to the data analysis device. The multimodal large model determines whether the robot starts to move and whether it successfully picks up the cup, and generates the next instruction (test intermediate execution instruction), such as "play the next test instruction", "play the 'return to original position' instruction" or "replay the current instruction", etc., which are sent back to the test engine for execution to complete the subsequent test process.

[0151] In one implementation mode, the data analysis device can be optionally deployed to the end side and combined with the test execution device to form an integrated device to directly control the test engine and the cooperating devices: if the test execution device is mounted on a high-computing power device, the data analysis device can be deployed to the device to achieve efficient control of the test execution device. For example, when using a ROS robot to cooperate in executing the test, a large multimodal model can be deployed on the device, and the video recorded by the camera can be directly input into the large model to determine the real-time status of the device under test, while controlling the ROS robot to move with the device under test. This method is usually used to test tasks such as intelligent robots moving across rooms.

[0152] S309, the data analysis module obtains the test result information of the analysis test and analyzes the test result information.

[0153] S310, the data analysis module generates a test report based on the test result information, and uploads the test report to the test management and control device.

[0154] S311, end.

[0155] Using this implementation process, the data analysis device organizes all the index results of this test according to the test plan, outputs the test report and uploads it to the test control device. The user can view the test results and the process data of this test on the test control device, determine whether the test is effective, record the results in the historical evaluation data and update the baseline data. By adding new test data to the plug-in knowledge base and fine-tuning data set, the multimodal large model can obtain the latest index values ​​and test process, improve the accuracy of the model generation content and the timeliness of the knowledge, and improve the effectiveness of the system in a long operation cycle. At this point, the entire process of using the system to perform a test is completed.

[0156] The test method described in the embodiment of the present application is adopted, and the strong multimodal information understanding and reasoning capabilities of the multimodal large model are utilized. Based on the test process interaction rules specified by the tester, the "test execution-result analysis-process feedback" interaction cycle with the tested product during the test process and the analysis of the multimodal test results are realized. This solution is not only applicable to the interactive testing field of multi-purpose intelligent terminal products such as intelligent terminals, robots, and intelligent bodies, but can also be widely used in traditional product testing fields such as various software testing and AI testing.

[0157] Specifically, one of the implementation methods of the present application uses a multimodal large model to analyze various types of data generated during the test process, and can simultaneously analyze multimodal data such as text, pictures, and videos generated synchronously during the test process, solving the problem that traditional tools can only analyze one type of test data or one test scenario. In one of the implementation methods, due to the high versatility brought about by the emergence of multimodal large model capabilities, testers can directly use natural language prompts to describe the test requirements of different scenarios, so that they can adapt to different test tasks, without the need to develop test analysis tools or code scripts in a targeted manner, which greatly reduces the workload of preliminary test preparation, improves test efficiency, and saves a lot of manpower and event costs. In one of the implementation methods, the multimodal large model can truly and vividly imitate human language, voice or behavior. The method can describe the test process or interaction rules to the large model through natural language, and the large model replaces the tester to complete the human-computer interaction requirements during the test process, solving the problems of single interaction rules, difficulty in judging abnormal situations, and large differences between interaction data and real scenarios in traditional automated testing.

[0158] One embodiment of the present application also provides a testing method, such as Figure 4 As shown, including:

[0159] S410, during the test process of executing the test scheme, sending the test process information of the test scheme to the test analysis device;

[0160] S420, obtaining a test intermediate execution instruction sent by the test analysis device, wherein the test intermediate execution instruction is obtained by analyzing the test process information according to the test requirement information of the to-be-tested scheme using a multimodal large model;

[0161] S430: Continue to execute the test of the to-be-tested solution according to the test intermediate execution instruction.

[0162] The test method described in the embodiment of the present application can, optionally, be applied to a test execution device in this implementation mode. In this implementation mode, the test execution device is combined with a test analysis device that deploys a multimodal big model, and the multimodal big model's analysis and reasoning capabilities for multimodal information can be used to simultaneously analyze multimodal data such as text, images, and videos generated synchronously during the test process, thereby realizing test process analysis during the multimodal test process, and generating test intermediate execution instructions to guide test execution based on test requirement information for subsequent test step execution, thereby realizing automation of the test interaction process in a multimodal test scenario.

[0163] Optionally, the testing method further comprises:

[0164] Obtaining test result information after testing the scheme to be tested;

[0165] The test result information is transmitted to the test analysis device.

[0166] Optionally, in the test method, the test process information includes one or more of the current state of the test execution tool, the current state of the product under test and test intermediate result information.

[0167] The specific implementation of the test method described in the embodiments of the present application applied to the test execution device can refer to the detailed description of the specific implementation of the test analysis device, and will not be repeated here.

[0168] One embodiment of the present application also provides a testing device, such as Figure 5 As shown, the test device 500 includes a processor 510 and a transceiver 520, wherein:

[0169] The processor 510 is used to obtain test requirement information of the scheme to be tested, and to obtain test process information obtained by the test execution device executing the scheme to be tested; and to analyze the test process information according to the test requirement information using a multimodal large model to obtain a test intermediate execution instruction;

[0170] The transceiver 520 is used to send the test intermediate execution instruction to the test execution device for further test execution of the to-be-tested solution.

[0171] Optionally, in the test device, wherein the test requirement information includes test rules and / or test indicators described in natural language and / or in the form of pictures, the processor 510 is further configured to:

[0172] The multimodal large model is used to analyze the test rules and / or test indicators to obtain test guidance information, and the test guidance information is used to generate an execution script for the scheme to be tested, and the execution script is used by the test execution device to execute the scheme to be tested.

[0173] Optionally, the test device, wherein the test guidance information includes one or more of a test indicator reference value, a test networking mode, a test step and script related information.

[0174] Optionally, in the test device, the processor 510 is further configured to:

[0175] Obtaining test result information of the test execution device executing the scheme to be tested;

[0176] The multimodal large model is used to analyze the test result information according to the test requirement information to obtain a test report for the to-be-tested solution.

[0177] Optionally, in the test device, the test process information includes one or more of a current state of a test execution tool, a current state of a tested product, and test intermediate result information.

[0178] Optionally, in the test device, the test requirement information includes one or more of the following:

[0179] Test plan information;

[0180] Test interaction rules;

[0181] Test indicators;

[0182] Test judging criteria;

[0183] Data analysis rules;

[0184] Data analysis process;

[0185] Test execution uses prompt words.

[0186] Optionally, in the test device, the processor 510 is further configured to:

[0187] Obtain test data sets and model adjustment strategies for multiple test application scenarios;

[0188] The pre-trained initial model is trained using the test data set and the model adjustment strategy to obtain the multimodal large model.

[0189] Optionally, in the test device, the model adjustment strategy includes one or more of instruction fine-tuning, loss function adjustment, transfer learning strategy and multimodal fusion strategy.

[0190] Optionally, in the test device, the processor 510 acquires test data sets in multiple test application scenarios, including:

[0191] Acquire application data included in the to-be-tested solutions in multiple test application scenarios; the application data includes one or more of application components, icon images, and text data;

[0192] The data to be tested in the application data is marked to obtain the test data set.

[0193] One embodiment of the present application also provides a testing device, such as Figure 6 As shown, the test device 600 includes a transceiver 610 and a processor 620, wherein:

[0194] The transceiver 610 is used to send the test process information of the scheme to be tested to the test analysis device during the test process of the scheme to be tested; and obtain the test intermediate execution instruction sent by the test analysis device, wherein the test intermediate execution instruction is obtained by analyzing the test process information according to the test requirement information of the scheme to be tested using a multi-modal large model;

[0195] The processor 620 is used to continue to execute the test of the to-be-tested solution according to the test intermediate execution instruction.

[0196] Optionally, in the test device, the transceiver 610 is further used for:

[0197] Obtaining test result information after testing the scheme to be tested;

[0198] The test result information is transmitted to the test analysis device.

[0199] Optionally, in the test device, the test process information includes one or more of a current state of a test execution tool, a current state of a tested product, and test intermediate result information.

[0200] One embodiment of the present application also provides a testing device, such as Figure 7 As shown, including:

[0201] The information acquisition module 710 is used to acquire the test requirement information of the scheme to be tested, and acquire the test process information obtained by the test execution device executing the scheme to be tested;

[0202] An analysis module 720 is used to analyze the test process information according to the test requirement information using a multi-modal large model to obtain a test intermediate execution instruction;

[0203] The first sending module 730 is used to send the test intermediate execution instruction to the test execution device for further test execution of the to-be-tested solution.

[0204] Optionally, in the test device, wherein the test requirement information includes test rules and / or test indicators described in natural language and / or in the form of pictures, the analysis module 720 is further used to:

[0205] The multimodal large model is used to analyze the test rules and / or test indicators to obtain test guidance information, and the test guidance information is used to generate an execution script for the scheme to be tested, and the execution script is used by the test execution device to execute the scheme to be tested.

[0206] Optionally, in the test device, the test guidance information includes one or more of a test indicator reference value, a test networking mode, a test step and script related information.

[0207] Optionally, in the test device, the analysis module 720 is further used for:

[0208] Obtaining test result information of the test execution device executing the scheme to be tested;

[0209] The multimodal large model is used to analyze the test result information according to the test requirement information to obtain a test report for the to-be-tested solution.

[0210] Optionally, in the test device, the test process information includes one or more of a current state of a test execution tool, a current state of a tested product, and test intermediate result information.

[0211] Optionally, in the test device, the test requirement information includes one or more of the following:

[0212] Test plan information;

[0213] Test interaction rules;

[0214] Test indicators;

[0215] Test judging criteria;

[0216] Data analysis rules;

[0217] Data analysis process;

[0218] Test execution uses prompt words.

[0219] Optionally, the testing device further comprises a training module 740, which is used to:

[0220] Obtain test data sets and model adjustment strategies for multiple test application scenarios;

[0221] The pre-trained initial model is trained using the test data set and the model adjustment strategy to obtain the multimodal large model.

[0222] Optionally, in the testing device, the model adjustment strategy includes one or more of instruction fine-tuning, loss function adjustment, transfer learning strategy and multimodal fusion strategy.

[0223] Optionally, in the test device, the training module 740 obtains test data sets in multiple test application scenarios, including:

[0224] Acquire application data included in the to-be-tested solutions in multiple test application scenarios; the application data includes one or more of application components, icon images, and text data;

[0225] The data to be tested in the application data is marked to obtain the test data set.

[0226] One embodiment of the present application also provides a testing device, such as Figure 8 As shown, including:

[0227] The second sending module 810 is used to send the test process information of the scheme to be tested to the test analysis device during the test process of the scheme to be tested;

[0228] An acquisition module 820 is used to obtain the test intermediate execution instruction sent by the test analysis device, wherein the test intermediate execution instruction is obtained by analyzing the test process information according to the test requirement information of the scheme to be tested using a multi-modal large model;

[0229] The testing module 830 is used to continue to execute the test of the to-be-tested solution according to the test intermediate execution instruction.

[0230] Optionally, in the test device, the acquisition module 820 is further used for:

[0231] Obtaining test result information after testing the scheme to be tested;

[0232] The test result information is transmitted to the test analysis device.

[0233] Optionally, in the test device, the test process information includes one or more of a current state of a test execution tool, a current state of a tested product, and test intermediate result information.

[0234] One embodiment of the present application further provides a network device, which includes a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program implements any of the above test methods when executed by the processor.

[0235] Among them, the specific implementation method of executing the test method by the program running on the processor of the network device can refer to the detailed description when the test method is applied to the above-mentioned test device, and will not be repeated here.

[0236] In addition, a specific embodiment of the present application further provides a readable storage medium on which a computer program is stored, wherein when the program is executed by a processor, the steps in the test method described in any one of the above items are implemented.

[0237] Specifically, the readable storage medium is applied to the above-mentioned test device. When applied to the test device, the execution steps in the corresponding test method are described in detail above and will not be repeated here.

[0238] Another embodiment of the present application further provides a computer program product, which includes computer instructions, and when the computer instructions are executed by a processor, the steps in any one of the test methods described above are implemented.

[0239] Optionally, the embodiments of the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0240] The computer program product described in the embodiment of the present application includes computer instructions that implement the various processes of the test method embodiment shown above when executed by a processor, and can achieve the same technical effect. To avoid repetition, they are not repeated here.

[0241] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0242] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0243] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0244] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A testing method, characterized in that: include: Acquire test requirement information of the scheme to be tested, and acquire test process information obtained by the test execution device executing the scheme to be tested; Adopting a multi-modal large model, analyzing the test process information according to the test requirement information, and obtaining the test intermediate execution instructions; The test intermediate execution instruction is sent to the test execution device for further test execution of the to-be-tested solution.

2. The testing method according to claim 1, characterized in that: The test requirement information includes test rules and / or test indicators described in natural language and / or in the form of pictures, and the method further includes: The multimodal large model is used to analyze the test rules and / or test indicators to obtain test guidance information, and the test guidance information is used to generate an execution script for the scheme to be tested, and the execution script is used by the test execution device to execute the scheme to be tested.

3. The testing method according to claim 2, characterized in that: The test guidance information includes one or more of a test indicator reference value, a test networking mode, a test step, and script related information.

4. The testing method according to claim 1, characterized in that: The method further comprises: Obtaining test result information of the test execution device executing the scheme to be tested; The multimodal large model is used to analyze the test result information according to the test requirement information to obtain a test report for the to-be-tested solution.

5. The testing method according to claim 1, characterized in that: The test process information includes one or more of the current state of the test execution tool, the current state of the tested product, and the test intermediate result information.

6. The testing method according to claim 1, characterized in that: The test requirement information includes one or more of the following: Test plan information; Test interaction rules; Test indicators; Test judging criteria; Data analysis rules; Data analysis process; Test execution uses prompt words.

7. The testing method according to claim 1, characterized in that: The method further comprises: Obtain test data sets and model adjustment strategies for multiple test application scenarios; The pre-trained initial model is trained using the test data set and the model adjustment strategy to obtain the multimodal large model.

8. The testing method according to claim 7, characterized in that: The model adjustment strategy includes one or more of instruction fine-tuning, loss function adjustment, transfer learning strategy and multimodal fusion strategy.

9. The testing method according to claim 7, characterized in that: Get test data sets for multiple test application scenarios, including: Acquire application data included in the to-be-tested solutions in multiple test application scenarios; the application data includes one or more of application components, icon images, and text data; The data to be tested in the application data is marked to obtain the test data set.

10. A testing method, characterized in that: include: During the test process of the scheme to be tested, the test process information of the scheme to be tested is sent to the test analysis device; Obtaining a test intermediate execution instruction sent by the test analysis device, wherein the test intermediate execution instruction is obtained by analyzing the test process information according to the test requirement information of the scheme to be tested using a multimodal large model; According to the test intermediate execution instruction, continue to execute the test of the to-be-tested solution.

11. The testing method according to claim 10, characterized in that: The method further comprises: Obtaining test result information after testing the scheme to be tested; The test result information is transmitted to the test analysis device.

12. The testing method according to claim 10, characterized in that: The test process information includes one or more of the current state of the test execution tool, the current state of the tested product, and the test intermediate result information.

13. A testing device, characterized in that: It includes a processor and a transceiver, wherein: The processor is used to obtain test requirement information of the scheme to be tested, and to obtain test process information obtained by the test execution device executing the scheme to be tested; and, using a multi-modal large model, analyze the test process information according to the test requirement information to obtain a test intermediate execution instruction; The transceiver is used to send the test intermediate execution instruction to the test execution device for further test execution of the to-be-tested solution.

14. A testing device, characterized in that: It includes a transceiver and a processor, wherein: The transceiver is used to send the test process information of the scheme to be tested to the test analysis device during the test process of the scheme to be tested; and obtain the test intermediate execution instruction sent by the test analysis device, wherein the test intermediate execution instruction is obtained by analyzing the test process information according to the test requirement information of the scheme to be tested using a multi-modal large model; The processor is used to continue to execute the test of the to-be-tested solution according to the test intermediate execution instruction.

15. A testing device, characterized in that: include: An information acquisition module, used to acquire test requirement information of the scheme to be tested, and to acquire test process information obtained by the test execution device executing the scheme to be tested; An analysis module, used to analyze the test process information according to the test requirement information by using a multi-modal large model, and obtain a test intermediate execution instruction; The first sending module is used to send the test intermediate execution instruction to the test execution device for further test execution of the to-be-tested solution.

16. A testing device, characterized in that: include: A second sending module is used to send the test process information of the scheme to be tested to the test analysis device during the test process of the scheme to be tested; An acquisition module, used to obtain the test intermediate execution instruction sent by the test analysis device, wherein the test intermediate execution instruction is obtained by analyzing the test process information according to the test requirement information of the scheme to be tested using a multimodal large model; The testing module is used to continue to execute the test of the to-be-tested solution according to the test intermediate execution instruction.

17. A network device, characterized in that: The invention comprises a processor, a memory and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the test method according to any one of claims 1 to 9, or implements the test method according to any one of claims 10 to 12.

18. A readable storage medium, characterized in that: The readable storage medium stores a program, and when the program is executed by a processor, the program implements the steps in the test method according to any one of claims 1 to 9, or implements the steps in the test method according to any one of claims 10 to 12.

19. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the test method according to any one of claims 1 to 9, or implement the steps of the test method according to any one of claims 10 to 12.

Images