A virtual test system and its usage method

Abstract

A virtual test system and its usage method are provided, which relate to the field of virtual test technology and include: a database module, an information acquisition module, an analysis module, and a virtual test module. A virtual test database is established, and the virtual test database includes virtual test models; the design scheme to be tested and its application scenario are obtained; the virtual test logic suitable for the design scheme is selected according to the design scheme to be tested and the virtual test models; the design scheme is subjected to virtual test according to the selected virtual test logic to obtain a test result. By applying virtual technology to the test process, before the software and system development are completed, the design scheme is subjected to virtual test, and the design scheme is subjected to multi-angle, multi-path, multi-sequence, and multi-level virtual tests through the virtual test logic required by the design scheme and the application scenario, which can greatly accelerate the test process, improve the test efficiency, and reduce the test cost.

Description

[0001] This application claims the priority of a Chinese patent application with the application number 202310317132.6 and the invention title "A Virtual Test System and Its Usage Method", which was filed with the Chinese Patent Office on March 25, 2023. The entire content thereof is incorporated herein by reference. Technical Field

[0002] The present invention relates to the technical field of virtual testing, and particularly to a virtual test system and its usage method. Background Art

[0003] With the continuous progress of science and technology, the scale and complexity of software and systems have been greatly improved, and the associated testing workload has increased sharply. Traditional manual testing usually has to be carried out after the software and system development are completed. Moreover, it has high requirements for testers, complex operations, a large amount of repetitive labor, high labor costs, low testing efficiency, and is also prone to omissions in testing due to personal reasons.

[0004] Most existing tests are based on the code level and adopt the method of testing through test cases. Due to the very complex actual application scenarios of the system, it is difficult to achieve complete coverage of test cases, conduct sufficient testing, and expose all risk points. And the actual real system problems, or the problems that are not easy to discover, are often the loopholes in the design and development plans. Therefore, a method for analyzing and evaluating from the system and code design logic levels is needed to ensure the integrity of the business logic and prevent system risks in various business scenarios. Summary of the Invention

[0005] The purpose of the present invention is to provide a virtual test system and its usage method, which apply virtual technology to the testing process. Before the software and system development are completed, virtual testing can be carried out on the design plan. Through the virtual test logic required by the design plan and application scenarios, multi-angle, multi-path, multi-sequence, and multi-level virtual testing can be carried out on the design plan, which can greatly accelerate the testing process, improve testing efficiency, and reduce testing costs.

[0006] To achieve the above purpose, the present invention provides the following solutions:

[0007] In a first aspect, the present invention provides a virtual test system, which includes: a database module, an information acquisition module, an analysis module, and a virtual test module.

[0008] The information acquisition module is used to acquire the design plan to be tested and its application scenario.

[0009] The database module is used to store a virtual test database, which includes a virtual test model and virtual test logics for different design schemes. The virtual test model is used to select a virtual test logic suitable for a design scheme according to the design scheme. The elements of the virtual test model include at least one of the following: target, user, scenario, interface system environment and interface conditions, data organization, process, link, control, intervention, evaluation, prediction, interaction, and interaction. The virtual test logic is used to perform virtual tests on the design scheme;

[0010] The analysis module is used to select a virtual test logic suitable for the design scheme to be tested according to the design scheme to be tested and the virtual test model;

[0011] The virtual test module is used to perform a virtual test on the design scheme according to the selected virtual test logic and obtain a test result.

[0012] In some embodiments, the virtual test logics of the design scheme include virtual test logics for functional modules and virtual test logics for the overall system. The virtual test logics for functional modules are used to perform virtual tests on each functional module of the software, including at least one virtual test logic of the following: core operations, queries, classifications, fuzzy matching, layout of the interface styles of functional modules, correct integrity of data display, paging of data lists, deletion, removal, addition, configuration, and linkage processes. The virtual test logics for the overall system are used to perform virtual tests on the overall system of the software, including at least one virtual test logic of the following: function implementation of the entire software system, rationality / stability / scalability / reusability of the architecture, usability / aesthetics / consistency / friendliness of the user interface, applicability / security / performance optimization of the database, interfaces / inputs / outputs / independence of modules, and complexity / time resource consumption / space resource consumption of data structures and algorithms.

[0013] In some embodiments, the virtual test system analyzes each element of the design scheme and the relationships between them from the design scheme, establishes connections in different dimensions of each element according to the relationships, and is convenient to display according to the requirements of the user. The elements of the design scheme include at least one of the following: link, node, data relationship, causality, output, trigger, input, and output.

[0014] In some embodiments, the virtual test module is further used to set at least one evaluation item of logic, computing power, display, permission, security, efficiency, load, capacity, and time under the virtual test of the virtual test logic according to the design scheme to be tested, set corresponding levels / score values according to the results of different items, and calculate the comprehensive level / score value of the design scheme to be tested in the virtual test according to the weights of each evaluation item in the virtual test, so as to facilitate giving a test result.

[0015] In some embodiments, the method for determining the test result includes: judging whether the comprehensive level / score is greater than a preset pass threshold to obtain a first judgment result; if the first judgment result is no, the test result is a test failure; if the first judgment result is yes, judging whether the level / score corresponding to each of the assessment items is higher than the single pass threshold corresponding to the assessment item to obtain a second judgment result; if the second judgment result is yes, the test result is a test pass; if the second judgment result is no, the test result is a test failure.

[0016] In some embodiments, the virtual testing system is also used to determine problem points and problem scenarios that are prone to errors based on the test results, and to prompt system risk points and risk scenarios based on the type, level, impact and difficulty of solving the error.

[0017] In some embodiments, the virtual test system further includes an adjustment module, which is used to determine the reason why the design scheme to be tested failed the test according to the test results, and to adjust and optimize the design scheme to be tested in a targeted manner based on the reason.

[0018] In some embodiments, the virtual test system further includes a code acquisition module and a standard code library, the standard code library includes standard codes suitable for different design schemes; the code acquisition module is used to directly acquire the corresponding standard code from the standard code library according to the first design scheme when the standard code library includes the standard code of the first design scheme, and to select the standard code most suitable for the first design scheme from the standard code library according to the first design scheme when the standard code library does not include the standard code of the first design scheme, and then modify the standard code most suitable for the first design scheme based on the difference between the first design scheme and the standard code most suitable for the first design scheme to obtain the standard code corresponding to the first design scheme;

[0019] The virtual test system also includes a code verification module, which is used to verify whether the standard code complies with the design plan, specifically to determine whether the standard code complies with the design plan in terms of input, output, function, efficiency, and resource occupancy. If not, the user is prompted to adjust the standard code.

[0020] In some embodiments, the virtual test system further includes a user interaction module, and the user interaction module is used to obtain signals sent by one or more users and present feedback information to the users.

[0021] In a second aspect, the present invention is used to provide a method for using the above-mentioned virtual test system, the method comprising:

[0022] Establish a virtual test database, where the virtual test database includes a virtual test model and virtual test logics for different design schemes. The virtual test model is used to select a virtual test logic suitable for a design scheme according to the design scheme. The elements of the virtual test model include at least one of the following: objective, user, scenario, interface system environment and interface conditions, data organization, process, link, control, intervention, evaluation, prediction, interaction, and interaction. The virtual test logic is used to perform virtual tests on design schemes.

[0023] Obtain the design scheme to be tested and its application scenario.

[0024] Select a virtual test logic suitable for the design scheme according to the design scheme to be tested and the virtual test model.

[0025] Perform a virtual test on the design scheme according to the selected virtual test logic to obtain a test result.

[0026] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention:

[0027] The present invention is used to provide a virtual test system and its usage method. The virtual test system includes: a database module, an information acquisition module, an analysis module, and a virtual test module. The usage method of the virtual test system includes: establishing a virtual test database, where the virtual test database includes a virtual test model; obtaining the design scheme to be tested and its application scenario; selecting a virtual test logic suitable for the design scheme according to the design scheme to be tested and the virtual test model; performing a virtual test on the design scheme according to the selected virtual test logic to obtain a test result. For a virtual test system and its usage method of the present invention, virtual technology is applied to the test process. Before the software and system development are completed, virtual tests are performed on the design scheme. Through the virtual test logics required by the design scheme and the application scenario, multi-angle, multi-path, multi-sequence, and multi-level virtual tests are performed on the design scheme, which can greatly accelerate the test process, improve the test efficiency, and reduce the test cost. Description of the Drawings

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0029] Figure 1 It is a method flow chart of a usage method of a virtual test system provided by Embodiment 2 of the present invention. Detailed Embodiments

[0030] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0031] The object of the present invention is to provide a virtual test system and its usage method. By applying virtual technology to the test process, virtual testing of the design scheme can be carried out before the software and system development are completed. Through the virtual test logic required by the design scheme and the application scenario, multi-angle, multi-path, multi-sequence, and multi-level virtual testing of the design scheme can be performed. According to the test results, the design scheme can be adjusted and optimized. Additionally, the corresponding standard code can be obtained from the standard code library based on the determined design scheme, and it can be verified whether the standard code conforms to the design scheme, which can greatly accelerate the test process, improve test efficiency, and reduce test costs.

[0032] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in combination with the accompanying drawings and specific embodiments.

[0033] Embodiment 1:

[0034] This embodiment is used to provide a virtual test system, and the virtual test system includes: a database module, an information acquisition module, an analysis module, and a virtual test module.

[0035] The information acquisition module is used to acquire the design scheme to be tested and its application scenario.

[0036] The database module is used to store a virtual test database. The virtual test database includes virtual test models and virtual test logics for different design schemes. The virtual test model is used to select a virtual test logic suitable for the design scheme according to the design scheme. The elements of the virtual test model include at least one of the following: target, user, scenario, interface system environment and interface conditions, data organization, process, link, control, intervention, evaluation, prediction, interaction, and interaction. The virtual test logic is used to perform virtual testing on the design scheme.

[0037] The analysis module is used to select a virtual test logic suitable for the design scheme to be tested according to the design scheme to be tested and the virtual test model.

[0038] The virtual test module is used to perform virtual testing on the design scheme to be tested according to the selected virtual test logic and obtain a test result.

[0039] In this embodiment, the information acquisition module is used to acquire the design solution to be tested and its application scenario. Among them, the design solution may include: objectives, requirements, functions, structures, development environments, interface system environments and interface conditions, data organizations, processes, links, controls, interventions, inputs, outputs, data indicators, configurations, predictions, interactions, interactions, etc. The application scenario may include: occurrence time, location, user, functional requirements, occurrence frequency, input range, output range, etc.

[0040] The data acquisition methods of the information acquisition module may include: (1) The user fills in a given template to acquire the design solution to be tested and its application scenario; (2) Acquire the design document and perform semantic recognition on the design document to fill in the given template to acquire the design solution to be tested and its application scenario; (3) Acquire the design document, and manually fill in the given template according to the design document to acquire the design solution to be tested and its application scenario; (4) Acquire the design document, perform digital processing on the design document, and then organize the information according to the multi-level template to acquire the design solution to be tested and its application scenario. Of course, other suitable data acquisition methods may also be adopted in this embodiment, as long as the purpose of acquiring the design solution to be tested and its application scenario can be achieved.

[0041] In this embodiment, the virtual test database includes a virtual test model and virtual test logics for different design solutions.

[0042] The virtual test model is used to select the virtual test logic suitable for the design solution according to the design solution, or the virtual test model is used to select the virtual test logic suitable for the design solution according to the design solution and its application scenario. The elements of the virtual test model include: objectives, users, scenarios, interface system environments and interface conditions, data organizations, processes, links, controls, interventions, evaluations, predictions, interactions, interactions, and may also include other elements related to the characteristics of the design solution.

[0043] The virtual test logic includes at least one of the virtual test logics corresponding to input, output, function, efficiency, and resource occupation. It is necessary to comprehensively consider multiple dimensions such as the application scenario, the structure of the software or system product, and the quality requirements, select the appropriate virtual test logic to perform virtual testing on the design solution, and different weights or priorities can also be set for the selected virtual test logic. The weights or priorities can be used for comprehensive scoring. A design solution may need to select multiple virtual test logics. By setting different weights or priorities for each selected virtual test logic, after testing the design solution using the selected virtual test logics, the comprehensive test score of the design solution can be calculated according to the test scores and weights or priorities of each selected virtual test logic, and whether the design solution passes the test can be evaluated based on the comprehensive test score.

[0044] Specifically, the virtual test logic of the design scheme can include the virtual test logic of the functional modules and the virtual test logic of the overall system. The virtual test logic of the functional modules is mainly used to perform virtual tests on the various functional modules of the software, and can include: virtual test logics such as core operations, queries, classifications, fuzzy matching, functional module interface style layout, correct integrity of data display, data list page turning, deletion, removal, addition, configuration, linkage process, etc. involved in each module, and can also include other virtual test logics related to the virtual test of functional modules. The virtual test logic of the overall system is mainly used to perform virtual tests on the overall system of the software, and can include: functional implementation of the entire software system, rationality / stability / scalability / reusability of the architecture, ease of use / aesthetics / consistency / friendliness of the user interface, applicability / security / performance optimization of the database, module interface / input / output / independence, complexity of data structure and algorithm / time resource consumption / space resource consumption, etc., and can also include other virtual test logics related to the virtual test of the overall system.

[0045] The virtual testing logic of the rationality of the architecture may include: whether the architecture is suitable for the "functional requirements" and "non-functional requirements" of the software, whether it just meets customer needs instead of adopting the most advanced structure at any cost, etc.

[0046] The virtual test logic of the architecture stability may include: determining whether the architecture is based on stable and unchanging requirements through demand analysis, whether the architecture needs to be modified when the requirements change, etc.

[0047] The virtual test logic of the architecture scalability may include: whether the time cost / space cost / work cost of software expansion functions meet the requirements, whether the time cost / space cost / work cost of developing a new version meets the requirements, etc.

[0048] The virtual test logic of architecture reusability may include: whether the universality / reusability / difficulty / cost / workload of the software architecture meets the requirements, etc.

[0049] The usability of the user interface refers to the degree to which the user interface is in harmony with the functions of the software. The functions of the software need to be presented through the user interface, and the user interface must be suitable for the functions of the software. The virtual test logic for the usability of the user interface can include: whether all elements in the user interface (such as menus, toolbars, etc.) are error-free and not misleading; whether all interface elements provide sufficient and necessary prompts; whether the interface structure can clearly reflect the workflow so that users can operate step by step; for complex user interfaces, whether an interface "wizard" is provided to promptly let users know their positions in the interface structure, etc. For example, for web-based application software, displaying "Current Location" on the user interface to prevent users from getting lost; when the mouse moves over an icon button on the toolbar, a function prompt should appear next to the icon.

[0050] The virtual test logic for the aesthetics of the user interface can include: whether the layout of the user interface is reasonable, whether the layout of the user interface matches the workflow, whether the layout of the user interface is neat and beautiful, whether the interface elements are aligned horizontally or vertically / the spacing between rows and columns is consistent, whether the size of the form is appropriate, whether various controls are overcrowded / too loose, whether the blanks and dividing lines of the form and controls are reasonable, whether colors are selected according to the importance of the object, whether important objects are represented by eye-catching colors, whether the use of colors is consistent, whether too much reliance is placed on colors when expressing information, etc. For example, if error prompt messages are represented by red and normal messages are represented by green, then red and green must not be misused; if it is not for displaying realistic graphics and images, then the number of colors on one screen frame should be limited because it is difficult for people to remember multiple colors simultaneously when observing the screen.

[0051] The virtual test logic for the consistency of the user interface can include: whether similar interface elements have the same visual sense and the same operation method, whether the user interfaces of the same type of software have a certain degree of similarity, etc. For example, command buttons are the most common interface elements, and the shapes, colors, and mouse response methods of all command buttons are consistent; the operation methods for similar functions provided by the software are the same.

[0052] The virtual test logic for user interface friendliness may include: timely prompting of the software's status, timely feedback of operation feedback information, and whether the error handling of various different errors is user-friendly. For example, after a user performs a certain operation, whether there is feedback on the user interface. If there is no response at all, it will make the user feel confused and uneasy because he doesn't know whether he made a wrong operation or the software crashed due to some reason; for operations such as downloading files, the interface should display "percentage" or relevant numbers to indicate the download progress, otherwise people don't know how long they have to wait; if some transaction processing cannot provide data such as progress, then at least a prompt message such as "Processing, please wait..." should be given; provide a function to verify the input data, and when the user enters incorrect data, timely remind the user to correct the data; for menu items and command buttons that should not be used in certain situations, "disable" (block) them to prevent the incorrect use of this function; provide an Undo function to undo unwanted operations; before performing a destructive operation such as deleting a file, the user's confirmation should be obtained.

[0053] The virtual test logic for database applicability may include: whether the selection of the database system meets the functional requirements / industry standards, and whether the unique functions of the database system are applied, etc.

[0054] The virtual test logic for database security may include: whether the way users log in to the software is secure, whether users' access to the database is secure, whether there are other ways to operate the database, whether the passwords of user accounts are encrypted and ensure that the plaintext of the password does not appear anywhere, and whether the operation permissions of each user for the database are appropriate, etc.

[0055] The virtual test logic for database performance optimization may include: whether the data redundancy degree / data structure clarity / performance / scalability of the database meet the requirements, whether the environmental parameters of the database meet the requirements, and whether the configuration of the database matches the actual data storage, etc. For example, when installing the database system, a "block size" (the number of bytes designed for a single physical read / write operation) should be specified for the system. When creating a table, a certain space, that is, a "table space", should also be specified for the table. If the "block size" and "table space" do not match the actual data storage, then a lot of disk fragmentation will occur, which will reduce the performance of database physical operations.

[0056] The virtual test logic for module interfaces may include: whether the functions of the module interfaces are suitable, whether the functions of the execution module or communicating information with the module need to be implemented by calling public interfaces, etc. For example, if the module is a C++ object, then the public interface of the module corresponds to the public functions of the object. If the module is a COM object, then the public interface of the module is the interface of the COM object. A COM object can have multiple interfaces, and each interface is essentially a collection of some functions.

[0057] In this embodiment, the virtual test database is established based on various product specifications, industry standards, teaching materials, papers, works, data analysis reports, test reports, inspection reports, approval documents, relevant regulations, relevant guiding opinions, relevant policies, relevant systems, relevant catalogs, and relevant literature materials. It can also be established based on probability speculation of existing data, or through various weights / grades / sorts that need to be manually set, or based on information reorganization / information analysis / big data analysis, or through artificial intelligence deep learning, or through data mining analysis, or through data statistical analysis / artificial intelligence deep learning followed by manual setting. The virtual test database can also be established by combining the above methods. The virtual test models and virtual test logics of different design schemes in the virtual test database of this embodiment can also be continuously updated and adjusted during use. The update and adjustment can be completed manually or automatically through data statistical analysis / artificial intelligence deep learning.

[0058] In this embodiment, the analysis module is used to select the virtual test logic suitable for the design scheme to be tested according to the design scheme to be tested and the virtual test model, that is, taking the design scheme to be tested as the input and using the virtual test model to determine the virtual test logic suitable for the design scheme to be tested.

[0059] The design scheme includes various different elements. By inputting the values of various different elements into the virtual test model, the virtual test logic suitable for this design scheme can be obtained. The elements of the design scheme can include: objectives, requirements, functions, structures, development environments, interface system environments and interface conditions, data organizations, processes, links, controls, interventions, inputs, outputs, data indicators, configurations, predictions, interactions, and mutual influences, etc. The values of the elements can be numerical values or characters. The values of the elements can be fixed, variable, or a range of changes. A change in the value of the element may cause a change in the selected virtual test logic suitable for this design scheme.

[0060] Specifically, the input objects, output objects, and the calculation / judgment / logical relationships between the input and output objects corresponding to different design schemes, the factors affecting the calculation / judgment / logical relationships, and various settings / configurations, the symbols, forms, times, trigger conditions, stop conditions, ranges, users, permissions, and management mechanisms of the input and output are different. Therefore, the design schemes are different, and the selected virtual test logics are naturally different. The corresponding relationship between the design scheme and the virtual test logic can be to manually complete the screening / matching of the virtual test logic through documents, system functions, operation pages, etc., adjust / develop the virtual test logic according to the design scheme, or can be to complete the above screening / matching / adjustment and other work through artificial intelligence.

[0061] In this embodiment, the virtual test module is used to perform a virtual test on the design scheme to be tested according to the selected virtual test logic, and obtain a test result.

[0062] The virtual test module is also used to set at least one evaluation item among logic, computing power, display, permission, security, efficiency, load, capacity, and time under the virtual test of the virtual test logic according to the design scheme to be tested, perform a virtual test on the design scheme to be tested according to the selected virtual test logic, obtain the results of each evaluation item, set corresponding levels / score values respectively according to the results of different evaluation items, and calculate the comprehensive level / score value of the design scheme to be tested in the virtual test according to the weights of each evaluation item in the virtual test, so as to give a test result.

[0063] Among them, the logic can be that the input and output meet the expectations, and this expectation can be set manually or set through other calculation methods; the computing power / load / capacity can be set through experience, simulation operation, or referring to the arranged design documents; the display can be determined according to the user's requirements / design documents or conventional interaction forms; the permissions / time can be determined according to the product design, organizational structure, business process, security system, etc.; the security / efficiency can be determined according to the business requirements, user requirements, or design documents.

[0064] For each evaluation item, different results are set corresponding to different levels / score values. For example: when the evaluation item is time, it can be set that the time is less than 0.1s is 10 points, the time is between 0.1 - 0.2s is 8 points, the time is between 0.2 - 0.5s is 5 points, the time is between 0.5 - 1s is 3 points, and the time is more than 1s is 0 points. In this way, according to the result of the time, combined with the specific setting, the level / score value of the evaluation item of time can be obtained. Combining the levels / score values and weights of different evaluation items, the comprehensive level / score value of the virtual test is obtained.

[0065] The calculation formula of the comprehensive level / score value includes:

[0066]

[0067] Among them, F is the comprehensive level / score of the design scheme to be tested; n is the total number of evaluation items; p i represents the i-th evaluation item; T(p i ) is the weight of the i-th evaluation item; S(p i ) is the level / score corresponding to the result of the i-th evaluation item.

[0068] This embodiment can further give the test result of the virtual test based on the comprehensive level / score of the design scheme to be tested in the virtual test. For example, a passing threshold for the virtual test can be set. If the comprehensive level / score of the design scheme to be tested is greater than the preset passing threshold, the test result of the design scheme to be tested is a pass. A single-item passing threshold for a certain evaluation item of the design scheme can also be set. If the level / score corresponding to the result of this evaluation item is lower than the single-item passing threshold of this evaluation item, the test result of the design scheme to be tested is a fail. The test result also includes the results of each evaluation item. The virtual test is divided into different evaluation items, and each evaluation item will determine the corresponding level / score according to its own result. It can be judged whether this evaluation item is prone to problems based on the level / score of the evaluation item. For example: if the level / score of the evaluation item of safety is only 2 points, it means that the safety of the design scheme is not very good and there may be safety risks.

[0069] Specifically, the method for determining the test result includes: judging whether the comprehensive level / score is greater than a preset passing threshold to obtain a first judgment result. If the first judgment result is negative, the test result is that the test fails. If the first judgment result is positive, then judge whether the level / score corresponding to each evaluation item is higher than the single-item passing threshold corresponding to the evaluation item to obtain a second judgment result. If the second judgment result is positive, that is, the level / score corresponding to all evaluation items is higher than the single-item passing threshold corresponding to the evaluation item, then the test result is that the test passes; if the second judgment result is negative, that is, the level / score corresponding to at least one evaluation item is not higher than the single-item passing threshold corresponding to the evaluation item, then the test result is that the test fails. Thus, it is possible to obtain the test result by comprehensively considering the level / score corresponding to the single evaluation item and the comprehensive level / score. Only when the comprehensive level / score is greater than the preset passing threshold and the level / score corresponding to all evaluation items is higher than the single-item passing threshold corresponding to the evaluation item, is the test considered to pass, greatly improving the strictness of the test and ensuring the effectiveness of the design solution passing the test. Among them, each evaluation item corresponds to a single-item passing threshold. For each evaluation item, the level / score corresponding to the evaluation item is compared with the single-item passing threshold corresponding to the evaluation item. The single-item passing thresholds corresponding to each evaluation item can be the same or different.

[0070] The virtual test system of this embodiment can also prompt system risk points and risk scenarios according to the test result. Specifically, the virtual test system of this embodiment is also used to determine the problem points and problem scenarios prone to errors according to the test result of the virtual test, that is, the level / score of each evaluation item, and prompt relevant system risk points and risk scenarios according to the type, level, impact and difficulty of solution of the error, so that users can better understand the design solution.

[0071] The virtual test system of this embodiment can also include an adjustment module. The adjustment module is used to determine the reason why the design solution to be tested fails the test according to the test result of the design solution to be tested, and adjust and optimize the design solution to be tested based on the reason. For example, if the test result of the autonomous driving software in the virtual test is that the test fails, and the test result indicates that it is because the safety test and resource occupancy test do not pass, then it is necessary to adjust and optimize the design solution of the autonomous driving software, improve the relevant parts (i.e., the parts related to safety and resource occupancy), and retest until the test passes.

[0072] After a project with problems / risks is discovered through virtual testing, prompting the user about the existing problems and their specific manifestations can help the user put forward requirements for improving / optimizing the design scheme. Similarly, the design scheme can be adjusted and optimized based on the levels / scores of each evaluation item in the design scheme test. For example: If it is known from the test results that the security score is low, then the security of the design scheme needs to be adjusted and optimized; if the time score is high, then there is no need to optimize the time of the design scheme.

[0073] The virtual testing system provided in this embodiment can conduct virtual testing on the design scheme before the software and system development are completed. Through the virtual testing logic required by the design scheme and the application scenario, multi-angle, multi-path, multi-order, and multi-level virtual testing can be performed on the design scheme. Moreover, the design scheme can be adjusted and optimized according to the test results, greatly accelerating the testing process, improving the testing efficiency, and reducing the testing cost.

[0074] For example, if a user wants to develop an autonomous driving software, they can first conduct a scheme design to obtain the design scheme of the autonomous driving software. The design scheme may include: goals, requirements, functions, structures, development environments, interface designs, inputs, outputs, data metrics, configurations, etc. Then, use the virtual testing system of this embodiment to conduct virtual testing on the design scheme. Specifically, first select the virtual testing logic suitable for this design scheme according to the design scheme and the virtual testing model. The virtual testing logic may include core operations, queries, classifications, fuzzy matches, functional module interface style layouts, correct integrity of data displays, data list paging, deletions, removals, additions, configurations, linkage processes, functional implementation of the entire software system, rationality / stability / scalability / reusability of the architecture, usability / aesthetics / consistency / friendliness of the user interface, applicability / security / performance optimization of the database, interfaces / inputs / outputs / independence of modules, complexity of data structures and algorithms / time resource consumption / space resource consumption, etc., and conduct virtual testing on the design scheme based on the selected virtual testing logic to obtain the test results.

[0075] After testing the design solution, the virtual test system of this embodiment can also test code, databases, interfaces, system environments, input data, etc. Specifically, it can refer to the combination of virtual testing and conventional testing, or the application of virtual testing methods to further test the actual situation of the product. It can also refer to continuing to test specific contents such as actual code and databases after completing the testing of the design solution, including: functional testing of roles, functions, permissions, organizations, processes, steps, links, settings, and configurations; testing of data addition, deletion, modification, query, verification, validation, and logical calculation; testing of data classification / clustering, screening, exclusion, combination, statistics, arrangement, model algorithms, systems, databases, storage, prediction, judgment, input, output, processing, analysis, presentation, processing, design, combination, query, screening, exclusion, model processing, data mining, statistics, regression, virtual, prediction, validation, review, etc.

[0076] The virtual test system of this embodiment further includes a code acquisition module and a standard code library. The standard code library includes standard codes suitable for different design solutions. The code acquisition module is used to directly obtain the corresponding standard code from the standard code library according to the first design solution when the standard code library includes the standard code of the first design solution. When the standard code library does not include the standard code of the first design solution, the most suitable standard code for the first design solution is selected from the standard code library according to the first design solution, and then the most suitable standard code for the first design solution is modified based on the difference between the first design solution and the most suitable standard code for the first design solution to obtain the standard code corresponding to the first design solution.

[0077] The standard code library of this embodiment includes several standard codes, and each standard code has a corresponding suitable design solution. Then, when there is a design solution, first check in the standard code library whether there is a standard code that completely suits this design solution. If there is, use it directly. If not, find the closest standard code, and then modify the closest standard code according to the difference between the design solution corresponding to the closest standard code and the current design solution.

[0078] Specifically, the standard code library can include complete standard codes corresponding to different design solutions, or standard codes corresponding to different functional modules. When the standard code library includes complete standard codes corresponding to different design solutions, the corresponding standard code is directly obtained from the standard code library according to the design solution. When the standard code library includes standard codes corresponding to different functional modules, the standard codes suitable for each functional module are selected from the standard code library according to the functional modules in the design solution, and then the standard codes of each functional module are integrated to obtain a complete standard code suitable for the design solution.

[0079] The multiple standard codes stored in the standard code library of this embodiment, which are suitable for different design schemes / different functional modules, can be set manually by experts, or obtained through data statistics / data analysis, or obtained through information reorganization / big data analysis, or obtained through artificial intelligence deep learning / optimization, and can be continuously accumulated and optimized during use.

[0080] If there is a corresponding standard code for the design scheme, the corresponding standard code can be directly obtained from the standard code library; if there is no corresponding standard code for the design scheme, the most suitable standard code can be selected from the standard code library by analyzing and comparing according to the design scheme, and then considering the differences between the design scheme and the existing most suitable standard code, the most suitable standard code can be appropriately modified so that the modified standard code is suitable for the design scheme. The modification can be completed manually by professionals or automatically by the system relying on artificial intelligence through self-learning.

[0081] The virtual test system of this embodiment further includes a code verification module, and the code verification module is used to verify whether the standard code conforms to the design scheme. The verification process includes: generating a system / product through the standard code, and applying the system / product to test whether it conforms to the design scheme. Specifically, it is judged whether the standard code conforms to the design scheme in terms of input, output, function, efficiency, and resource occupation. If it does not conform, the user is prompted to adjust the standard code.

[0082] For the standard code directly obtained from the standard code library, or the standard code obtained by combining the standard codes of different functional modules obtained from the standard code library, or the standard code obtained by modifying the standard code obtained from the standard code library, or the standard code written by the user himself, the code verification module can verify whether the standard code conforms to the design scheme and whether it can conform to the design scheme in terms of input, output, function, efficiency, resource occupation, etc. If it does not conform, the user can also be prompted to make adjustments.

[0083] The virtual test system of this embodiment can also analyze the elements of the design scheme and their relationships from the design scheme, establish connections in different dimensions of the elements according to the relationships, and facilitate display according to the requirements of the user. The elements of the design scheme include at least one of the following: links, nodes, data relationships, causality, outputs, triggers, inputs, and outputs.

[0084] Each element is related through mutual relationships, and through these relationships, it can be expanded to form a system with multiple dimensions and multiple levels. This system can be conveniently presented to users from different dimensions, making it easier for users to accept, understand, and apply, and easier to master the mutual relationships and influences between different elements. During the analysis process, relevant elements and mutual relationships can be recorded / annotated, and relevant corresponding relationships can be established. This can be to mark the elements and mutual relationships for easy use, or to use the elements and relationships themselves as marks. Through the connections between different elements and mutual relationships, a knowledge / information system with different dimensions and different levels can be established.

[0085] The virtual test system of this embodiment further includes a user interaction module. The user interaction module is used to obtain signals sent by one or more users and present the feedback information to the users. That is, the user interaction module obtains the signals sent by the users and presents the feedback information to the users. The feedback information includes the information required by the users to be presented, which can be test results, design schemes, standard codes, relevant content of virtual test logic, elements / mutual relationships, or other information that can be presented to the users during the use of the virtual test system. Based on this, the user interaction module can include an information acquisition device and an information presentation device. The information acquisition device is used to obtain the signals sent by the users to the virtual test system, and the information presentation device is used to present the feedback information to the users by the virtual test system.

[0086] In this embodiment, users can send signals to the virtual test system through voice, gestures, actions, expressions, sight, smell, body temperature, body indicators / parameters / reactions, brain waves, neurotransmitters, etc., or send signals to the virtual test system by operating a tablet computer, mobile phone, mouse, keyboard, smart handle, remote control, smart wearable device, etc., or send signals to the virtual test system through other suitable means. Therefore, the information acquisition device can include at least one of a voice acquisition device, a gesture acquisition device, a body movement acquisition device, an expression acquisition device, a body signal acquisition device, a smart wearable device, a tablet computer, a mobile phone, a mouse, a keyboard, a smart handle, and a remote control, and can also include other devices suitable for acquiring signals sent by users.

[0087] The information presentation device can be at least one of a display screen, smart glasses, headphones, speakers, hearing aids, and brain-computer interfaces, or other suitable devices.

[0088] In this embodiment, the user interaction module is preferably an AR / VR glasses, a human-machine interface, a keyboard, a mouse, a tablet computer, a mobile phone, a smart handle, a remote control, a naked-eye 3D device, a 3D projection device, a display screen, headphones, a wearable device. Among them, the human-machine interface can be a brain-computer interface or other types of human-machine interfaces. The user interaction module can also include a device for scanning / receiving / intervening brain waves or nerve information / biological electrical signals.

[0089] In the virtual test system of this embodiment, the user interaction module can be based on at least one of visual signals, auditory signals, tactile signals, olfactory signals, gustatory signals, texture signals, temperature signals, humidity signals, and human-machine interface signals. Through different combinations, the feedback information can be presented to the user in at least one form of points, lines, graphics, images, planes, curved surfaces, three-dimensional graphics, three-dimensional images, marks, symbols, codes, numbers, texts, colors, sounds, touches, smells, temperatures, and formulas. A coordinate system can be established in the established virtual space / system, and multi-dimensional information coordinates are formed by spatial positions, visual signals, auditory signals, tactile signals, olfactory signals, gustatory signals, texture signals, temperature signals, humidity signals, and human-machine interface signals, etc., to display relevant information to the user.

[0090] The user can also interact with other users through the user interaction module, or perform at least one of the following operations on the presented content, elements, relationships, or their combinations: annotation, insertion, query, extraction, comparison, translation, analysis, copying, merging, backward movement, conversion, connection, calculation, sorting, forward movement, deletion, rising, design, search, replacement, statistics, drilling down, deduction, addition, modification, annotation, and conversion.

[0091] The user interaction module can also present the feedback information to the user through auditory perception signals and tactile perception signals.

[0092] The auditory perception signal refers to representing information through at least one of the characteristics of sound frequency, rhythm, melody, interval, azimuth, distance, size, height, length, and timbre, enabling the user to perceive the information through this signal. For example, spatial azimuth information can be transmitted to the user through sound. Since the transmission characteristics of sound waves from a sound source to a specific azimuth can be expressed as a function data set, this function data set representing the transmission characteristics of sound waves can be used to process audio signals, making the audio signals reflect the transmission characteristics of sound waves from the sound source to that azimuth. When this processed audio signal is converted into sound by a playback device, the sound exhibits the transmission characteristics of sound waves from the sound source to that azimuth, enabling the user to feel the virtual sound source spatial azimuth. The specific azimuth can include the direction, position, height, etc. of the sound.

[0093] The tactile perception signal refers to information represented by contacting the position of the user's body and at least one of the contact head type, shape, contact area, movement, temperature, and pressure, enabling the user to perceive the information through this signal. Different information can be represented by contacting different positions of the user's body. For example, different positions such as the user's head, hands, fingers, palms, backs of hands, forearms, upper arms, chest, back, legs, feet, etc. represent different orientation information or various other information. In addition to representing information by contacting the position of the user's body, the tactile perception signal can also represent information by at least one of the contact head type, shape, contact area, movement, temperature, and pressure when contacting the user's body. The movement of contacting the user's body can include: light touch, sliding, rotation, clicking, pressing, etc.

[0094] The virtual test system of this embodiment can also use a virtual space to display the feedback information. For example, the goals, requirements, functions, structures, development environments, interface designs, inputs, outputs, data metrics, configurations, etc. of the design scheme can be represented by different points, and their mutual relationships / connections are represented by lines. It can also include marks / words / symbols to represent other information. After clicking and selecting, the virtual test logic corresponding to this information can be specifically displayed. Different colors can also be used to represent different information. For example, a high score in a certain dimension can be represented by green / a low score by red, etc. The user can select the information to be displayed according to their own needs, such as only displaying the interface design without displaying the specific structure. The user can also perform various operations, such as modification, addition, etc. The modification can be modifying the content such as text / images, etc., or modifying the mutual relationship, such as removing the connection line between a certain requirement and a certain application scenario and replacing it with a connection to another application scenario; operations can also be performed on the set of content / elements / mutual relationships in the virtual space, such as directly deleting / transferring a certain requirement, and the inputs / outputs, etc. under this requirement are also deleted / transferred together.

[0095] The virtual test system of this embodiment can also present the design scheme, application scenario, and virtual test logic in one or more display orders, levels, and dimensions. After being selected by the user, the feedback information is presented to the user according to this order, level, and dimension. For example, in the virtual space of the virtual test deduction result, the user can choose to present the deduction result in different orders, levels, and dimensions.

[0096] Of course, the information feedback above can be presented directly and in detail, or only a part can be presented in detail while the other part is presented as a title or label, and then presented in detail after the user sends a signal indicating a desire to view through the user interaction module, reducing the complexity of the presented content. The signal sent by the user through the user interaction module indicating a desire to view can be that the user gazes at the title of relevant content for a certain period of time, which means the user wants to view the content under that title. The user interaction module obtains the signal sent by the user based on eye tracking, and the virtual test system gives the feedback information, that is, the content under that title. At the same time, the relevant content associated through elements and their interrelationships in the content under that title can also be presented. When presenting, different signals can be used to reflect different elements or interrelationships. For example, different elements can be presented through different sounds, images, colors, and brainwaves.

[0097] The virtual test system of this embodiment may further include a permission management module for managing the usage permissions of different users to prevent privacy leakage or disclosure of secrets. Different usage permission levels can be set for different users and / or different design schemes. The usage permissions may include reading, copying, viewing, converting, analyzing and calculating, processing, comparing, editing, storing, etc. of content information. The usage permission levels can be set manually or through artificial intelligence analysis.

[0098] The virtual test system of this embodiment may not include a database module. In this case, the virtual test database can be deployed on a network server or a cloud server, and the information acquisition module obtains the information in the virtual test database from the network server or the cloud server for the virtual test system to use. It can also be a virtual test database provided by other service providers, and this embodiment does not limit this.

[0099] Embodiment 2:

[0100] This embodiment is used to provide a usage method of the virtual test system as described in Embodiment 1. As Figure 1 shown, the method includes:

[0101] S1: Establish a virtual test database, where the virtual test database includes a virtual test model and virtual test logics for different design schemes. The virtual test model is used to select a virtual test logic suitable for the design scheme according to the design scheme. The elements of the virtual test model include at least one of the following: target, user, scenario, interface system environment and interface conditions, data organization, process, link, control, intervention, evaluation, prediction, interaction, and interaction; the virtual test logic is used to perform virtual tests on the design scheme;

[0102] S2: Obtain the design scheme to be tested and its application scenario;

[0103] S3: Select the virtual test logic suitable for the design scheme according to the design scheme to be tested and the virtual test model;

[0104] S4: Perform virtual testing on the design scheme according to the selected virtual test logic to obtain a test result.

[0105] The usage method of the virtual test system may further include:

[0106] Adjust and optimize the design scheme according to the virtual test result of the design scheme;

[0107] Obtain the corresponding standard code from the standard code library according to the design scheme;

[0108] Verify whether the standard code conforms to the design scheme.

[0109] The usage method of the virtual test system provided in this embodiment, the virtual test system includes: a database module, an information acquisition module, an analysis module, and a virtual test module. The usage method of the virtual test system includes: establishing a virtual test database; obtaining the design scheme to be tested and its application scenario; selecting the virtual test logic suitable for the design scheme to be tested according to the design scheme to be tested and the virtual test model; performing virtual testing on the design scheme to be tested according to the selected virtual test logic to obtain a test result. This embodiment applies virtual technology to the testing process. Before the software and system development are completed, virtual testing is performed on the design scheme. The design scheme is subjected to multi-angle, multi-path, multi-sequence, and multi-level virtual testing through the virtual test logic required by the design scheme and the application scenario. The design scheme is adjusted and optimized according to the test result. It is also possible to obtain the corresponding standard code from the standard code library according to the determined design scheme and verify whether the standard code conforms to the design scheme, which can speed up the test process, improve test efficiency, and reduce test costs.

[0110] The various embodiments in this specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0111] Specific examples are used in this article to elaborate on the principles and implementation manners of the present invention. The descriptions of the above embodiments are only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A virtual testing system, characterized in that: The virtual test system includes: a database module, an information acquisition module, an analysis module and a virtual test module. The information acquisition module is used to obtain the design scheme to be tested and its application scenario; The database module is used to store a virtual test database, which includes a virtual test model and virtual test logics of different design schemes. The virtual test model is used to select a virtual test logic suitable for the design scheme according to the design scheme. The elements of the virtual test model include at least one of the following: target, user, scenario, interface system environment and interface condition, data organization, process, link, control, intervention, evaluation, prediction, mutual influence, and interaction. The virtual test logic is used to perform virtual testing on the design scheme; the virtual test logic of the design scheme includes the virtual test logic of the functional module or the virtual test logic of the overall system. The virtual test logic of the functional module is used to perform virtual testing on each functional module of the software. Conduct virtual testing, including: at least one virtual testing logic in the core operations, query, classification, fuzzy matching, functional module interface style layout, correctness and integrity of data display, data list page turning, deletion, removal, addition, configuration, and linkage process involved in each module. The virtual testing logic of the overall system is used to conduct virtual testing on the overall system of the software, including: at least one virtual testing logic in the functional implementation of the entire software system, rationality / stability / scalability / reusability of the architecture, ease of use / aesthetics / consistency / friendliness of the user interface, applicability / security / performance optimization of the database, interface / input / output / independence of the module, complexity of the data structure and algorithm / time resource consumption / space resource consumption; The analysis module is used to select a virtual test logic suitable for the design scheme according to the design scheme to be tested and the virtual test model; The virtual test module is used to perform a virtual test on the design solution according to the selected virtual test logic to obtain a test result.

2. The virtual test system according to claim 1, characterized in that: The virtual testing system analyzes the various elements of the design scheme and their relationships from the design scheme, and establishes connections of different dimensions of the elements based on the relationships, so as to display them according to the user's requirements. The elements of the design scheme include at least one of: links, nodes, data relationships, cause and effect, output, trigger, input, and output.

3. The virtual test system according to claim 1, characterized in that: The virtual testing module is also used to set at least one evaluation item among logic, computing power, presentation, authority, security, efficiency, load, capacity, and time under the virtual test of the virtual testing logic according to the design scheme to be tested, set corresponding levels / scores according to the results of different items, and calculate the comprehensive level / score of the design scheme to be tested in the virtual test according to the weight of each evaluation item in the virtual test, so as to give the test results.

4. The virtual test system according to claim 3, characterized in that: The method for determining the test result includes: judging whether the comprehensive level / score is greater than a preset pass threshold to obtain a first judgment result; if the first judgment result is no, the test result is a test failure; if the first judgment result is yes, judging whether the level / score corresponding to each of the assessment items is higher than the single pass threshold corresponding to the assessment item to obtain a second judgment result; if the second judgment result is yes, the test result is a test pass; if the second judgment result is no, the test result is a test failure.

5. The virtual test system according to claim 1, characterized in that: The virtual test system is also used to determine problem points and problem scenarios that are prone to errors based on the test results, and to prompt system risk points and risk scenarios based on the type, level, impact and difficulty of solving the error.

6. The virtual test system according to claim 1, characterized in that: The virtual test system also includes an adjustment module, which is used to determine the reason why the design scheme to be tested failed the test according to the test results, and to adjust and optimize the design scheme to be tested in a targeted manner based on the reason.

7. The virtual test system according to claim 1, characterized in that: The virtual test system further includes a code acquisition module and a standard code library, wherein the standard code library includes standard codes suitable for different design schemes; the code acquisition module is used to directly acquire the corresponding standard code from the standard code library according to the first design scheme when the standard code library includes the standard code of the first design scheme, and to select the standard code most suitable for the first design scheme from the standard code library according to the first design scheme when the standard code library does not include the standard code of the first design scheme, and then modify the standard code most suitable for the first design scheme based on the difference between the first design scheme and the standard code most suitable for the first design scheme to obtain the standard code corresponding to the first design scheme; The virtual test system also includes a code verification module, which is used to verify whether the standard code complies with the design plan, specifically to determine whether the standard code complies with the design plan in terms of input, output, function, efficiency, and resource occupancy. If not, the user is prompted to adjust the standard code.

8. The virtual test system according to claim 1, characterized in that: The virtual test system further comprises a user interaction module, which is used to obtain signals sent by one or more users and present feedback information to the users.

9. A method for using the virtual test system according to claim 1, characterized in that: The method comprises: A virtual test database is established, wherein the virtual test database includes a virtual test model and virtual test logics of different design schemes, wherein the virtual test model is used to select a virtual test logic suitable for the design scheme according to the design scheme, wherein the elements of the virtual test model include at least one of: a target, a user, a scenario, an interface system environment and an interface condition, a data organization, a process, a link, a control, an intervention, an evaluation, a prediction, a mutual influence, and an interaction, wherein the virtual test logic is used to perform a virtual test on the design scheme; wherein the virtual test logic of the design scheme includes a virtual test logic of a functional module or a virtual test logic of the overall system, wherein the virtual test logic of the functional module is used to perform a virtual test on each functional module of the software; Test, including: at least one virtual test logic in the core operations, query, classification, fuzzy matching, functional module interface style layout, correctness and integrity of data display, data list page turning, deletion, removal, addition, configuration, and linkage process involved in each module. The virtual test logic of the overall system is used to conduct virtual testing on the overall system of the software, including: at least one virtual test logic in the functional implementation of the entire software system, rationality / stability / scalability / reusability of the architecture, ease of use / aesthetics / consistency / friendliness of the user interface, applicability / security / performance optimization of the database, interface / input / output / independence of the module, complexity of the data structure and algorithm / time resource consumption / space resource consumption; Obtain the design solutions to be tested and their application scenarios; Selecting a virtual test logic suitable for the design scheme according to the design scheme to be tested and the virtual test model; The design scheme is virtually tested according to the selected virtual test logic to obtain a test result.

Images