System testing method and apparatus, electronic device, and storage medium

Abstract

The disclosure provides a system testing method and device, electronic equipment and storage medium, relates to the technical field of computers, in particular to the field of artificial intelligence, big data and knowledge graph. The specific implementation scheme is: receiving a test request, wherein the test request is used for testing a plurality of modules contained in a target system; determining at least one test node corresponding to the test request, wherein each test node is composed of at least one target node, and the target node is not detachable; determining an execution path of the at least one test node based on the execution cost of all target nodes contained in the at least one test node; and testing the at least one test node according to the execution path to obtain a test result corresponding to the test request. The disclosure solves the problems of high test threshold and low efficiency of the system testing method in the related art.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to artificial intelligence, big data and knowledge graph. Specifically, the present disclosure relates to a system testing method and device, an electronic device and a storage medium. BACKGROUND

[0002] The evaluation and output of the conventional system testing scheme mainly depend on the evaluation and output of the field experts manually. However, the system involved in the above scheme usually contains multiple modules with complex relationships, and the testing effect depends on the field experience of the experts, which leads to high testing threshold, great limitation and low testing efficiency. SUMMARY

[0003] The present disclosure provides a system testing method and device, an electronic device and a storage medium.

[0004] According to an aspect of the present disclosure, a system testing method is provided, comprising: receiving a testing request, wherein the testing request is used for testing a plurality of modules contained in a target system; determining at least one testing node corresponding to the testing request, wherein each testing node is composed of at least one target meta-node, and the target meta-node is not splittable; determining an execution path of the at least one testing node based on the execution cost of all target meta-nodes contained in the at least one testing node; and testing the at least one testing node according to the execution path to obtain a testing result corresponding to the testing request.

[0005] According to another aspect of the present disclosure, a system testing device is provided, comprising: a receiving module configured to receive a testing request, wherein the testing request is used for testing a plurality of modules contained in a target system; a node determining module configured to determine at least one testing node corresponding to the testing request, wherein each testing node is composed of at least one target meta-node, and the target meta-node is not splittable; a path determining module configured to determine an execution path of the at least one testing node based on the execution cost of all target meta-nodes contained in the at least one testing node; and a testing module configured to test the at least one testing node according to the execution path to obtain a testing result corresponding to the testing request.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0007] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to perform the above method.

[0008] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method according to the above.

[0009] It should be understood that the details described in this section are not intended to identify key or critical features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them:

[0011] Figure 1 is a flowchart of a system test method according to the first embodiment of the present disclosure;

[0012] Figure 2 is a directed graph of a test activity according to an embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram of a test activity network according to an embodiment of the present disclosure;

[0014] Figure 4 is a schematic diagram of test activity coverage optimization according to an embodiment of the present disclosure;

[0015] Figure 5 is a schematic diagram of a system test device according to the first embodiment of the present disclosure;

[0016] Figure 6 is a block diagram of an electronic device for implementing a system test method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0018] With the continuous development and improvement of the form, architecture and business complexity of Internet products, the challenges of Internet product quality assurance are becoming more and more serious, which are mainly reflected in quality and efficiency. First of all, in terms of quality, the range of users affected by each Internet product is extremely large, and each user's use of the product and user data is extremely different. In order to better ensure the quality, a large amount of manpower needs to be consumed in advance to verify and test various possible application scenarios. However, Internet products attach great importance to iteration efficiency, and if the efficiency cannot keep up, the product may lose the reason and possibility to exist. As a result, the time left for testing is seriously insufficient and is often compressed by various circumstances. This obvious difference from the extreme pursuit of efficiency of traditional software products brings greater challenges to a large number of product tests, and efficiency and quality are both wanted, but often cannot be both good.

[0019] A big reason for this contradiction is that when facing a product system with multiple modules, complex topological relationship and complex functions, it is impossible to accurately assess how much range, how much function, performance and other index types will be affected by each change, resulting in meaningless regression of a large number of functions, performance and other functions that may not be affected at all, on the one hand, wasting valuable test time, on the other hand, occupying the time that should be used for more comprehensive testing of the actual affected functions. In the actual operation process, in order to alleviate this problem, the general approach is to make judgments based on historical situations, code changes and manual experience, etc. to determine the scope of the system to be tested and the related test scheme, and then to build the environment of the related modules and prepare the test data and test implementation scheme according to the experience. This can still be coped with in the case of relatively simple system and not very urgent iteration, but in the face of increasingly complex system and extreme pursuit of test efficiency, the disadvantages of this manual evaluation method are becoming increasingly obvious:

[0020] First of all, with the continuous complexity of business and the division of service cloud native and micro service architecture, the requirement for the ability of the domain test experts to evaluate and determine the test range is getting higher and higher, and the demand for manpower is also getting bigger and bigger. This cannot be met in the context of improving efficiency in the Internet industry, resulting in many business demands cannot be put online quickly, losing many business development opportunities.

[0021] Secondly, this expert evaluation scheme is relatively static, while the online architecture, data, risk and code changes are changing in real time. It is impossible to consider the changing factors in the evaluation scheme in a timely manner with a relatively static evaluation method, which inevitably brings some omissions and unpredictable risks, thereby causing great experience and loss of benefits to users and business.

[0022] Finally, although the accumulation of such expert evaluation experience can also be passed down in the form of documents, etc., a larger part still exists in the expert's knowledge system, and once personnel changes and business adjustments occur, it is a greater challenge for the students who take over, and the business also faces greater risks.

[0023] To solve these problems, artificial intelligence algorithms are used instead of domain experts to sort out the factors affected by each iteration change and the possible impact range, so that in the system-level testing process, the algorithm gives a system-level testing plan based on these scenario inputs, which tests which business modules, prepares what type of data and test scenarios, and follows what order to test. The system-level testing plan can well reduce the dependence on domain experts, while the relevant factors can be dynamically included in the evaluation model, and the model can also evolve and iterate based on historical problem situations, making it easier to accumulate evaluation knowledge.

[0024] There are some related cases in the current industry that apply artificial intelligence algorithms to the testing field, such as the generation of related unit test cases in unit testing, the selection of related interface test cases in functional testing, and the evaluation of test data effectiveness in performance testing. The evaluation effect data of the related algorithms is relatively in line with expectations, and with the continuous evolution of the algorithm, the related evaluation effect has been getting closer to and even exceeding the evaluation effect of experts. However, in the field of system-level testing, due to the complexity of related problems and the large impact of related risks, there are fewer related cases of implementing system-level testing evaluation and landing based on intelligent algorithms.

[0025] In view of the above situation, the present disclosure uses intelligent algorithms to evaluate and output the testing plan within the system testing scenario based on code, risk, business topology, etc., thereby improving the efficiency of system-level testing.

[0026] According to an embodiment of the present disclosure, the present disclosure provides a system testing method, as shown in Figure 1 The method can include the following steps:

[0027] Step S102, receiving a test request, wherein the test request is used to test a plurality of modules contained in a target system.

[0028] The test request in the above step can be a request to test a plurality of modules contained in a target system, which can be a requirement implementation, code change, online service change, adjustment, etc. of the target system. The target system here can be a product system with multiple modules, complex topology, and complex functions.

[0029] In an alternative embodiment, when the tester needs to perform system-level testing on the target system, the tester can operate on the pre-provided operation interface, select the target system to be tested, and generate a corresponding test request, which can be forwarded to the corresponding test platform for testing.

[0030] In step S104, at least one test node corresponding to the test request is determined, wherein each test node is composed of at least one target meta-node, and the target meta-node is not detachable.

[0031] Overall, the execution efficiency of system-level testing is mainly related to the range of business modules covered by the formulated test scheme and the time required for executing a single module single-type test. It can be considered that the system-level test scheme is a directed graph, as shown in Figure 2 The efficiency from the starting point (beginning) to the ending point (end) is related to the number of nodes in the graph and the execution path, and is also related to the execution efficiency of a single node.

[0032] As shown in Figure 2 The test activities can be a single type / single node test in system-level testing (i.e., the test node described above). The resources and time required for executing these single test activities are different and may be subject to external resource dependency constraints. Therefore, the system-level test scheme is first to determine how many test activities need to be covered, and then to determine the execution order of these activities. Many times, if an unnecessary activity is introduced, the additional cost to the overall scheme execution efficiency may not only be the time consumption of the execution of a single activity, but also may be caused by the dependency of some external resources, resulting in the failure of the entire scheme execution.

[0033] In an alternative embodiment, the minimum set of test activities to be covered by the current test can be determined according to the input information such as the requirement implementation, code change, online service change and adjustment of the target system contained in the test request, combined with an artificial intelligence strategy algorithm. This ensures that unnecessary test activities are not introduced and time is wasted. Then, by optimizing the implementation architecture of the test activities, defining meta-activities, and splitting a test activity into multiple non-detachable meta-activities (i.e., the target meta-node described above), the related input, output and constraint relationship of different meta-activity combinations are described through meta-information. This ensures that the internal dynamic arrangement of a single test activity can be performed, and the combination of related meta-activities can meet the requirements of system-level testing to form a directed graph of test activities as shown in Figure 2 The meta-information here can be specific change information, such as module name change, code line number, function complexity, etc., but is not limited thereto. The constraint relationship here can be a pre-post dependency constraint relationship required for executing a test activity.

[0034] Step S106: Determine the execution path of at least one test node based on the execution cost of all target meta nodes contained in at least one test node.

[0035] The execution cost in the above steps can be a value comprehensively evaluated by combining historical execution information such as time consumption, dependencies, stability, and the number of issues discovered. It represents the input and potential benefits of performing the test activity, and can specifically include time costs and resource costs. The execution path can be the execution order of at least one test node, including both parallel execution and sequential execution.

[0036] In one optional embodiment, the test activity network is a directed graph that reflects the combination relationships between test activities and the flow of data generated during the testing process. Since the execution time and order of different nodes may lead to waiting for related dependent resources, dynamic metrics such as execution time, number of issues discovered, and resource consumption costs can be added to meta-activities based on historical execution information. These metrics serve as the execution cost of a single test activity, which can be considered the weight of the connection between two activities in the aforementioned directed graph. This allows for the dynamic optimal selection of execution paths that satisfy meta-information constraints, thereby achieving dynamic optimization of the overall test activity execution.

[0037] For example, as Figure 3 Taking the test activity network shown as an example, if each test activity is evaluated based on the aforementioned time consumption, dependencies, and stability, for simplicity, let's assume the cost of each node is 1 unit. If the selected modules are executed sequentially, the final execution time cost would be 8 units. However, if the execution order is dynamically adjusted based on optimization algorithms and dynamic values, the execution order can be changed. Test activities without dependencies can be executed in parallel, while those with dependencies can be executed sequentially. That is, test activities 1, 2, 4, and 5 can be executed in parallel, followed by test activities 3, 6, and 8, and finally test activity 7. Therefore, the final execution cost is only 3 units.

[0038] Step S108: Test at least one test node according to the execution path to obtain the test results corresponding to the test request.

[0039] In one alternative embodiment, each test activity can be executed according to the execution path, and the final test result can be obtained after all test activities have been executed. For example, as... Figure 3 The execution path of the test activity network shown is that test activity 1, test activity 2, test activity 4 and test activity 5 are executed in parallel, then test activity 3, test activity 6 and test activity 8 are executed, and finally test activity 7 is executed. Therefore, each test activity can be executed according to this execution path.

[0040] In this embodiment, upon receiving a test request, at least one test node corresponding to the test request can be identified. The test node can be divided into indivisible target meta-nodes. Based on the execution cost of all target meta-nodes, an execution path for at least one test node is determined. Then, the at least one test node is tested according to this execution path to achieve the purpose of system testing. It is noteworthy that when system testing is required, the user only needs to send a test request; no additional manual operation is required. At least one test activity is the minimum set required for the test request, avoiding the introduction of unnecessary test activities and wasting time. Furthermore, since the execution path is determined based on the execution cost of the meta-nodes divided for each test node, there is no need to execute test nodes according to a fixed path. This lowers the testing threshold and improves the testing efficiency of the system, thereby solving the problem of high testing threshold and low efficiency in related technologies.

[0041] In the above embodiments of this disclosure, determining at least one test node corresponding to a test request includes: acquiring data corresponding to the test request based on a preset association relationship, wherein the preset association relationship is used to characterize the association relationship between different data and different modules; acquiring knowledge information corresponding to the test request, wherein the knowledge information is used to characterize the association relationship between different data and test meta nodes; determining the target meta node and the meta information of the target meta node corresponding to the data based on the knowledge information, wherein the meta information is used to characterize the mapping relationship between the input attributes and output attributes of the target meta node; and combining the target meta nodes corresponding to the data based on the meta information of the target meta node to obtain at least one test node.

[0042] Test activity coverage optimization mainly involves defining the test activities that need to be covered in the current test scenario. Different scenarios are based on different test objectives, so the test activities that need to be covered are different, and the test results are also different. To achieve optimal test activity coverage, it is necessary to assess the scope and type of modules that may be affected by the current changes (code adjustments, machine replacements, scaling up or down).

[0043] Some of these judgments of the range of influence can be described based on fixed association, such as based on code / module call relationship, the number of upstream and downstream modules that need to be evaluated for a certain type of change in a piece of code can be easily analyzed through some white box / trace technical means. Some are not so direct, such as the adjustment of a module machine CPU will affect the performance of modules in a large range. To evaluate this range, in addition to the aforementioned call association, it is also necessary to refer to the impact of such environmental resource adjustment on the performance of a single module and the capacity buffer between different modules in the entire call chain, such as if the adjustment of CPU is not a factor in the performance of the current module, then there is no need to continue to circumscribe the test range based on the call relationship of the node, if although it affects the performance of the module, but this module is a bottleneck point in the entire link, then there is no need to expand the test range if the performance of the module is improved within a certain range, if it is determined that there will be an impact, which type of test scheme can better evaluate the impact.

[0044] The data in the above steps can be some module indicator data, code change data, etc. For example, 100 lines of code are changed, and 10 logics are added to the core interface. The knowledge information can be the knowledge of relevant field experts.

[0045] In an optional embodiment, as shown in Figure 4 , such fixed indicators and associations can be sorted out, data in the data space can be constructed through data collection and unified management, knowledge in the upper knowledge space can be output based on the knowledge of field experts, and a single test activity that needs to be covered can be formed based on the deduction of data and knowledge. The test activity will trigger new knowledge deduction and data changes, and the range of all test activities that need to be covered in a single test is finally determined based on the continuous deduction of the above process. Among them, Figure 4 The diamond-shaped box in the figure indicates data, the rectangular box indicates knowledge, the dashed line indicates the evolution path, the solid line indicates the static association, and the pentagram box indicates the change factor.

[0046] Through the above scheme, by combining the data and knowledge information in the target system, the accuracy of the evaluation is continuously iterated to determine the minimum set of test activities that need to be covered in the current test, and the effect of improving the test efficiency is achieved.

[0047] In the above embodiments of the present disclosure, determining the target meta-node corresponding to the data and the meta-information of the target meta-node based on the knowledge information includes: constructing a knowledge model based on the knowledge information; and predicting the data and the plurality of test meta-nodes by using the knowledge model to determine the target meta-node corresponding to the data and the meta-information of the target meta-node.

[0048] In an alternative embodiment, the knowledge network layer is in the middle layer of the combined network, and its main task is to realize the comprehensive integration of various meta-activities by the knowledge output of domain experts, and a test activity network reflects the entire system-level test scheme. Different test activities have different effects on different modules under test, and different test requirements require different test activities. Domain knowledge determines the combination relationship of test activities based on test requirements and scenario information of meta-activities. The knowledge model is composed of information, assertions, and model chooser.

[0049] knowledge Model = (Information, Assertions, Model Chooser),

[0050] Among them, the information describes the relevant scenario information of the test meta-activity and the test requirement, which is the basis for knowledge selection. The assertion describes the relationship between the test requirement and the meta-activity function in a reasoning manner

[0051] , and realizes the logic that after some test activities are taken based on what change / test requirement in the system-level test process, what test activities should be taken next. These relationships have the following forms:

[0052] If (module performance diff activated and pressure meta-activity activated)

[0053] [And / Or (core check activity activated)]

[0054] Then build the old version test environment [in series / parallel build the new version test environment]

[0055] The selector describes the relevant targets, parameters, and methods for selecting specific implementations from meta-activities according to the combination relationship. For example, based on the protocol, pressure type, and result collection requirement, different pressure tools are selected to complete the pressure meta-activity.

[0056] Through the above scheme, the meta-information of the target meta-node is predicted through the knowledge model, which accurately determines the meta-information of the meta-activity, ensures that the internal single test activity can be dynamically arranged, and meets the system-level test requirements.

[0057] In the above embodiment of the present disclosure, the above meta-information at least includes: attribute state of input attribute, attribute state of output attribute and mapping relationship, and the attribute state at least includes: attribute measurement dimension and time-varying function, and the time-varying function is used to represent the change law of the attribute with time.

[0058] In an alternative embodiment, the implementation optimization of test activities is mainly the architecture enhancement to support the above deduction. In order to achieve the optimization of a single activity, the test activity must be decomposed into individual meta-activities which cannot be further decomposed, and then based on the input and output and constraint relationship of the meta-activities, a test activity can be automatically constructed.

[0059] For a specific meta-activity m, let N m be the concept and attribute name corresponding to the meta-activity; A m represent the attribute state set corresponding to it, where the attribute state can be the attribute value of the attribute; R m represent the mapping relationship set on A m × A m , which is used to describe the attribute state change and interaction relationship. Then its general description model can be represented as:

[0060] K m = (N m , A m , R m ),

[0061] where p a is the attribute name of attribute a; d a is the attribute measurement dimension of attribute a, that is, the measurement unit of the attribute; f a is the time-varying function of attribute a, which is used to describe the change law of the meta-activity's own attribute with time, then:

[0062] K a = (p a , d a , f a ),

[0063] If p r is the relationship name of relationship r, is the input attribute state set corresponding to relationship r, is the output attribute state set corresponding to relationship r, and f r is the mapping function between the input attribute and the output attribute, then:

[0064]

[0065] This relationship is between the attributes of the meta-activity itself, based on which multiple meta-activities are automatically combined into a test activity. The association relationship between one test activity and another test activity attribute needs to be associated with the help of high-level knowledge, which is represented as an edge of the test activity coverage network.

[0066] Through the above scheme, the related input, output and constraint relationship of different meta activity combination are described in detail through the meta information, the combination of the related meta activities is ensured to meet the requirements of the system level test, and a directed graph of the test activities as shown in Figure 2 is formed.

[0067] In the above embodiment of the present disclosure, determining the execution path of the at least one test node based on the execution cost of all target meta nodes contained in the at least one test node includes: determining the execution cost of each test node based on the execution cost of the at least one target meta node and the meta information of each target meta node; and determining the execution path based on the execution cost of each test node and the combination relationship of the at least one test node.

[0068] In an optional embodiment, the execution cost of each meta node can be determined in advance, then the execution cost of each test node is determined based on the meta information of the meta activity, and finally the execution path of the test activity can be optimized based on the execution cost, for example, the execution order of the at least one test activity can be dynamically adjusted based on an optimization algorithm and a dynamic value.

[0069] Through the above scheme, the execution path is dynamically optimized based on the execution cost of each test node, the dynamic optimization of the test activity execution of the system level test is realized, and the effect of reducing the test cost is achieved.

[0070] In the above embodiment of the present disclosure, the method further includes: obtaining historical execution information corresponding to the test request; determining the execution index of each meta node based on the historical execution information; and determining the execution cost of each meta node based on the execution index of each meta node.

[0071] In an optional embodiment, the dynamic index data such as execution time, number of problem findings, resource occupation cost can be added to the meta activity based on the historical execution information, as the execution cost of a single meta activity.

[0072] Through the above scheme, the execution cost of the meta activity is determined based on the historical execution information, the dynamic optimal selection of the execution path meeting the constraint of the meta information is realized, and the dynamic optimization of the overall test activity execution is realized.

[0073] According to embodiments of the present disclosure, the present disclosure also provides a system test device, which is used to implement the above embodiments and preferred embodiments, and will not be described again. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and contemplated. As Figure 5 shown, the device includes:

[0074] The receiving module 52 is configured to receive a test request, wherein the test request is used for testing a plurality of modules contained in a target system;

[0075] The node determining module 54 is configured to determine at least one test node corresponding to the test request, wherein each test node is composed of at least one target meta-node, and the target meta-node is not splittable;

[0076] The path determining module 56 is configured to determine an execution path of the at least one test node based on execution costs of all target meta-nodes contained in the at least one test node;

[0077] The testing module 58 is configured to test the at least one test node according to the execution path, to obtain a test result corresponding to the test request.

[0078] In the above embodiments of the present disclosure, the node determining module 54 comprises: a data obtaining unit configured to obtain data corresponding to the test request based on a preset association relationship, wherein the preset association relationship is used to represent an association relationship between different data and different modules; a knowledge obtaining unit configured to obtain knowledge information corresponding to the test request, wherein the knowledge information is used to represent an association relationship between different data and test meta-nodes; a node determining unit configured to determine a target meta-node corresponding to the data and meta-information of the target meta-node based on the knowledge information, wherein the meta-information is used to represent a mapping relationship between input attributes and output attributes of the target meta-node; and a node combining unit configured to combine the target meta-node corresponding to the data based on the meta-information of the target meta-node, to obtain the at least one test node.

[0079] In the above embodiments of the present disclosure, the node determining unit is further configured to: construct a knowledge model based on the knowledge information; and predict the data and the plurality of test meta-nodes by using the knowledge model, to determine the target meta-node corresponding to the data and the meta-information of the target meta-node.

[0080] In the above embodiments of the present disclosure, the path determining module 56 comprises: a cost determining unit configured to determine an execution cost of each test node based on the execution cost of the at least one target meta-node and the meta-information of each target meta-node; and a path determining unit configured to determine the execution path based on the execution cost of each test node and a combination relationship of the at least one test node.

[0081] In the above embodiments of the present disclosure, the device further comprises: an information obtaining module configured to obtain historical execution information corresponding to the test request; an index determining module configured to determine an execution index of each meta-node based on the historical execution information; and a cost determining module configured to determine the execution cost of each meta-node based on the execution index of each meta-node.

[0082] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0083] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0084] As shown in Figure 6 The device 600 includes a computing unit 601 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0085] Various components in the device 600 are connected to the I / O interface 605, including an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; the storage unit 608, such as a magnetic disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0086] The computing unit 601 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs various methods and processes described above, such as the system test method. For example, in some embodiments, the system test method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded onto the RAM 603 and executed by the computing unit 601, one or more steps of the system test method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the system test method by any other suitable means, such as by means of firmware.

[0087] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0088] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0089] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0091] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0092] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0093] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology disclosed in the present disclosure are achieved.

[0094] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A system testing method, comprising: receiving a test request, wherein the test request is used for testing a plurality of modules contained in a target system; obtaining data corresponding to the test request based on a preset association relationship, wherein the preset association relationship is used for representing an association relationship between different data and different modules; obtaining knowledge information corresponding to the test request, wherein the knowledge information is used for representing an association relationship between different data and test meta-nodes; constructing a knowledge model based on the knowledge information; predicting the data and a plurality of test meta-nodes by using the knowledge model, determining a target meta-node corresponding to the data and meta-information of the target meta-node, wherein the meta-information is used for representing a mapping relationship between input attributes and output attributes of the target meta-node; combining the target meta-node corresponding to the data based on the meta-information of the target meta-node, obtaining at least one test node corresponding to the test request, wherein each test node is composed of at least one target meta-node, and the target meta-node is not detachable; determining an execution path of the at least one test node based on execution costs of all target meta-nodes contained in the at least one test node; testing the at least one test node according to the execution path, and obtaining a test result corresponding to the test request.

2. The method of claim 1, wherein, The meta-information at least includes attribute states of the input attributes, attribute states of the output attributes and the mapping relationship, and the attribute states at least include a measurement dimension and a time-varying function of the attributes, and the time-varying function is used for representing a change law of the attributes over time.

3. The method of claim 1, wherein, Determining the execution path of the at least one test node based on the execution costs of all target meta-nodes contained in the at least one test node comprises: determining an execution cost of each test node based on the execution cost of the at least one target meta-node and the meta-information of each target meta-node; determining the execution path based on the execution cost of each test node and a combination relationship of the at least one test node. 4.The method according to claim 3, further comprising: obtaining historical execution information corresponding to the test request; determining an execution index of each meta-node based on the historical execution information; determining the execution cost of each meta-node based on the execution index of each meta-node. 5.A system testing apparatus, comprising: a receiving module configured to receive a test request, wherein the test request is used for testing a plurality of modules contained in a target system; The node determination module comprises a data acquisition unit, a knowledge acquisition unit, a node determination unit and a node combination unit. The data acquisition unit is configured to acquire data corresponding to the test request based on a preset association relationship, wherein the preset association relationship is used to represent the association relationship between different data and different modules. The knowledge acquisition unit is configured to acquire knowledge information corresponding to the test request, wherein the knowledge information is used to represent the association relationship between different data and test meta-nodes. The node determination unit is configured to construct a knowledge model based on the knowledge information, predict the data and the plurality of test meta-nodes by using the knowledge model, determine a target meta-node corresponding to the data and meta-information of the target meta-node, and wherein the meta-information is used to represent the mapping relationship between the input attribute and the output attribute of the target meta-node. The node combination unit is configured to combine the target meta-node corresponding to the data based on the meta-information of the target meta-node, and obtain at least one test node corresponding to the test request, wherein each test node is composed of at least one target meta-node, and the target meta-node is not detachable. The path determination module is configured to determine an execution path of the at least one test node based on the execution cost of all target meta-nodes contained in the at least one test node. The test module is configured to test the at least one test node according to the execution path, and obtain a test result corresponding to the test request.

6. The apparatus of claim 5, wherein, The path determination module comprises: The cost determination unit is configured to determine the execution cost of each test node based on the execution cost of the at least one target meta-node and the meta-information of each target meta-node. The path determination unit is configured to determine the execution path based on the execution cost of each test node and the combination relationship of the at least one test node.

7. The apparatus of claim 6, further comprising: The information acquisition module is configured to acquire historical execution information corresponding to the test request. The index determination module is configured to determine the execution index of each meta-node based on the historical execution information. The cost determination module is configured to determine the execution cost of each meta-node based on the execution index of each meta-node.

8. An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4. The computer instructions are used to enable the computer to perform the method of any one of claims 1-4.

9. A non-transitory computer readable storage medium having computer instructions stored therein, wherein, 10. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-4. ​

Images