Function test method of information system based on artificial intelligence

Abstract

The invention discloses a function test method of an information system based on artificial intelligence. The invention relates to the technical field of system testing. The method comprises the steps of determining at least one to-be-tested process based on test demand information in response to an event of performing a function test on a to-be-tested system based on the test demand information; determining test input information of a to-be-tested function module corresponding to the test node, and transmitting the test input information to the to-be-tested function module; when a processing result fed back based on the to-be-tested function module is received, determining text consumption and processing response duration generated in the process from triggering of the to-be-tested function module to receiving of the processing result; and determining a process function performance attribute corresponding to the to-be-tested process based on the text consumption and / or the processing response duration corresponding to the to-be-tested function module in the to-be-tested process, and determining a test result for testing the to-be-tested system based on the process function performance attribute. According to the technical scheme, the convenience and efficiency of system function testing are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of system testing, and particularly to a functional testing method for an information system based on an artificial intelligence agent. Background Art

[0002] With the development of computer technology, more and more information systems have emerged. To ensure the stability of the operation of information systems, it is usually necessary to test the functions of the information systems themselves to test whether their functions are normal.

[0003] Currently, the testing methods mainly rely on manual testing or test the fixed content of the system through specific test scripts. However, when there are changes or updates in the functions of the information system, it is necessary to manually retest or redesign new test scripts, which has the problems of high testing costs and low efficiency. Summary of the Invention

[0004] The present invention provides a functional testing method for an information system based on an artificial intelligence agent to achieve the technical effect of reducing the testing cost of system functions and improving the testing efficiency.

[0005] According to one aspect of the present invention, there is provided a method for testing system functions, the method comprising:

[0006] In response to an event of performing a functional test on a system under test based on test requirement information in the current round, based on the test requirement information, determining at least one process to be tested; wherein, at least one test node is included in the process to be tested, and the test node is used to test a function module to be tested in the system under test;

[0007] For some of the test nodes in the process to be tested, determining test input information of the function module to be tested corresponding to the test node, and transmitting the test input information to the function module to be tested, so that the function module to be tested processes the test input information and feeds back a processing result;

[0008] When receiving the processing result fed back by the function module to be tested, determining the text consumption amount and the processing response duration generated during the process from triggering the function module to be tested to receiving the processing result; wherein, the text consumption amount is used to characterize the number of text characters processed.

[0009] For each process to be tested, based on the text consumption amount and / or the processing response duration corresponding to some of the function modules to be tested in the process to be tested, determining a process function performance attribute corresponding to the process to be tested, and based on the process function performance attribute, determining a test result of the test on the system under test.

[0010] According to another aspect of the present invention, there is provided a testing device for system functions, the device comprising:

[0011] A to-be-tested process determination module, configured to respond to an event of performing a function test on a to-be-tested system based on test requirement information in the current round, and determine at least one to-be-tested process based on the test requirement information; wherein, at least one test node is included in the to-be-tested process, and the test node is used to test a to-be-tested function module in the to-be-tested system;

[0012] A test input information determination module, configured to, for some of the test nodes in the to-be-tested process, determine test input information of the to-be-tested function module corresponding to the test node, and transmit the test input information to the to-be-tested function module, so that the to-be-tested function module processes the test input information and feeds back a processing result;

[0013] A text consumption determination module, configured to, when receiving a processing result fed back by the to-be-tested function module, determine the text consumption and the processing response duration generated during the process from triggering the to-be-tested function module to receiving the processing result; wherein, the text consumption is used to characterize the number of text characters processed;

[0014] A test result determination module, configured to, for each of the to-be-tested processes, determine a process function performance attribute corresponding to the to-be-tested process based on the text consumption and / or the processing response duration corresponding to some of the to-be-tested function modules in the to-be-tested process, and determine a test result of the test on the to-be-tested system based on the process function performance attribute.

[0015] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0016] At least one processor; and a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can execute the testing method for system functions according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, there is provided a computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the testing method for system functions according to any embodiment of the present invention when executed by a processor.

[0019] According to another aspect of the present invention, there is provided a computer program product, including a computer program which, when executed by a processor, implements the method for testing system functions as described in any embodiment of the present invention.

[0020] The technical solution of the embodiment of the present invention responds to the event of performing a function test on the system to be tested based on test requirement information in the current round, and determines at least one process to be tested based on the test requirement information; at least one test node is included in the process to be tested, and the test node is used to test the function module to be tested in the system to be tested; determine the test input information of the function module to be tested corresponding to the test node, and transmit the test input information to the function module to be tested, so that the function module to be tested processes the test input information and feeds back the processing result; when receiving the processing result fed back by the function module to be tested, determine the text consumption amount and the processing response duration generated during the process from triggering the function module to be tested to receiving the processing result; for each process to be tested, determine the process function performance attribute corresponding to the process to be tested based on the text consumption amount and / or the processing response duration corresponding to some function modules to be tested in the process to be tested, and determine the test result of the test on the system to be tested based on the process function performance attribute, which solves the problems of high test cost and low efficiency in the prior art that rely on manual or script testing, and realizes the function test on the system to be tested by an intelligent agent based on test requirement information. By combining the test requirement information, at least one process to be tested that needs to be tested is determined. Furthermore, the test input information of the function module to be tested is determined, and the test input information is transmitted to the function module to be tested, so that the function module to be tested can automatically process the test input information and feed back the processing result, improving the convenience and efficiency of the test. Further, when receiving the processing result fed back by the function module to be tested, determine the text consumption amount and the processing response duration generated during the process from triggering the function module to be tested to receiving the processing result; evaluate the process function performance attribute of executing the process to be tested according to the text consumption amount and / or the processing response duration corresponding to some function modules to be tested in the process to be tested, improving the accuracy of the process function performance evaluation while enabling the system to be tested based on the process function performance attribute, so as to maintain the system in a timely manner when the system is abnormal, ensure the accuracy of the system process function execution, ensure the stability of the system operation, and meet the user requirements.

[0021] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0023] Figure 1 is a flowchart of a method for testing system functions provided in Embodiment 1 of the present invention;

[0024] Figure 2 is a schematic diagram for characterizing a process to be tested provided in Embodiment 1 of the present invention;

[0025] Figure 3 is a flowchart of a method for testing system functions provided in Embodiment 2 of the present invention;

[0026] Figure 4 is a schematic structural diagram of a device for testing system functions provided in Embodiment 3 of the present invention;

[0027] Figure 5 is a schematic structural diagram of an electronic device for implementing the method for testing system functions in the embodiments of the present invention. Detailed implementation manners

[0028] To enable those skilled in the art to better understand the solutions of the present invention, the following clearly and completely describes the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, rather than all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above accompanying drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device comprising a series of steps or units does not necessarily need to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0030] It should be noted that before introducing the technical solution provided by the present invention, an exemplary description of the application scenario can be given first.

[0031] The application scenario of the embodiments of the present invention can be any scenario that needs to test the system functions. For example, the technical solution provided by the embodiments of the present invention can be integrated into an intelligent agent. This intelligent agent can be integrated into any system that hopes to test its functions; it can also be independent of the system that hopes to test its functions. The manifestation of this intelligent agent can be a software system, a module, a robot, or an interface. The system that hopes to test its functions is the to-be-tested system described below. For example, the to-be-tested system can be an information system for processing information or information flow. For example, the information system includes functions such as input, storage, processing, output, and control of information.

[0032] Of course, an intelligent agent can also be developed according to the solution provided by the embodiments of the present invention to autonomously explore each function and interface of the to-be-tested system based on the intelligent agent, so as to implement the function test of the to-be-tested system.

[0033] Embodiment 1

[0034] Figure 1 It is a flowchart of a method for testing system functions provided according to Embodiment 1 of the present invention. This embodiment is applicable to the situation of testing the functions of an information system. This method can be executed by a testing device for system functions. The testing device for system functions can be implemented in the form of hardware and / or software, and the testing device for system functions can be configured in a computing device. As Figure 1 shown, this method includes:

[0035] S110. In response to the event of performing a function test on the to-be-tested system based on test requirement information in the current round, determine at least one to-be-tested process based on the test requirement information; the to-be-tested process includes at least one test node, and the test node is used to test the to-be-tested function module in the to-be-tested system.

[0036] Among them, the test requirement information can be information used to describe the requirements and needs for testing the system to be tested. For example, the test requirement information includes, but is not limited to, the user role, the number of test rounds, the number of test times in a single round, the test frequency, and the test time, etc. The test frequency refers to how often the system to be tested is functionally tested; the number of test rounds refers to how many rounds of testing are required for the system to be tested; the number of test times refers to how many times the system to be tested is functionally tested in a single round; the test time can refer to when the system to be tested is functionally tested. The user role refers to the role in which the system to be tested is functionally tested. For example, the user role can be user roles in different positions or departments. The system to be tested includes multiple functional modules, and each functional module is used to implement a specific function in the system. For example, the functional module can be a calculation module, a verification module, an audit module, a data transmission module, etc., which will not be elaborated here. The process to be tested can be a work process composed of a series of operation steps that need to be executed in a certain order. That is to say, in the process of system testing, in order to verify whether a certain function or business logic is running properly, a series of operation steps can be executed in a certain order. Each operation step is a test node in the process and corresponds to a functional module to be tested. That is, the process to be tested is used to test one or more functional modules to be tested in the system to be tested.

[0037] In this embodiment, different test requirement information can be determined according to different test requirements. The test requirement information can be input in real time or imported from a database. When the test requirement information is received, it can be considered that an event of functionally testing the system to be tested is responded; or, the test requirement information includes a timed test task, and when the timed test task is triggered, it is considered that an event of functionally testing the system to be tested is responded. Further, the test requirement information can be parsed to clarify which functions are to be tested and which test processes need to be executed, so as to determine at least one process to be tested and the functional modules to be tested corresponding to the test nodes involved in the process to be tested. For example, the process to be tested can be a user login process, a user registration process, a search process, a transaction process, an order processing process, an audit process, a submit information process, etc. Exemplarily, the process to be tested to be executed and the number of test times can be set according to the test requirements through the ability of artificial intelligence process orchestration, such as setting to complete the test of executing the process to be tested a + 1 times.

[0038] Exemplarily, referring to Figure 2 , the process to be tested includes Process 1, Process 2, and Process 3; the functional modules to be tested in Process 1 include Module 1, Module 2, Module 3, and Module 4; the functional modules to be tested in Process 2 include Module 5, Module 6, Module 7, Module 8, and Module 9; the functional modules to be tested in Process 3 include Module 3, Module 4, Module 7, Module 8, and Module 9.

[0039] It should be noted that the number of processes to be tested includes one or more. If there are multiple processes, the types of functions implemented by each process to be tested can be the same or different. Each process to be tested can be executed independently, and each process to be tested can be tested separately in parallel or serially. It should also be noted that depending on the different usage roles, the usage of the information system by the system to be tested is different, and the process of executing functions may also be different; that is, different usage roles use the system to be tested in different ways and complete different tasks. At this time, based on different usage roles of the test requirement information, the usage of the system to be tested by simulating the corresponding usage roles can be used to perform functional testing on the system to be tested, so as to improve the convenience of testing while improving the authenticity and accuracy of testing.

[0040] S120. For some test nodes in the process to be tested, determine the test input information of the function module to be tested corresponding to the test node, and transmit the test input information to the function module to be tested, so that the function module to be tested processes the test input information and feeds back the processing result.

[0041] Among them, the test input information can refer to the information required for the function module to be tested to implement its own function, that is, the test input information is the information used to test the function of the function module to be tested. The processing result can be the result obtained after processing the test input information, or the result indicating whether the processing of the test input information is successful. It should be noted that the method of performing functional testing on each process to be tested is the same, and the method of performing functional testing on the function modules to be tested of some test nodes in the process to be tested is also the same. Hereinafter, an example of performing functional testing on the function module to be tested corresponding to any one of the test nodes in any one of the processes to be tested will be introduced.

[0042] In this embodiment, during the process of performing functional testing on the function module to be tested corresponding to a certain test node, the test input information of the function module to be tested can be determined according to the function implementation requirements or business attributes of the function module to be tested, and the test input information is submitted to the function module to be tested. Among them, the function implementation requirements can refer to the specific function requirements that the function module needs to implement, such as information including function characteristics. The business attribute can refer to the characteristics or rules of the function module in the business scenario, such as information related to business processes, data processing, user permissions, and other businesses. Alternatively, the test input information of the function module to be tested can also be analyzed in combination with the historical knowledge base. The historical knowledge base can be determined based on the historical function processing records of the function module to be tested of the system to be tested and the management regulation information of the system to be tested. After receiving the test input information, the function module to be tested automatically processes the test input information to obtain the processing result.

[0043] It should be noted that after the function module under test processes the test input information, it can also automatically pass the processing result to the next function module under test, so that the second function module under test can continue to execute its function logic based on the output of the previous module.

[0044] In this embodiment, passing the test input information to the function module under test to enable the function module under test to process the test input information and feedback the processing result includes: sending the test input information to the function module under test, so that when it is determined based on the function module under test that there is an associated function module depending on it in the test process to be tested, using the processing result obtained by processing the test input information based on the function module under test as the test input information of the associated function module, and sending it to the associated function module, so that the associated function module processes the received test input information and feedbacks the processing result. Among them, the associated function module can be a module whose function is triggered by the function module under test.

[0045] Specifically, input the test input information into the function module under test, trigger the function module under test through a triggering method (such as calling an interface, clicking a button, sending an instruction, etc.), so that the function module under test starts to process the incoming test input information. And if there is an associated function module depending on the function module under test in the test process to be tested, then the function module under test can pass the processing result as the test input information to the associated function module, so that the associated function module receives the new test input information, processes it, generates the final processing result, and feedbacks the result for subsequent verification or further processing by the next function module. Realize the collaborative work of multiple function modules to complete complex test tasks and improve the convenience of testing. It should be noted that when determining the test input information of the associated function module, the test input information can also be generated assisted by an agent to ensure the accuracy of the test.

[0046] Exemplarily, assume that all function modules under test except the first function module under test in each test process to be tested are used for review, verification and revision, and the content of review, verification and revision is different. The first function module under test in each test process to be tested needs to fill in the test input information. At this time, analyze the knowledge base of the intelligent agent through the large language model to determine the test input information of the function module under test, and submit it to the function module under test. After submitting it to the function module under test, the function module under test processes the information, and the processing result can automatically enter the second function module under test (i.e., the associated function module), so that the second function module under test continues to process and obtains the processing result. Or, the intelligent agent can combine the specific requirements of the associated function module to process the test input information passed in by the first function module under test, so that the associated function module reviews and revises the processed information.

[0047] S130. When receiving the processing result fed back by the function module to be tested, determine the text consumption and processing response duration generated during the process from triggering the function module to be tested to receiving the processing result.

[0048] Among them, the text consumption is used to characterize the number of text characters processed. For example, the text consumption can be the amount of text data transmitted and processed during the entire process from triggering the function module to be tested to receiving the processing result, or the number of tokens into which the involved text data is decomposed. A token is the basic unit in the text and can be a character such as a Chinese character, word, number, or symbol. The processing response duration can be the duration from sending the request to receiving the response.

[0049] In this embodiment, after the function module to be tested finishes processing the test input information, it returns the processing result. When obtaining the processing result of the function module to be tested, the total token consumption involved during the entire process from triggering the function module to be tested to receiving its processing result can be determined as the text consumption, and the time interval from triggering the function module to be tested to receiving its processing result can be determined as the processing response duration. To evaluate the performance and efficiency of the test function module by measuring the text consumption and processing response duration. For example, when using a large language model, the input text "123456" will be split into several tokens, and the generated reply by the model will also be split into several tokens. The sum of the tokens from the two splits is the text consumption.

[0050] Exemplarily, the text consumption generated during the process from triggering the function module to be tested in the test process to be tested to receiving the processing result of the function module to be tested can be recorded. For example, a certain test process to be tested numbered 2 has 5 function modules to be tested. Then the tokens consumed each time during the first test of this test process to be tested (i.e., the text consumption) can be expressed as tk201, tk211, tk221, tk231, tk241. The subscript is up to 4, indicating that the test process to be tested has 4 + 1 function modules to be tested.

[0051] To ensure that when a module function error is detected, the test process can be terminated in time to avoid performing invalid tests. After receiving the processing result fed back by the function module to be tested, if the processing result is a processing failure result, the test operation for the test process to which the function module to be tested belongs can be stopped.

[0052] That is to say, based on the processing result returned by the function module to be tested in the test process to be tested, it can be determined whether the function module to be tested has successfully executed its own function. If the processing result is a processing failure result, the test for this test process to be tested is terminated to avoid continuing to perform invalid test operations under the known failure situation, saving time and resources.

[0053] Exemplarily, if the feedback processing result is a processing failure result, it indicates that there may be reasons such as the inability to pass the self-verification of the system under test or the abnormal function of the system under test. At this time, it will cause the inability to be passed to the next function module to be tested, so the process under test at this time is a process that cannot be executed to the set function module, that is, the process that has not actually been executed to the final function module.

[0054] S140. For each process under test, based on the text consumption amount and / or processing response duration corresponding to some function modules to be tested in the process under test, determine the process function performance attribute corresponding to the process under test, and based on the process function performance attribute, determine the test result of testing the system under test.

[0055] Among them, the test result can be the test result after executing the function of at least one process under test in the current test. The process function performance attribute can be used to characterize the efficiency and quality of executing the process under test and realizing its function. For example, the process function performance attribute can be response time, throughput, resource utilization rate, etc. The function refers to the specific tasks or operations that the process under test can complete.

[0056] In this embodiment, the execution performance of the process under test can be comprehensively evaluated according to the text consumption amount and / or processing response duration corresponding to some function modules to be tested in the process under test to obtain the process function performance attribute corresponding to the process under test. Correspondingly, the process function performance attribute corresponding to each process under test can be obtained. Further, the process function performance attribute corresponding to the process under test can be compared with the process function performance attribute obtained by testing the system under test based on the test requirement information in the previous round to determine the test result of testing the system under test. For example, if the text consumption situation or response situation in the process function performance attribute increases significantly compared with the previous round of testing, it indicates that the overall process audit complexity of the system under test has increased. At this time, the test result can be determined to be a test exception, that is, the test fails.

[0057] The technical solution of this embodiment determines at least one process to be tested based on test requirement information in response to an event of performing a functional test on the system under test in the current round. At least one test node is included in the process to be tested, and the test node is used to test the functional module to be tested in the system under test. Determine the test input information of the functional module to be tested corresponding to the test node, and transmit the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back the processing result. When receiving the processing result fed back by the functional module to be tested, determine the text consumption amount and the processing response duration generated during the process from triggering the functional module to be tested to receiving the processing result. For each process to be tested, determine the process function performance attribute corresponding to the process to be tested based on the text consumption amount and / or the processing response duration of some functional modules to be tested in the process to be tested, and determine the test result of the system under test based on the process function performance attribute, which solves the problems of high test cost and low efficiency in the prior art that rely on manual or script testing. It realizes that the agent performs a functional test on the system under test based on test requirement information, and combines with the test requirement information to determine at least one process to be tested that needs to be tested. Furthermore, determine the test input information of the functional module to be tested, and transmit the test input information to the functional module to be tested, so that the functional module to be tested can automatically process the test input information and feed back the processing result, improving the convenience and efficiency of the test. Further, when receiving the processing result fed back by the functional module to be tested, determine the text consumption amount and the processing response duration generated during the process from triggering the functional module to be tested to receiving the processing result. Evaluate the process function performance attribute of executing the process to be tested according to the text consumption amount and / or the processing response duration of some functional modules to be tested in the process to be tested, improving the accuracy of the process function performance evaluation, and at the same time, testing the system under test based on the process function performance attribute to maintain the system in time when the system is abnormal, ensuring the accuracy of the system process function execution and the stability of the system operation, and meeting the user requirements.

[0058] Embodiment 2

[0059] Figure 3 FIG. is a flowchart of a method for testing system functions provided in Embodiment 2 of the present invention. On the basis of the foregoing embodiment, optionally, the test requirement information includes the number of single-round tests, and the number of single-round tests includes at least two tests. Correspondingly, S140 is further refined. The specific implementation manner can refer to the technical solution of this embodiment. Wherein, the same or corresponding technical terms as those in the above embodiment are not described herein again.

[0060] As Figure 3 shown, the method specifically includes the following steps:

[0061] S210. In response to the event of performing a functional test on the system under test based on the test requirement information in the current round, determine at least one process to be tested based on the test requirement information; the process to be tested includes at least one test node, and the test node is used to test the function module to be tested in the system under test.

[0062] S220. For some of the test nodes in the process to be tested, determine the test input information of the function module to be tested corresponding to the test node, and transmit the test input information to the function module to be tested, so that the function module to be tested processes the test input information and feeds back the processing result.

[0063] S230. When receiving the processing result fed back by the function module to be tested, determine the text consumption amount and the processing response duration generated during the process from triggering the function module to be tested to receiving the processing result.

[0064] S240. For each process to be tested, based on the text consumption amounts corresponding to some of the function modules to be tested in the process to be tested under at least two tests and the number of single-round tests, determine the average text consumption attribute in the process function performance attribute corresponding to the process to be tested; and / or, based on the processing response durations corresponding to some of the function modules to be tested in the process to be tested under at least two tests and the number of single-round tests, determine the average processing response attribute in the process function performance attribute.

[0065] Among them, the number of single-round tests may refer to the number of times of performing a functional test on the system under test based on the test requirement information in a single round. The process function performance attribute includes the average text consumption attribute and / or the average processing response attribute. The average text consumption attribute can be used to characterize the average token consumption situation of the process to be tested under multiple tests in a single round. The average processing response attribute can be used to characterize the average processing response duration situation of the process to be tested under multiple tests in a single round.

[0066] In this embodiment, the sum of the text consumption amounts corresponding to some or all of the function modules to be tested in the process to be tested during multiple tests can be counted to obtain a sum value, and the sum value is divided by the number of single-round tests to obtain the average text consumption attribute corresponding to the process to be tested.

[0067] For example, during the process of performing each process to be tested based on the agent, the overall token consumption amount and response duration of each process to be tested can be calculated during the interaction process of the agent with the functions of the system under test, information completion, and information review. Among them, the token consumption amount (i.e., text consumption amount) of different function modules to be tested in different processes to be tested can be represented by represented, and the text consumption amount of all processes to be tested is represented by the matrix represented, represents the Text consumption matrix for the current test execution.

[0068]

[0069] Wherein, represents the index of different processes to be tested, is the index of the function module to be tested involved in a certain process to be tested. represents the test index for each round of test execution (characterizing the number of tests), represents the th test execution of the th process to be tested for the th function module to be tested. It should be noted that since the number of function modules to be tested in each process to be tested is not necessarily the same, so can take the number of function modules of the process to be tested with the largest number of function modules to be tested in the system to be tested minus one (because the matrix starts counting from 0, while the number of function modules to be tested generally starts counting from 1). If the number of function modules to be tested in a certain process to be tested is less than , then the text consumption of the function module without being tested is recorded as 0. Further, combining the recorded data in the text consumption matrix, calculate the average consumption token data of each process to be tested (i.e., the average text consumption attribute of the process).

[0070] Optionally, the calculation formula for determining the average text consumption attribute of the process to be tested can be expressed as: ; wherein takes values from 0 to , represents the number of tests, takes values from 0 to , represents the th function module to be tested, is the th process to be tested; represents the average text consumption attribute of the th process to be tested. The text consumption of each function module to be tested in the same process to be tested under different numbers of tests can be summed up as the total text consumption of the process to be tested for a single test. After executing times of this process to be tested, divide the total text consumption by the number of tests to obtain the average text consumption attribute of the process to be tested at the

[0071] In this embodiment, it is also possible to count the sum of the processing response durations corresponding to some or all of the to-be-tested functional modules in the to-be-tested process during multiple tests, obtain a sum value, divide the sum value by the number of single-round tests, and obtain the process average processing response attribute corresponding to the to-be-tested process.

[0072] The overall processing response duration of each to-be-tested process can be represented by and the processing response durations of all to-be-tested processes are represented by the matrix . represents the processing response duration matrix for the th test execution.

[0073]

[0074] Among them, represents the processing response duration of the th test execution for the rd to-be-tested process and the th to-be-tested functional module. It should be noted that since the number of to-be-tested functional modules in each to-be-tested process is not necessarily the same, so can take the number of functional modules of the to-be-tested process with the largest number of to-be-tested functional modules in the to-be-tested system minus one (because the matrix starts counting from 0, while the number of to-be-tested functional modules generally starts counting from 1). If the number of to-be-tested functional modules in a certain to-be-tested process is less than , the processing response durations of the non-existent to-be-tested functional modules are recorded as 0. Further, based on the recorded data in the processing response duration matrix, calculate the average duration consumed by each to-be-tested process (i.e., the process average processing response attribute).

[0075] Optionally, the calculation formula for determining the process average processing response attribute of the to-be-tested process can be expressed as: ; where ranges from 0 to , represents the number of tests, ranges from 0 to , represents the th to-be-tested functional module, is the nd to-be-tested process. represents the process average processing response attribute of the th to-be-tested process. The processing response durations of each to-be-tested functional module in the same to-be-tested process under different test times can be summed as the total processing response duration of the to-be-tested process for a single test. After executing times of this to-be-tested process, divide the total processing response duration by the number of tests to obtain The process of the next process to be tested processes the response attribute.

[0076] S250. Determine the test pass rate of all processes to be tested.

[0077] Among them, the test pass rate can be used to characterize the probability of passing the process to be tested. For example, the higher the test pass rate, the more processes to be tested with normal implementation of process functions.

[0078] In this embodiment, the test pass rate can be determined based on the ratio of the number of processes to be tested that pass the test to the total number of processes to be tested. Optionally, the method for determining the test pass rate of at least one process to be tested can be: for each process to be tested, when all the function modules to be tested in the process to be tested pass the test, update the process test pass data; determine the test pass rate based on the process test pass data and the number of processes of at least one process to be tested.

[0079] That is to say, whenever all the function modules to be tested in a process to be tested pass the test, perform an increment operation on the process test pass data, that is, update the process test pass data. Take the ratio between the updated process test pass data and the number of processes of at least one process to be tested as the test pass rate. The advantage of this setting is that by evaluating the overall test results of all processes to be tested, the execution situation of the system process to be tested can be accurately determined, thereby improving the accuracy of the overall performance evaluation of the system to be tested.

[0080] Exemplarily, during the test of all function modules to be tested in the process to be tested, when the processing result of the last function module to be tested is a successful processing result, it is considered that the final step in the process to be tested is actually executed, and at this time the process test pass data is automatically incremented by 1. And so on, for each (such as m) process to be tested of the system to be tested, the intelligent agent performs 1 test, that is, the execution of m processes is completed. The execution of each process includes at most n function modules to be tested. When the last function module to be tested in the process is executed, the process test pass data is automatically incremented by 1.

[0081] Optionally, the calculation method for determining the test pass rate of all processes to be tested can be expressed as:

[0082] ; among them, represents the test pass rate; represents the process test pass data; Indicates the number of processes. It should be noted that in this round of functional testing of the system under test based on the test requirement information, the number of tests can include multiple times. Then the number of processes can be the product between the number of processes of at least one process to be tested and the number of tests. The process test pass data is the number of processes that pass the functional test after multiple functional tests on each process to be tested.

[0083] S260. Determine the test result of the system under test based on the test pass rate and the process functional performance attributes.

[0084] In this embodiment, the test pass rate obtained in this round of testing, the average text consumption attribute and / or the average processing response attribute in the process functional performance attributes, and the test pass rate and the process functional performance attributes obtained in the previous round of testing can be compared to obtain the test result of the system under test. By comprehensively considering the test pass rate of the process, the average text consumption attribute in it, and the average processing response attribute, the performance and stability of the system process can be comprehensively evaluated, and it can be accurately determined whether the system process meets the expected performance requirements. For example, if the test pass rate is significantly lower than that of the previous round, it indicates that there is a situation where the process in the system under test cannot be accurately executed, and at this time, the test result can be determined as a test anomaly.

[0085] In this embodiment, the specific way to determine the test result of the system under test can be: obtain the historical text consumption attribute, historical processing response attribute, and historical process test pass attribute corresponding to each process to be tested when the system under test is functionally tested based on the test requirement information in the previous round; if the test pass rate is less than the historical process test pass attribute, determine that the test result of the system under test is a test failure; if the average text consumption attribute in the process functional performance attributes is greater than the historical text consumption attribute, determine that the test result of the system under test is a test failure; if the average processing response attribute in the process functional performance attributes is greater than the historical text consumption attribute, determine that the test result of the system under test is a test failure.

[0086] Among them, the historical process test pass attribute can refer to the test pass rate obtained when the system under test is functionally tested based on the test requirement information in the previous round. The historical text consumption attribute can refer to the average text consumption obtained when the system under test is functionally tested based on the test requirement information in the previous round. The historical processing response attribute can refer to the average processing response duration obtained when the system under test is functionally tested based on the test requirement information in the previous round.

[0087] Specifically, when performing functional testing, historical test data can be obtained when the previous round of functional testing of the system to be tested is performed based on the test requirement information. These data include: historical process test pass attribute, historical text consumption attribute and historical processing response attribute. Further, the test pass rate obtained by the current test can be compared with the historical process test pass attribute. If the test pass rate of the current test is less than the historical process test pass attribute, it means that the process in the system to be tested cannot be accurately executed compared with the previous test. At this time, the test result of the system to be tested is determined to be a test failure. When the process processing response attribute is greater than the historical text consumption data corresponding to the process to be tested, it indicates that there may be a performance problem, and the test result of the system to be tested is determined to be a test failure. It should be noted that the text consumption of different functional modules to be tested in different processes to be tested can also be compared with the historical text consumption of different functional modules to be tested in different processes to be tested obtained in the previous round of functional testing of the system to be tested based on the test requirement information. If the text consumption of some functional modules in the current test is higher than the historical text consumption, it indicates that there may be a performance problem, and the test result is determined to be a test failure. You can also compare the processing response time of different functional modules to be tested in different processes to be tested with the historical processing response time. If the processing response time of some functional modules in the current test is longer than the historical processing response time, it indicates that there may be a processing performance problem, and the test result is determined to be a test failure. Alternatively, you can also comprehensively consider the evaluation results of the test pass rate, text consumption, and processing response time to finally determine the test result of the system to be tested. For example, if the test pass rate is lower than the historical process test pass attribute, it is directly determined to be a test failure; or, if the test pass rate is in line with expectations, but the text consumption or processing response time exceeds the historical data range, the test result is determined to be a test failure; or, if all indicators are in line with or better than the historical data, it is determined to be a test pass. Through the above steps, you can comprehensively and accurately evaluate whether the functions and performance of the process of the system to be tested are normal.

[0088] It should be noted that if it is the first time to conduct the test, you only need to record the text consumption and processing response time, so as to determine the text consumption and processing response time during the subsequent test, and analyze the test results of the system to be tested in combination with the data recorded last time.

[0089] In order to accurately locate anomalies when the system has anomalies, after determining the test results for the system under test based on the process function performance attributes, when the test results are inconsistent with the preset results, for each function module to be tested in the process to be tested, based on the text consumption corresponding to the function module to be tested under at least two tests and the number of single-round tests, determine the module average text consumption attribute corresponding to the function module to be tested, and when the module average text consumption attribute is greater than the historical text consumption of the function module to be tested, determine that the test result for the test of the function module to be tested is a failed test; based on the processing response duration corresponding to the function module to be tested under at least two tests and the number of single-round tests, determine the module average processing response attribute corresponding to the function module to be tested, and when the module average processing response attribute is greater than the historical processing response duration of the function module to be tested, determine that the test result for the test of the function module to be tested is a failed test.

[0090] Among them, both the historical text consumption and the historical processing response duration of the function module to be tested are obtained when the function test of the system under test is performed in the previous round based on the test requirement information. The module average text consumption attribute can be used to represent the average amount of text consumed by the function module to be tested.

[0091] Specifically, the text consumption corresponding to the same function module to be tested under at least two tests can be summed to obtain a sum value, and the quotient value between the sum value and the number of tests is used as the module average text consumption attribute corresponding to the function module to be tested. When the module average text consumption attribute is greater than the historical text consumption of the function module to be tested, or when the value of the module average text consumption attribute greater than the historical text consumption of the function module to be tested exceeds the preset threshold, it is considered that the function of the function module to be tested is not being properly executed. At this time, determine that the test result for the test of the function module to be tested is a failed test. The processing response duration corresponding to the same function module to be tested under at least two tests can also be summed to obtain a sum value, and the quotient value between the sum value and the number of tests is used as the module average processing response attribute corresponding to the function module to be tested. When the module average processing response attribute is greater than the historical processing response duration of the function module to be tested, or when the value of the module average processing response attribute greater than the historical processing response duration of the function module to be tested exceeds the preset threshold, it is considered that the function of the function module to be tested is not being properly executed. At this time, determine that the test result for the test of the function module to be tested is a failed test. So that the function module to be tested with a failed test is sent to the target terminal or a test report is generated, so that the user can accurately maintain the system under test through the terminal or by viewing the test report.

[0092] Exemplarily, when the test result of the system under test is a failed test, it indicates that the system has obvious anomalies. At this time, fit the text consumption matrix and the time consumption matrix.

[0093] The text consumption matrix is expressed as:

[0094] The time consumption matrix is expressed as:

[0095] Wherein, represents the module average text consumption attribute of the th functional module to be tested in the th process to be tested; represents the module average processing response attribute of the th functional module to be tested in the th process to be tested. Suppose has a significant difference in order of magnitude from the data in the previous test process, it indicates that there is a suspected anomaly in the use of the second functional module to be tested in the first process to be tested of the system to be tested, and it can be shown that there is a suspected anomaly in the process setting of the system to be tested. Suppose has a significant difference in order of magnitude from the data in the previous test process, it indicates that there is a suspected anomaly in the use of the first functional module to be tested in the second process to be tested of the system to be tested, and it can be shown that there are other reasons such as the system to be tested cannot be accessed. By further detailed analysis of which functional module to be tested in which process to be tested has a functional anomaly when the test result of the system to be tested is a test failure, the data calculation amount can be effectively reduced, and the accurate positioning of the abnormal module can be realized.

[0096] In order to maintain the system in a timely manner when an anomaly occurs in the system, after determining the test result of the system to be tested based on the text consumption amount and / or processing response duration corresponding to some functional modules to be tested in at least one process to be tested, the method further includes: when the test result is inconsistent with the preset result, generating a warning prompt message, so as to perform functional maintenance on the system to be tested based on the warning prompt message. Wherein, the preset result is a test pass.

[0097] Specifically, when the test result is a test failure, it can be considered that the test result is inconsistent with the preset result. At this time, a warning prompt message is generated to timely notify relevant personnel or systems through the warning prompt message, and according to the warning prompt message, perform functional inspection, optimization or repair on the system to be tested, so as to ensure the performance and functional stability of the system.

[0098] In order to further clearly analyze the performance of the system to be tested when no abnormalities occur in the system to be tested, after determining the test results of the system to be tested based on the process function performance attributes, when the test results are consistent with the preset results, based on the text consumption corresponding to the function module to be tested under at least two tests and the average text consumption attribute of the module, determine the text standard deviation of the function module to be tested; based on the processing response duration corresponding to the function module to be tested under at least two tests and the average processing response attribute of the module, determine the duration standard deviation of the function module to be tested; based on the text standard deviation and / or the duration standard deviation, determine the system function performance attributes of the system to be tested.

[0099] Among them, the average processing response attribute of the process is determined based on the processing response duration corresponding to some function modules to be tested in the process to be tested under at least two tests; the system function performance attributes can be used to characterize the efficiency and quality of executing the system to be tested to achieve its functions, etc.

[0100] Specifically, when the test result is a pass, it can be considered that the test result is consistent with the preset result. For each function module to be tested in each process to be tested, the difference between the text consumption corresponding to the function module to be tested each time and the average processing response attribute of the function module to be tested can be calculated respectively. The differences can be summed to obtain a sum value, and the quotient of the sum value and the number of tests is used as the text standard deviation of the function module to be tested. In this way, the difference between the processing response duration corresponding to the function module to be tested each time and the average processing response attribute of the function module to be tested is calculated respectively, the differences are summed to obtain a sum value, and the quotient of the sum value and the number of tests is used as the duration standard deviation of the function module to be tested. Further, in combination with the text standard deviation and / or the duration standard deviation, analyze the system function performance attributes of the system to be tested. For example, the smaller the text standard deviation or the duration standard deviation, the more stable the system, that is, the more reasonable the process to be tested is set in the system to be tested. It should be noted that if there are many function modules or many processes to be tested in the process to be tested, a certain number of data can be randomly selected from them to participate in the calculation to improve the accuracy of using the standard deviation to represent the system stability.

[0101] Exemplarily, the calculation formula for determining the text standard deviation of the function module to be tested can be expressed as:

[0102] ; where represents the th function module to be tested in the

[0103] text standard deviation of the

[0104] ; where Indicates the th duration standard deviation of the th function module to be tested in the process to be tested.

[0105] The technical solution of this embodiment determines the average text consumption attribute in the process function performance attribute corresponding to the process to be tested based on the text consumption amount corresponding to some function modules to be tested in the process to be tested under at least two tests and the number of single-round tests; and / or determines the average processing response attribute in the process function performance attribute based on the processing response duration corresponding to some function modules to be tested in the process to be tested under at least two tests and the number of single-round tests. At the same time, the test pass rate of all processes to be tested is determined. Finally, based on the test pass rate and the process function performance attribute, the test result of the system to be tested is determined, improving the accuracy of process evaluation.

[0106] Embodiment III

[0107] Figure 4 is a schematic structural diagram of a test device for system functions provided according to Embodiment III of the present invention. As Figure 4 shown, the device includes: a process to be tested determination module 310, a test input information determination module 320, a text consumption amount determination module 330, and a test result determination module 340.

[0108] Among them, the process to be tested determination module 310 is configured to respond to an event of performing a function test on the system to be tested based on test requirement information in the current round, and determine at least one process to be tested based on the test requirement information; wherein, at least one test node is included in the process to be tested, and the test node is used to test the function module to be tested in the system to be tested; the test input information determination module 320 is configured to, for some test nodes in the process to be tested, determine the test input information of the function module to be tested corresponding to the test node, and transmit the test input information to the function module to be tested, so that the function module to be tested processes the test input information and feeds back a processing result; the text consumption amount determination module 330 is configured to, when receiving the processing result fed back based on the function module to be tested, determine the text consumption amount and the processing response duration generated during the process from triggering the function module to be tested to receiving the processing result; wherein, the text consumption amount is used to characterize the number of text characters processed; the test result determination module 340 is configured to, for each process to be tested, determine the process function performance attribute corresponding to the process to be tested based on the text consumption amount and / or the processing response duration corresponding to some function modules to be tested in the process to be tested, and determine the test result of the test on the system to be tested based on the process function performance attribute.

[0109] The technical solution of this embodiment is to, in response to an event of performing a functional test on a system under test based on test requirement information in the current round, determine at least one process to be tested based on the test requirement information; at least one test node is included in the process to be tested, and the test node is used to test a function module to be tested in the system under test; determine the test input information of the function module to be tested corresponding to the test node, and transmit the test input information to the function module to be tested, so that the function module to be tested processes the test input information and feeds back a processing result; when receiving the processing result fed back by the function module to be tested, determine the text consumption amount and the processing response duration generated during the process from triggering the function module to be tested to receiving the processing result; for each process to be tested, based on the text consumption amount and / or the processing response duration corresponding to some function modules to be tested in the process to be tested, determine the process function performance attribute corresponding to the process to be tested, and based on the process function performance attribute, determine the test result of the system under test, which solves the problems of high test cost and low efficiency in the prior art that rely on manual or script testing, and realizes that an intelligent agent performs a functional test on the system under test based on test requirement information. By combining the test requirement information, at least one process to be tested that needs to be tested is determined. Furthermore, the test input information of the function module to be tested is determined, and the test input information is transmitted to the function module to be tested, so that the function module to be tested can automatically process the test input information and feed back the processing result, improving the convenience and efficiency of the test. Further, when receiving the processing result fed back by the function module to be tested, determine the text consumption amount and the processing response duration generated during the process from triggering the function module to be tested to receiving the processing result; evaluate the process function performance attribute of executing the process to be tested according to the text consumption amount and / or the processing response duration corresponding to some function modules to be tested in the process to be tested, while improving the accuracy of the process function performance evaluation, and performing a test on the system under test based on the process function performance attribute, so as to maintain the system in time when the system is abnormal, ensure the accuracy of the system process function execution, ensure the stability of the system operation, and meet the user requirements.

[0110] Based on the above device, optionally, the test input information determination module 320 is configured to send the test input information to the function module to be tested, so that when it is determined based on the function module to be tested that there is an associated function module in the process to be tested that depends on it, the processing result obtained by processing the test input information based on the function module to be tested is used as the test input information of the associated function module and sent to the associated function module, so that the associated function module processes the received test input information and feeds back a processing result.

[0111] Based on the above device, optionally, the device further includes:

[0112] A process test termination module, configured to stop the test operation on the to-be-tested process to which the to-be-tested function module belongs if the processing result is a processing failure result.

[0113] Based on the above device, optionally, the test requirement information includes the number of single-round tests, and the number of single-round tests includes at least two tests. The test result determination module 340 includes:

[0114] A process function performance attribute determination unit, configured to, for each to-be-tested process, determine the average text consumption attribute in the process function performance attributes corresponding to the to-be-tested process based on the text consumption amount corresponding to some of the to-be-tested function modules in the to-be-tested process under at least two tests and the number of single-round tests; and / or, determine the average processing response attribute in the process function performance attributes based on the processing response duration corresponding to some of the to-be-tested function modules in the to-be-tested process under at least two tests and the number of single-round tests.

[0115] Based on the above device, optionally, the test result determination module 340 includes:

[0116] A test pass rate determination unit, configured to determine the test pass rate of all the to-be-tested processes;

[0117] A test result determination unit, configured to determine the test result of the test on the to-be-tested system based on the test pass rate and the process function performance attributes.

[0118] Based on the above device, optionally, the test result determination unit includes:

[0119] A historical processing information determination unit, configured to obtain the historical text consumption attribute, historical processing response attribute, and historical process test pass attribute corresponding to each to-be-tested process obtained when the to-be-tested system was functionally tested based on the test requirement information in the previous round;

[0120] A first test result determination unit, configured to determine that the test result of the test on the to-be-tested system is a test failure if the test pass rate is less than the historical process test pass attribute;

[0121] A second test result determination unit, configured to determine that the test result of the test on the to-be-tested system is a test failure if the average text consumption attribute in the process function performance attributes is greater than the historical text consumption attribute;

[0122] A third test result determination unit, configured to determine that the test result of the test on the to-be-tested system is a test failure if the average processing response attribute in the process function performance attributes is greater than the historical text consumption attribute.

[0123] Based on the above device, optionally, the test requirement information includes the number of single-round tests, the number of single-round tests includes at least two tests, and the device further includes:

[0124] The first unit for determining the module test result is used to, when the test result is inconsistent with the preset result, for each to-be-tested functional module in the to-be-tested process, based on the text consumption corresponding to the to-be-tested functional module under at least two tests and the number of single-round tests, determine the module average text consumption attribute corresponding to the to-be-tested functional module, and when the module average text consumption attribute is greater than the historical text consumption of the to-be-tested functional module, determine that the test result of the to-be-tested functional module test is a failed test;

[0125] The second unit for determining the module test result is used to, based on the processing response duration corresponding to the to-be-tested functional module under at least two tests and the number of single-round tests, determine the module average processing response attribute corresponding to the to-be-tested functional module, and when the module average processing response attribute is greater than the historical processing response duration of the to-be-tested functional module, determine that the test result of the to-be-tested functional module test is a failed test.

[0126] Based on the above device, optionally, the device further includes:

[0127] The function maintenance unit is used to, when the test result is inconsistent with the preset result, generate a warning prompt message to perform function maintenance on the to-be-tested system based on the warning prompt message.

[0128] Based on the above device, optionally, the device further includes:

[0129] The text standard deviation determination unit is used to, when the test result is consistent with the preset result, based on the text consumption corresponding to the to-be-tested functional module under at least two tests and the module average text consumption attribute, determine the text standard deviation of the to-be-tested functional module;

[0130] The function performance attribute duration standard deviation determination unit is used to, based on the processing response duration corresponding to the to-be-tested functional module under at least two tests and the module average processing response attribute, determine the duration standard deviation of the to-be-tested functional module;

[0131] The determination unit is used to determine the system function performance attribute of the to-be-tested system based on the text standard deviation and / or the duration standard deviation.

[0132] The test device for system functions provided by the embodiments of the present invention can execute the test method for system functions provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0133] Embodiment 4

[0134] Figure 5 It is a schematic structural diagram of an electronic device for implementing the system function test method according to the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0135] As Figure 5 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory 12, a random access memory 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory 12 or the computer program loaded from the storage unit 18 into the random access memory 13. In the random access memory 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the read-only memory 12, and the random access memory 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0136] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0137] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the system function test method.

[0138] In some embodiments, a method for testing system functions may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the read-only memory 12 and / or the communication unit 19. When the computer program is loaded into the random access memory 13 and executed by the processor 11, one or more steps of the method for testing system functions described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the method for testing system functions by any other suitable means (e.g., by means of firmware).

[0139] The various embodiments of the systems and techniques described above in this document may be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs may be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on a remote machine or server.

[0141] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] For providing interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used for providing interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0143] The systems and techniques described herein can be implemented in a computing system that includes backend components (such as, for example, a data server), or a computing system that includes middleware components (such as, for example, an application server), or a computing system that includes frontend components (such as, for example, a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (such as, for example, a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0144] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0145] In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication unit 19, or installed from the storage unit 18, or installed from the read-only memory 12. When the computer program is executed by the processor 11, the above-mentioned functions defined in the method of the embodiment of the present invention are executed.

[0146] An embodiment of the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method for testing system functions provided in any embodiment of the present invention.

[0147] In the process of implementing the computer program product, the computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0148] It should be understood that various forms of the flow shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0149] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for testing system functions, characterized in that, Including: In response to an event of performing a functional test on a system under test based on test requirement information in the current round, at least one process to be tested is determined based on the test requirement information; wherein, at least one test node is included in the process to be tested, and the test node is used to test a function module to be tested in the system under test; For some test nodes in the process to be tested, test input information corresponding to the function module to be tested of the test node is determined, and the test input information is transmitted to the function module to be tested, so that the function module to be tested processes the test input information and feeds back a processing result; When receiving the processing result fed back based on the function module to be tested, the text consumption amount and the processing response duration generated during the process from triggering the function module to be tested to receiving the processing result are determined; wherein, the text consumption amount is used to characterize the number of text characters processed; For each process to be tested, based on the text consumption amount and / or the processing response duration corresponding to some of the function modules to be tested in the process to be tested, a process function performance attribute corresponding to the process to be tested is determined, and based on the process function performance attribute, a test result of testing the system under test is determined.

2. The method according to claim 1, characterized in that The transmitting the test input information to the function module to be tested, so that the function module to be tested processes the test input information and feeds back a processing result includes: The test input information is sent to the function module to be tested, so that when it is determined based on the function module to be tested that there is an associated function module depending on it in the process to be tested, the processing result obtained by processing the test input information based on the function module to be tested is used as the test input information of the associated function module and sent to the associated function module, so that the associated function module processes the received test input information and feeds back a processing result.

3. The method according to claim 1, characterized in that, After receiving the processing result fed back based on the function module to be tested, the method further includes: If the processing result is a processing failure result, the test operation for the process to be tested to which the function module to be tested belongs is stopped.

4. The method according to claim 1, characterized in that The test requirement information includes the number of single-round tests, and the number of single-round tests includes at least two tests. The determining of the process function performance attribute corresponding to the process to be tested based on the text consumption amount and / or the processing response duration corresponding to some of the function modules to be tested in the process to be tested includes: For each process to be tested, based on the text consumption amount corresponding to some of the function modules to be tested in the process to be tested under at least two tests and the number of single-round tests, the average text consumption attribute in the process function performance attribute corresponding to the process to be tested is determined; and / or, Based on the processing response duration corresponding to some of the function modules to be tested in the process to be tested under at least two tests and the number of single-round tests, the average processing response attribute in the process function performance attribute is determined.

5. The method according to claim 1, wherein The determining of the test result of testing the system under test based on the process function performance attribute includes: Determine the test pass rate of all the to-be-tested processes; Based on the test pass rate and the process functional performance attributes, determine the test result of the to-be-tested system.

6. The method according to claim 5, wherein The determining the test result of the to-be-tested system based on the test pass rate and the process functional performance attributes includes: Obtain the historical text consumption attribute, historical processing response attribute, and historical process test pass attribute corresponding to each of the to-be-tested processes obtained when performing a functional test on the to-be-tested system based on test requirement information in the previous round; If the test pass rate is less than the historical process test pass attribute, determine that the test result of the to-be-tested system is a test failure; If the average text consumption attribute of the processes in the process functional performance attributes is greater than the historical text consumption attribute, determine that the test result of the to-be-tested system is a test failure; If the average processing response attribute of the processes in the process functional performance attributes is greater than the historical text consumption attribute, determine that the test result of the to-be-tested system is a test failure.

7. The method according to claim 1, wherein The test requirement information includes the number of single-round tests, and the number of single-round tests includes at least two tests. After determining the test result of the to-be-tested system based on the process functional performance attributes, the method further includes: When the test result is inconsistent with the preset result, for each to-be-tested functional module in the to-be-tested process, based on the text consumption amount corresponding to the to-be-tested functional module under at least two tests and the number of single-round tests, determine the average text consumption attribute corresponding to the to-be-tested functional module, and when the average text consumption attribute is greater than the historical text consumption amount of the to-be-tested functional module, determine that the test result of the to-be-tested functional module is a test failure; Based on the processing response duration corresponding to the to-be-tested functional module under at least two tests and the number of single-round tests, determine the average processing response attribute corresponding to the to-be-tested functional module, and when the average processing response attribute is greater than the historical processing response duration of the to-be-tested functional module, determine that the test result of the to-be-tested functional module is a test failure.

8. The method according to claim 1 or 7, characterized in that, After determining the test result of the to-be-tested system based on the process functional performance attributes, the method further includes: When the test result is inconsistent with the preset result, generate a warning prompt message to perform functional maintenance on the to-be-tested system based on the warning prompt message.

9. The method according to claim 7, characterized in that After determining the test result of the to-be-tested system based on the process functional performance attributes, the method further includes: When the test result is consistent with the preset result, based on the text consumption amount corresponding to the to-be-tested functional module under at least two tests and the average text consumption attribute, determine the text standard deviation of the to-be-tested functional module; Based on the processing response duration corresponding to the to-be-tested functional module under at least two tests and the average processing response attribute, determine the duration standard deviation of the to-be-tested functional module; Based on the text standard deviation and / or the duration standard deviation, determine the system functional performance attributes of the to-be-tested system.

10. A test device for system functions, characterized in that, Including: A to-be-tested process determination module, configured to respond to an event of performing a functional test on a to-be-tested system based on test requirement information in the current round, and determine at least one to-be-tested process based on the test requirement information; wherein, at least one test node is included in the to-be-tested process, and the test node is configured to test a to-be-tested functional module in the to-be-tested system; A test input information determination module, configured to, for some test nodes in the to-be-tested process, determine test input information of the to-be-tested functional module corresponding to the test node, and transmit the test input information to the to-be-tested functional module, so that the to-be-tested functional module processes the test input information and feeds back a processing result; A text consumption determination module, configured to, when receiving a processing result fed back by the to-be-tested functional module, determine the text consumption and the processing response duration generated during the process from triggering the to-be-tested functional module to receiving the processing result; wherein, the text consumption is used to characterize the number of text characters processed; A test result determination module, configured to, for each to-be-tested process, determine a process function performance attribute corresponding to the to-be-tested process based on the text consumption and / or the processing response duration corresponding to some of the to-be-tested functional modules in the to-be-tested process, and determine a test result of testing the to-be-tested system based on the process function performance attribute.

Images