A functional testing method for information systems based on artificial intelligence

Abstract

The present invention discloses a functional testing method for an information system based on an artificial intelligence entity. The present invention relates to the technical field of system testing. The method comprises: in response to an event of functional testing a system to be tested based on test requirement information, determining at least one process to be tested based on the test requirement information; determining the test input information of the functional module to be tested corresponding to the test node, and transferring the test input information to the functional module to be tested; upon receiving a processing result based on feedback from the functional module to be tested, determining the text consumption and processing response time generated in the process from triggering the functional module to be tested to receiving the processing result; determining the process function performance attributes corresponding to the process to be tested based on the text consumption and / or processing response time corresponding to the functional module to be tested in the process to be tested, and determining the test results of the system to be tested based on the process function performance attributes. This technical solution improves the convenience and efficiency of system function testing.

Description

Technical Field

[0001] The present invention relates to the technical field of system testing, and in particular to a function testing method for an information system based on an artificial intelligence entity. Background Art

[0002] With the development of computer technology, more and more information systems have emerged. In order to ensure the stability of the operation of information systems, it is usually necessary to test the functions of the information systems themselves to test whether their functions are normal.

[0003] Current testing methods primarily rely on manual testing or specific test scripts to test fixed system content. However, when information system functionality changes or updates, manual retesting or the redesign of new test scripts are required, resulting in high testing costs and low efficiency. Summary of the Invention

[0004] The present invention provides a method for functional testing of an information system based on an artificial intelligence entity, so as to reduce the cost of functional testing of the system and achieve the technical effect of improving testing efficiency.

[0005] According to one aspect of the present invention, a method for testing system functions is provided, the method comprising:

[0006] In response to an event of performing a functional test on the system to be tested based on test requirement information in the current round, determining at least one process to be tested based on the test requirement information; wherein the process to be tested includes at least one test node, and the test node is used to test the functional module to be tested in the system to be tested;

[0007] For some test nodes in the process to be tested, determining test input information of the functional module to be tested corresponding to the test node, and transferring the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back a processing result;

[0008] Upon receiving a processing result based on feedback from the functional module to be tested, determining the text consumption amount and processing response time generated from triggering the functional module to be tested to receiving the processing result; wherein the text consumption amount is used to represent the number of text characters processed;

[0009] For each of the processes to be tested, based on the text consumption and / or the processing response time corresponding to some of the functional modules to be tested in the process to be tested, the process functional performance attributes corresponding to the process to be tested are determined, and based on the process functional performance attributes, the test results of the system to be tested are determined.

[0010] According to another aspect of the present invention, there is provided a device for testing system functions, the device comprising:

[0011] A module for determining a process to be tested, configured to determine, in response to an event in a current round in which a functional test is performed on the system to be tested based on test requirement information, at least one process to be tested based on the test requirement information; wherein the process to be tested includes at least one test node, and the test node is configured to test a functional module to be tested in the system to be tested;

[0012] a test input information determination module, configured to determine, for some test nodes in the process to be tested, the test input information of the functional module to be tested corresponding to the test node, and to transfer the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back a processing result;

[0013] a text consumption determination module, configured to, upon receiving a processing result based on feedback from the functional module to be tested, determine the text consumption generated from triggering the functional module to be tested to receiving the processing result and the processing response time; wherein the text consumption is used to represent the number of text characters processed;

[0014] A test result determination module is used to determine, for each of the processes to be tested, the process functional performance attributes corresponding to the process to be tested based on the text consumption and / or the processing response time corresponding to some of the functional modules to be tested in the process to be tested, and to determine the test results of the system to be tested based on the process functional performance attributes.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising:

[0016] at least one processor; and a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method for testing system functions described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for testing system functions according to any embodiment of the present invention when executed.

[0019] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method for testing system functions as described in any embodiment of the present invention.

[0020] The technical solution of the embodiment of the present invention is to determine at least one process to be tested based on the test requirement information by responding to the event of functional testing of the system to be tested based on the test requirement information in the current round; the process to be tested includes at least one test node, and the test node is used to test the functional module to be tested in the system to be tested; determine the test input information of the functional module to be tested corresponding to the test node, and pass the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back the processing result; when receiving the processing result based on the feedback of the functional module to be tested, determine the text consumption generated in the process from triggering the functional module to be tested to receiving the processing result and the processing response time; for each process to be tested, based on the process to be tested, The text consumption and / or processing response time corresponding to some functional modules to be tested in the test process are determined, the process functional performance attributes corresponding to the process to be tested are determined, and based on the process functional performance attributes, the test results of the system to be tested are determined, which solves the problem of high testing cost and low efficiency in the existing technology that relies on manual or script testing. It realizes the functional testing of the system to be tested by the intelligent agent based on the test requirement information, and determines at least one process to be tested based on the test requirement information, and then determines the test input information of the functional module to be tested, and transmits the test input information to the functional module to be tested, so that the functional module to be tested can automatically process the test input information and feedback the processing results, thereby improving the convenience and efficiency of the test. Furthermore, when receiving the processing results based on the feedback from the functional module to be tested, the text consumption and processing response time generated in the process from triggering the functional module to be tested to receiving the processing results are determined; based on the text consumption and / or processing response time corresponding to some functional modules to be tested in the process to be tested, the process function performance attributes of the process to be tested are evaluated, and while improving the accuracy of the process function performance evaluation, the system to be tested is tested based on the process function performance attributes, so that when the system is abnormal, the system can be maintained in time to ensure the accuracy of the system process function execution, ensure the stability of the system operation, and meet user needs.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is a flowchart of a method for testing system functions provided according to the first embodiment of the present invention;

[0024] Figure 2 is a schematic diagram for characterizing a process to be tested provided according to the first embodiment of the present invention;

[0025] Figure 3 This is a flowchart of a method for testing system functions provided in accordance with a second embodiment of the present invention;

[0026] Figure 4 This is a schematic structural diagram of a device for testing system functions according to a third embodiment of the present invention;

[0027] Figure 5 The present invention is a schematic structural diagram of an electronic device for implementing the method for testing system functions according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] It should be noted that before introducing the technical solution provided by the present invention, an illustrative description of the application scenario may be given first.

[0031] The application scenario of the embodiments of the present invention can be any scenario where system functions need to be tested. For example, the technical solutions provided by the embodiments of the present invention can be integrated into an intelligent agent. The intelligent agent can be integrated into any system whose functions are to be tested; it can also be independent of the system whose functions are to be tested. The intelligent agent can be manifested as a software system, a module, a robot, or an interface. The system whose functions are to be tested is the system to be tested described below. For example, the system to be tested can be an information system for processing information or information flows, such as an information system that includes functions such as input, storage, processing, output, and control of information.

[0032] Of course, a corresponding intelligent agent can also be developed according to the solution provided by the embodiment of the present invention, so as to realize functional testing of the system to be tested based on the intelligent agent's autonomous exploration of various functions and interfaces of the system to be tested.

[0033] Example 1

[0034] Figure 1 This is a flowchart of a method for testing system functions according to the first embodiment of the present invention. This embodiment is applicable to the case of testing the functions of an information system. The method can be executed by a system function testing device. The system function testing device can be implemented in the form of hardware and / or software. The system function testing device can be configured in a computing device. Figure 1 As shown, the method includes:

[0035] S110 , in response to an event of performing functional testing on the system to be tested based on test requirement information in the current round, determining at least one process to be tested based on the test requirement information; the process to be tested includes at least one test node, and the test node is used to test the functional module to be tested in the system to be tested.

[0036] Test requirement information can be information describing the requirements and demands for testing the system under test. For example, test requirement information includes, but is not limited to, user roles, test rounds, number of tests per round, test frequency, and test time. Test frequency refers to the interval at which functional testing of the system under test is performed; test rounds refer to the number of test rounds required for the system under test; number of tests refers to the number of functional tests required for the system under test in a single round; and test time refers to the time when functional testing of the system under test is performed. User roles refer to the roles used to perform functional testing of the system under test. For example, user roles can be user roles in different positions or departments. The system under test includes multiple functional modules, each of which implements a specific function in the system. For example, a functional module can be a calculation module, a verification module, an audit module, a data transmission module, and so on. These are not described in detail here. A process under test can be a workflow consisting of a series of operational steps that must be executed in a specific order. That is, during system testing, a series of operational steps must be executed in a specific order to verify the proper functioning of a function or business logic. Each operational step is a test node in the process and corresponds to a functional module under test. That is, the process to be tested is used to test one or more functional modules to be tested in the system to be tested.

[0037] In this embodiment, different test requirement information can be determined according to different test requirements. The test requirement information can be input in real time or imported from a database. When the test requirement information is received, it can be considered that an event of functional testing of the system to be tested has been responded to; or, the test requirement information includes a scheduled test task, and when the scheduled test task is triggered, it is considered that an event of functional testing of the system to be tested has been responded to. Further, the test requirement information can be parsed to clarify which functions to test and which test processes need to be executed, thereby determining at least one process to be tested and the functional modules to be tested corresponding to the test nodes involved in the process to be tested. For example, the process to be tested can be a user login process, a user registration process, a search process, a transaction process, an order processing process, an audit process, an information submission process, etc. Exemplarily, the process to be tested and the number of tests to be executed can be set according to the test requirements through the process orchestration capability of the artificial intelligence body, such as setting the test to complete the execution of a+1 times of the process to be tested.

[0038] For example, see Figure 2 The processes to be tested include process 1, process 2, and process 3; the functional modules to be tested in process 1 include module 1, module 2, module 3, and module 4; the functional modules to be tested in process 2 include module 5, module 6, module 7, module 8, and module 9; the functional modules to be tested in process 3 include module 3, module 4, module 7, module 8, and module 9.

[0039] It should be noted that the number of processes to be tested includes one or more. If it includes more than one, the type of function implemented by each process to be tested can be the same or different. Each process to be tested can be executed independently of each other, and each process to be tested can be tested in parallel or serially. It should also be noted that the system to be tested uses the information system differently depending on the user role, and the process of executing functions may also be different; that is, different user roles use the system to be tested in different ways and complete different tasks. At this time, based on the different user roles of the test requirement information, the use of the system to be tested by the corresponding user role can be simulated, and the system to be tested can be functionally tested, so as to improve the authenticity and accuracy of the test while improving the convenience of the test.

[0040] S120 , for some test nodes in the process to be tested, determine the test input information of the function module to be tested corresponding to the test node, and transfer the test input information to the function module to be tested, so that the function module to be tested processes the test input information and feeds back the processing result.

[0041] Among them, the test input information may refer to the information required by the functional module to be tested to realize its own function, that is, the test input information is the information used to test the function of the functional module to be tested. The processing result may be the result obtained after processing the test input information, or it may be a result indicating whether the test input information is successfully processed. It should be noted that the method of performing functional testing on each process to be tested is the same, and the method of performing functional testing on the functional modules to be tested of some test nodes in the process to be tested is also the same. The following is an example of performing functional testing on the functional module to be tested corresponding to any test node in any part of any process to be tested.

[0042] In this embodiment, during the process of functional testing of the functional module to be tested corresponding to a certain test node, the test input information of the functional module to be tested can be determined based on the functional implementation requirements or business attributes of the functional module to be tested, and the test input information can be submitted to the functional module to be tested. Among them, the functional implementation requirements can refer to the specific functional requirements that the functional module needs to implement, such as information including the functional characteristics it has. Business attributes can refer to the characteristics or rules that the functional module has in the business scenario, such as information related to business processes, data processing, user permissions, etc. Alternatively, the test input information of the functional module to be tested can be analyzed in combination with a historical knowledge base. The historical knowledge base can be determined based on the historical function processing records of the functional module to be tested of the system to be tested and the management regulations information of the system to be tested. After receiving the test input information, the functional module to be tested automatically processes the test input information to obtain a processing result.

[0043] It should be noted that after the functional module to be tested processes the test input information, it can also automatically transfer the processing results to the next functional module to be tested, so that the second functional module to be tested can continue to execute its functional logic based on the output of the previous module.

[0044] In this embodiment, transmitting test input information to the functional module under test so that the functional module under test processes the test input information and feeds back a processing result includes: sending the test input information to the functional module under test so that, when the functional module under test determines that there is an associated functional module that depends on the functional module under test in the process under test, the functional module under test uses the processing result of the test input information as test input information for the associated functional module, and sends the result to the associated functional module so that the associated functional module processes the received test input information and feeds back a processing result. The associated functional module may be a module that is triggered by the functional module under test to execute a function.

[0045] Specifically, the test input information is input into the functional module to be tested, and the functional module to be tested is triggered by a triggering method (such as calling an interface, clicking a button, sending an instruction, etc.), so that the functional module to be tested begins to process the incoming test input information. In addition, if there are associated functional modules in the process to be tested that depend on the functional module to be tested, then the functional module to be tested can pass the processing result as test input information into the associated functional module, so that the associated functional module receives the new test input information, processes it, generates the final processing result, and feeds back the result for subsequent verification or further processing by the next functional module. This enables multiple functional modules to work together to complete complex test tasks and improves the convenience of testing. It should be noted that when determining the test input information of the associated functional modules, the test input information can also be generated based on the assistance of the intelligent agent to ensure the accuracy of the test.

[0046] For example, it is assumed that in each process to be tested, all functional modules to be tested except the first functional module to be tested are used for review, verification and revision, and the contents of the review, verification and revision are different. The first functional module to be tested in each process to be tested needs to fill in the test input information. At this time, the knowledge base of the intelligent agent is analyzed through the large language model to determine the test input information of the functional module to be tested and submit it to the functional module to be tested. After submitting it to the functional module to be tested, the functional module to be tested processes the information, and the processing result can automatically enter the second functional module to be tested (i.e., the associated functional module) so that the second functional module to be tested continues to process and obtain the processing result. Alternatively, the test input information passed into the first functional module to be tested can be processed by the intelligent agent in combination with the specific needs of the associated functional module, so that the associated functional module can review and revise the processed information.

[0047] S130 : upon receiving a processing result based on feedback from the function module to be tested, determining the text consumption amount generated in the process from triggering the function module to be tested to receiving the processing result and the processing response time.

[0048] Text consumption represents the number of text characters processed. For example, text consumption can be the amount of text data transmitted and processed from the time the functional module under test is triggered to the time the processing result is received, or the number of tokens into which the text data is broken down. A token is the basic unit of text and can be a character such as a word, character, number, or symbol. Response processing time can be the time from sending a request to receiving a response.

[0049] In this embodiment, after the functional module to be tested processes the test input information, it returns the processing result. When the processing result of the functional module to be tested is obtained, the token consumption involved in the entire process from triggering the functional module to receiving its processing result can be determined as the text consumption, and the time interval from triggering the functional module to be tested to receiving its processing result can be determined as the processing response time. The performance and efficiency of the functional module tested this time can be evaluated by measuring the text consumption and the processing response time. For example, when using a large language model, the input text "123456" will be split into several tokens, and the reply generated by the model will also be split into several tokens. The sum of the tokens split in two rounds is the text consumption.

[0050] For example, the amount of text consumed from triggering a function module in a process to receiving its processing results can be recorded. For example, if a process numbered 2 has five function modules, the tokens (i.e., the amount of text consumed) consumed during the first test of that process can be represented as tk201, tk211, tk221, tk231, and tk241. The maximum number of subscripts is 4, indicating that the process has 4+1 function modules.

[0051] In order to ensure that the test process can be terminated in time when a module function error is detected to avoid invalid testing, after receiving the processing result based on the feedback of the tested functional module, if the processing result is a processing failure result, the test operation of the tested process to which the tested functional module belongs can be stopped.

[0052] In other words, the processing result returned by the module in the process to be tested can be used to determine whether the module has successfully executed its function. If the processing result is a failure, the test of the process to be tested is terminated, avoiding the need to continue to perform invalid test operations when failure is known, saving time and resources.

[0053] For example, if the feedback processing result is a processing failure result, it means that there may be reasons such as failure to pass the self-verification of the system to be tested or abnormality in the function of the system to be tested, which will result in the inability to pass to the next functional module to be tested for testing. Then the process to be tested at this time is a process that cannot be executed to the set functional module, that is, the process has not actually been executed to the final functional module.

[0054] S140. For each process to be tested, based on the text consumption and / or processing response time corresponding to some functional modules to be tested in the process to be tested, determine the process function performance attributes corresponding to the process to be tested, and based on the process function performance attributes, determine the test results of the system to be tested.

[0055] The test result can be the test result after executing the function of at least one process under test. The process function performance attributes can be used to characterize the efficiency and quality of executing the process under test and achieving its functions. For example, process function performance attributes can include response time, throughput, resource utilization, etc. Function refers to the specific tasks or operations that the process under test can complete.

[0056] In this embodiment, the execution performance of the process to be tested can be comprehensively evaluated to see whether it meets the business requirements based on the text consumption and / or processing response time corresponding to some of the functional modules to be tested in the process to be tested, and the process functional performance attributes corresponding to the process to be tested can be obtained. Accordingly, the process functional performance attributes corresponding to each process to be tested can be obtained. Furthermore, the process functional performance attributes corresponding to the process to be tested can be compared with the process functional performance attributes obtained in the previous round of testing the system to be tested based on the test requirement information to determine the test results of the system to be tested. For example, if the text consumption or response in the process functional performance attributes has increased significantly compared with the previous round of testing, it means that the overall process audit complexity of the system to be tested has increased. At this time, the test result can be determined as a test abnormality, that is, the test has not passed.

[0057] The technical solution of this embodiment is to determine at least one process to be tested based on the test requirement information in response to the event of functional testing of the system to be tested based on the test requirement information in the current round; the process to be tested includes at least one test node, and the test node is used to test the functional module to be tested in the system to be tested; determine the test input information of the functional module to be tested corresponding to the test node, and pass the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back the processing result; when receiving the processing result based on the feedback of the functional module to be tested, determine the text consumption and processing response time generated in the process from triggering the functional module to be tested to receiving the processing result; for each process to be tested, based on the process to be tested, The text consumption and / or processing response time corresponding to some functional modules to be tested in the test process are determined, the process functional performance attributes corresponding to the process to be tested are determined, and based on the process functional performance attributes, the test results of the system to be tested are determined, which solves the problem of high testing cost and low efficiency in the existing technology that relies on manual or script testing, and realizes the functional testing of the system to be tested by the intelligent agent based on the test requirement information, and determines at least one process to be tested based on the test requirement information, and then determines the test input information of the functional module to be tested, and transmits the test input information to the functional module to be tested, so that the functional module to be tested can automatically process the test input information and feedback the processing results, thereby improving the convenience and efficiency of the test. Furthermore, when receiving the processing results based on the feedback from the functional module to be tested, the text consumption and processing response time generated in the process from triggering the functional module to be tested to receiving the processing results are determined; based on the text consumption and / or processing response time corresponding to some functional modules to be tested in the process to be tested, the process function performance attributes of the process to be tested are evaluated, and while improving the accuracy of the process function performance evaluation, the system to be tested is tested based on the process function performance attributes, so that when the system is abnormal, the system can be maintained in time to ensure the accuracy of the system process function execution, ensure the stability of the system operation, and meet user needs.

[0058] Example 2

[0059] Figure 3 This is a flowchart of a method for testing system functions according to a second embodiment of the present invention. Based on the previous embodiment, the test requirement information optionally includes the number of single-round tests, where the number of single-round tests includes at least two tests. Accordingly, S140 is further refined. For specific implementations, please refer to the technical solution of this embodiment. Technical terms that are identical or corresponding to those in the previous embodiment are not repeated here.

[0060] like Figure 3 As shown, the method specifically includes the following steps:

[0061] S210. In response to an event of performing functional testing on the system to be tested based on test requirement information in the current round, determine at least one process to be tested based on the test requirement information; the process to be tested includes at least one test node, and the test node is used to test the functional module to be tested in the system to be tested.

[0062] S220 , for some test nodes in the process to be tested, determine the test input information of the functional module to be tested corresponding to the test node, and transfer the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back the processing result.

[0063] S230: upon receiving a processing result based on feedback from the function module to be tested, determining the text consumption amount generated in the process from triggering the function module to be tested to receiving the processing result and the processing response time.

[0064] S240. For each process to be tested, based on the text consumption corresponding to some functional modules to be tested in the process to be tested under at least two tests and the number of single-round tests, determine the process average text consumption attribute in the process functional performance attributes corresponding to the process to be tested; and / or, based on the processing response time corresponding to some functional modules to be tested in the process to be tested under at least two tests and the number of single-round tests, determine the process average processing response attribute in the process functional performance attributes.

[0065] The number of single-round tests may refer to the number of functional tests performed on the system to be tested based on the test requirement information in a single round. The process functional performance attributes include the average text consumption attribute of the process and / or the average processing response attribute of the process. The average text consumption attribute of the process can be used to characterize the average token consumption of the process to be tested under multiple tests in a single round. The average processing response attribute of the process can be used to characterize the average processing response time of the process to be tested under multiple tests in a single round.

[0066] In this embodiment, the sum of the text consumption corresponding to some or all of the functional modules to be tested in the process to be tested during multiple tests can be counted to obtain a sum value, and the sum value is divided by the number of single-round tests to obtain the average text consumption attribute of the process corresponding to the process to be tested.

[0067] For example, in the process of executing each process to be tested based on the agent, the overall token consumption and response time of each process to be tested can be calculated during the interaction between the agent and the system function to be tested, information completion, and information review. The token consumption (i.e., text consumption) of different functional modules to be tested in different processes to be tested can be calculated using The text consumption of all the processes to be tested is represented by the matrix express, Indicates the Text consumption matrix for each test execution.

[0068]

[0069] in, Indicates the index of different processes to be tested, It is the index of the functional module to be tested involved in a process to be tested. Indicates the test index (indicating the number of tests) executed in each round of testing. Indicates the The test execution The first step of the process to be tested It should be noted that the number of function modules to be tested is not necessarily the same for each process to be tested. The number of function modules in the process to be tested with the largest number of function modules in the system to be tested can be subtracted by one (because the matrix starts counting from 0, and the number of function modules to be tested generally starts counting from 1). , then the text consumption of the functional module without testing is recorded as 0. Further, combined with the recorded data in the text consumption matrix, the average token consumption data of each process to be tested (i.e., the average text consumption attribute of the process) is calculated.

[0070] Optionally, the calculation formula for determining the average text consumption attribute of the process to be tested can be expressed as: ;in Take 0 to , Indicates the number of tests, Take 0 to , Indicates the Functional modules to be tested, For the processes to be tested; Indicates the The text consumption attribute of each process to be tested. The text consumption of each functional module to be tested in the same process to be tested under different test times can be summed up as the total text consumption of the process to be tested in a single test. After the test process is completed, the total text consumption is divided by the number of tests to obtain During the test, all processes of the process to be tested have text consumption attributes.

[0071] In this embodiment, the sum of the processing response times corresponding to some or all of the functional modules to be tested in the process to be tested during multiple tests can also be counted to obtain a sum value, and the sum value is divided by the number of single-round tests to obtain the process processing response attributes corresponding to the process to be tested.

[0072] The overall processing response time of each process to be tested can be Indicates that the processing response time of all processes to be tested is expressed in matrix express, Indicates the Matrix of processing response times for each test execution.

[0073]

[0074] in, Indicates the Test execution The first step of the process to be tested It should be noted that the number of function modules to be tested is not necessarily the same for each process to be tested. The number of function modules in the process to be tested with the largest number of function modules in the system to be tested can be subtracted by one (because the matrix starts counting from 0, and the number of function modules to be tested generally starts counting from 1). , then the processing response time of no function module to be tested is recorded as 0. Further, combined with the recorded data in the processing response time matrix, the average consumption time of each process to be tested is calculated (that is, all processes process response attributes).

[0075] Optionally, the calculation formula for determining the process processing response attribute of the process to be tested can be expressed as: ;in Take 0 to , Indicates the number of tests, Take 0 to , Indicates the Functional modules to be tested, For the A process to be tested. Indicates the The processing response attributes of each process to be tested are processed. The processing response time of each function module to be tested in the same process to be tested under different test times can be summed up as the total processing response time of the process to be tested in a single test. After testing the process, divide the total processing response time by the number of tests to get The processes of the process to be tested all process response attributes.

[0076] S250: Determine the test pass rate of all processes to be tested.

[0077] The test pass rate can be used to represent the probability of passing the process to be tested. For example, a higher test pass rate means that more processes to be tested have their functions successfully implemented.

[0078] In this embodiment, the test pass rate can be determined based on the ratio of the number of to-be-tested processes that have passed the test to the total number of to-be-tested processes. Optionally, the test pass rate of at least one to-be-tested process can be determined by: for each to-be-tested process, when all to-be-tested functional modules in the to-be-tested process have passed the test, updating the process test pass data; and determining the test pass rate based on the process test pass data and the number of processes in the at least one to-be-tested process.

[0079] In other words, whenever all the functional modules in a process to be tested pass the test, the process test pass data can be incremented by one, thus updating the process test pass data. The ratio between the updated process test pass data and the number of processes in at least one process to be tested is used as the test pass rate. The advantage of this setting is that by evaluating the overall test results of all processes to be tested, the execution status of the process of the system to be tested can be accurately determined, thereby improving the accuracy of the overall performance evaluation of the system to be tested.

[0080] For example, during the testing of all the functional modules in the process to be tested, if the processing result of the last functional module to be tested is a successful result, it is considered that the final step of the process to be tested has actually been executed, and the process test pass data is automatically incremented by 1. Similarly, for each (e.g., m) process to be tested in the system to be tested, the agent performs one test, i.e., completes the execution of m processes. The execution of each process includes at most n functional modules to be tested. When the execution reaches the last functional module to be tested in the process, the process test pass data is automatically incremented by 1.

[0081] Optionally, the calculation method for determining the test pass rate of all processes to be tested can be expressed as:

[0082] ;in, Indicates the test pass rate; Indicates that the process test passed the data; Represents the number of processes. Note that the number of functional tests performed on the system under test based on the test requirements in this round can include multiple times. In this case, the number of processes can be the product of the number of processes in at least one process under test and the number of tests. The process test pass data is the number of process tests that passed after multiple functional tests on each process under test.

[0083] S260. Determine the test result of the system to be tested based on the test pass rate and the process function performance attributes.

[0084] In this embodiment, the test pass rate, the average text consumption attribute of the process in the process function performance attributes and / or the average processing response attribute of the process obtained in this round of testing can be combined with the test pass rate and process function performance attributes obtained in the previous round of testing to obtain the test results of the system to be tested. By comprehensively considering the test pass rate of the process, the average text consumption attribute of the process and the average processing response attribute of the process, the performance and stability of the system process can be comprehensively evaluated, and it can be accurately judged whether the system process meets the expected performance requirements. For example, if the test pass rate is significantly lower than that of the previous round, it means that the process cannot be accurately executed in the system to be tested. At this time, the test result can be determined to be a test abnormality.

[0085] In this embodiment, the method for determining the test result of the system to be tested can specifically be: obtaining the historical text consumption attributes and historical processing response attributes corresponding to each process to be tested and the historical process test pass attributes obtained when the functional test of the system to be tested was performed based on the test requirement information in the previous round; if the test pass rate is less than the historical process test pass attribute, then the test result of the system to be tested is determined to be a test failure; if the process average text consumption attribute in the process functional performance attribute is greater than the historical text consumption attribute, then the test result of the system to be tested is determined to be a test failure; if the process average processing response attribute in the process functional performance attribute is greater than the historical text consumption attribute, then the test result of the system to be tested is determined to be a test failure.

[0086] The historical process test pass attribute may refer to the test pass rate obtained during the previous round of functional testing of the system under test based on the test requirement information. The historical text consumption attribute may refer to the average text consumption obtained during the previous round of functional testing of the system under test based on the test requirement information. The historical processing response attribute may refer to the average processing response time obtained during the previous round of functional testing of the system under test based on the test requirement information.

[0087] Specifically, when performing functional testing, historical test data from a previous round of functional testing of the system to be tested based on test requirement information can be obtained. This data includes: historical process test pass attributes, historical text consumption attributes, and historical processing response attributes. Furthermore, the test pass rate obtained from the current test can be compared with the historical process test pass attributes. If the test pass rate of the current test is less than the historical process test pass attributes, it indicates that the process in the system to be tested cannot be accurately executed compared to the previous test. In this case, the test result of the test of the system to be tested is determined to be a test failure. If the process processing response attribute is greater than the historical text consumption data corresponding to the process to be tested, it indicates that there may be a performance problem, and the test result of the test of the system to be tested is determined to be a test failure. It should be noted that the text consumption of different functional modules to be tested in different processes to be tested can also be compared with the historical text consumption of different functional modules to be tested in different processes to be tested obtained from a previous round of functional testing of the system to be tested based on test requirement information. If the text consumption of certain functional modules in the current test is higher than the historical text consumption, it indicates that there may be a performance problem, and the test result is determined to be a test failure. You can also compare the processing response time of different functional modules to be tested in different processes to be tested with the historical processing response time. If the processing response time of some functional modules in the current test is longer than the historical processing response time, it indicates that there may be a processing performance problem, and the test result is determined to be a test failure. Alternatively, you can also determine the test result of the system to be tested by comprehensively considering the evaluation results of the test pass rate, text consumption, and processing response time. For example, if the test pass rate is lower than the test pass attribute of the historical process, it is directly determined that the test has failed; or, if the test pass rate is in line with expectations, but the text consumption or processing response time exceeds the historical data range, the test result is determined to be a test failure; or, if all indicators are in line with or better than the historical data, it is determined to be a test pass. Through the above steps, you can comprehensively and accurately evaluate whether the functions and performance of the process of the system to be tested are normal.

[0088] It should be noted that if this is your first test, you only need to record the text consumption and processing response time. This will facilitate subsequent determination of the text consumption and processing response time during the test, and can be combined with the previously recorded data to analyze the test results of the system under test.

[0089] In order to accurately locate the anomaly when there is an anomaly in the system, after determining the test result of the system to be tested based on the process function performance attributes, when the test result is inconsistent with the preset result, for each functional module to be tested in the process to be tested, based on the text consumption corresponding to the functional module to be tested under at least two tests and the number of single-round tests, determine the module average text consumption attribute corresponding to the functional module to be tested, and when the module average text consumption attribute is greater than the historical text consumption of the functional module to be tested, determine the test result of the functional module to be tested as test failure; based on the processing response time corresponding to the functional module to be tested under at least two tests and the number of single-round tests, determine the module average processing response attribute corresponding to the functional module to be tested, and when the module average processing response attribute is greater than the historical processing response time of the functional module to be tested, determine the test result of the functional module to be tested as test failure.

[0090] The historical text consumption and historical processing response time of the functional module under test are obtained from the previous round of functional testing of the system under test based on the test requirements. The module average text consumption attribute can be used to represent the average amount of text consumed by the functional module under test.

[0091] Specifically, the text consumption corresponding to the same functional module under test under at least two tests can be summed to obtain a sum value, and the quotient between the sum value and the number of tests can be used as the module-average text consumption attribute corresponding to the functional module under test. When the module-average text consumption attribute is greater than the historical text consumption of the functional module under test, or when the value of the module-average text consumption attribute is greater than the historical text consumption of the functional module under test by more than a preset threshold, it is considered that the function of the functional module under test is not being executed normally, and the test result of the test of the functional module under test is determined to be a test failure. Alternatively, the processing response time corresponding to the same functional module under test under at least two tests can be summed to obtain a sum value, and the quotient between the sum value and the number of tests can be used as the module-average processing response attribute corresponding to the functional module under test. When the module-average processing response attribute is greater than the historical processing response time of the functional module under test, or when the value of the module-average processing response attribute is greater than the historical processing response time of the functional module under test by more than a preset threshold, it is considered that the function of the functional module under test is not being executed normally, and the test result of the test of the functional module under test is determined to be a test failure. This allows the tested functional modules that fail the test to be sent to the target terminal or a test report to be generated, so that users can view the test report through the terminal and perform accurate maintenance on the tested system.

[0092] For example, when the test result of the system to be tested is that the test fails, it indicates that the system has obvious abnormalities, and at this time, the text consumption matrix and the time consumption matrix are fitted.

[0093] The text consumption matrix is ​​expressed as:

[0094] The time consumption matrix is ​​expressed as:

[0095] in, Indicates the The first step of the process to be tested Each module of the functional module to be tested has text consumption attributes; Indicates the The first step of the process to be tested All modules of the functional module under test process response attributes. If there is a significant order of magnitude difference in the data from the previous test process, it means that the second function module under test in the first process under test of the system under test has a suspected abnormality during use, which may indicate that the system under test has a suspected abnormality during the process setting. If the data differs significantly by an order of magnitude from the previous test, it indicates that the first module under test in the second process under test of the system under test is experiencing a suspected anomaly during use, which could indicate other reasons, such as the system under test being inaccessible. When the system under test fails the test, further detailed analysis of which module under test in which process under test experienced the anomaly can effectively reduce the amount of data calculations and accurately locate the abnormal module.

[0096] To enable timely system maintenance when an anomaly occurs, after determining a test result for the system under test based on text consumption and / or processing response time corresponding to some functional modules under test in at least one process under test, the method may further include generating an early warning message when the test result is inconsistent with a preset result, so as to perform functional maintenance on the system under test based on the early warning message. The preset result is a test pass.

[0097] Specifically, when the test result is that the test fails, it is considered that the test result is inconsistent with the preset result. At this time, an early warning prompt information is generated to promptly notify relevant personnel or systems through the early warning prompt information. According to the early warning prompt information, the test system is functionally checked, optimized or repaired to ensure the performance and functional stability of the system.

[0098] In order to further clearly analyze the performance of the system when no abnormalities occur in the system to be tested, after determining the test results of the system to be tested based on the process function performance attributes, when the test results are consistent with the preset results, the text standard deviation of the function module to be tested can be determined based on the text consumption corresponding to the function module to be tested under at least two tests and the average text consumption attributes of the module; the duration standard deviation of the function module to be tested can be determined based on the processing response time corresponding to the function module to be tested under at least two tests and the average processing response attributes of the module; and the system function performance attributes of the system to be tested can be determined based on the text standard deviation and / or duration standard deviation.

[0099] Among them, the process processing response attribute is determined based on the processing response time corresponding to some functional modules to be tested in the process to be tested under at least two tests; the system functional performance attribute can be used to characterize the efficiency and quality of executing the system to be tested to realize its functions.

[0100] Specifically, when the test result is a test pass, it is considered that the test result is consistent with the preset result. For each function module to be tested in each process to be tested, the difference between the text consumption corresponding to the function module to be tested during each test and the module-average processing response attribute of the function module to be tested can be calculated respectively, and each difference can be summed to obtain a sum value, and the quotient between the sum value and the number of tests is used as the text standard deviation of the function module to be tested. In this way, the difference between the processing response duration corresponding to the function module to be tested during each test and the module-average processing response attribute of the function module to be tested can be calculated respectively, and each difference can be summed to obtain a sum value, and the quotient between the sum value and the number of tests is used as the duration standard deviation of the function module to be tested. Further, in combination with the text standard deviation and / or duration standard deviation, the system function performance attributes of the system to be tested are analyzed. For example, the smaller the text standard deviation or duration standard deviation, the more stable the system is, that is, the more reasonable the process to be tested is set in the system to be tested. It should be noted that if there are many functional modules or processes to be tested in the process to be tested, a certain amount of data can be randomly extracted from them to participate in the calculation, so as to improve the accuracy of the standard deviation in representing the system stability.

[0101] Exemplarily, the calculation formula for determining the text standard deviation of the functional module to be tested can be expressed as:

[0102] ;in, Indicates the The first of the processes to be tested The text standard deviation of the functional modules to be tested.

[0103] The calculation formula for determining the standard deviation of the duration of the functional module to be tested can be expressed as:

[0104] ;in, Indicates the The first of the processes to be tested The standard deviation of the duration of the functional modules to be tested.

[0105] The technical solution of this embodiment determines the process average text consumption attribute in the process functional performance attributes corresponding to the process to be tested based on the text consumption corresponding to some functional modules to be tested in the process to be tested under at least two tests and the number of single-round tests; and / or determines the process average processing response attribute in the process functional performance attributes based on the processing response time corresponding to some functional modules to be tested in the process to be tested under at least two tests and the number of single-round tests, and at the same time, determines the test pass rate of all processes to be tested. Finally, based on the test pass rate and the process functional performance attributes, determines the test results of the system to be tested, thereby improving the accuracy of process evaluation.

[0106] Example 3

[0107] Figure 4 FIG. 1 is a schematic diagram of a structure of a system function testing device according to the third embodiment of the present invention. Figure 4 As shown, the device includes: a to-be-tested process determination module 310 , a test input information determination module 320 , a text consumption determination module 330 and a test result determination module 340 .

[0108] Among them, the module for determining the process to be tested 310 is used to respond to the event of performing functional testing on the system to be tested based on the test requirement information in the current round, and determine at least one process to be tested based on the test requirement information; wherein, the process to be tested includes at least one test node, and the test node is used to test the functional module to be tested in the system to be tested; the test input information determination module 320 is used to determine the test input information of the functional module to be tested corresponding to the test node for some test nodes in the process to be tested, and pass the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back the processing results ; A text consumption determination module 330 is used to determine the text consumption and processing response time generated from triggering the functional module to be tested to receiving the processing result based on the feedback of the functional module to be tested, when receiving the processing result; wherein, the text consumption is used to characterize the number of processed text characters; a test result determination module 340 is used to determine, for each of the processes to be tested, the process function performance attributes corresponding to the process to be tested based on the text consumption and / or the processing response time corresponding to some of the functional modules to be tested in the process to be tested, and determine the test results of the system to be tested based on the process function performance attributes.

[0109] The technical solution of this embodiment is to determine at least one process to be tested based on the test requirement information in response to the event of functional testing of the system to be tested based on the test requirement information in the current round; the process to be tested includes at least one test node, and the test node is used to test the functional module to be tested in the system to be tested; determine the test input information of the functional module to be tested corresponding to the test node, and pass the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back the processing result; when receiving the processing result based on the feedback of the functional module to be tested, determine the text consumption and processing response time generated in the process from triggering the functional module to be tested to receiving the processing result; for each process to be tested, based on the process to be tested, The text consumption and / or processing response time corresponding to some functional modules to be tested in the test process are determined, the process functional performance attributes corresponding to the process to be tested are determined, and based on the process functional performance attributes, the test results of the system to be tested are determined, which solves the problem of high testing cost and low efficiency in the existing technology that relies on manual or script testing, and realizes the functional testing of the system to be tested by the intelligent agent based on the test requirement information, and determines at least one process to be tested based on the test requirement information, and then determines the test input information of the functional module to be tested, and transmits the test input information to the functional module to be tested, so that the functional module to be tested can automatically process the test input information and feedback the processing results, thereby improving the convenience and efficiency of the test. Furthermore, when receiving the processing results based on the feedback from the functional module to be tested, the text consumption and processing response time generated in the process from triggering the functional module to be tested to receiving the processing results are determined; based on the text consumption and / or processing response time corresponding to some functional modules to be tested in the process to be tested, the process function performance attributes of the process to be tested are evaluated, and while improving the accuracy of the process function performance evaluation, the system to be tested is tested based on the process function performance attributes, so that when the system is abnormal, the system can be maintained in time to ensure the accuracy of the system process function execution, ensure the stability of the system operation, and meet user needs.

[0110] On the basis of the above-mentioned device, optionally, the test input information determination module 320 is used to send the test input information to the functional module to be tested, so that when it is determined based on the functional module to be tested that there is an associated functional module that depends on it in the process to be tested, the functional module to be tested will use the processing result obtained by processing the test input information as the test input information of the associated functional module, and send it to the associated functional module, so that the associated functional module processes the received test input information and feeds back the processing result.

[0111] On the basis of the above device, optionally, the device further includes:

[0112] The process test termination module is used to stop the test operation on the process to be tested to which the function module to be tested belongs if the processing result is a processing failure result.

[0113] Based on the above device, optionally, the test requirement information includes the number of single-round tests, and the number of single-round tests includes at least two tests. The test result determination module 340 includes:

[0114] The process function performance attribute determination unit is used to determine, for each of the processes to be tested, the process average text consumption attribute in the process function performance attributes corresponding to the process to be tested based on the text consumption corresponding to some of the function modules to be tested in the process to be tested under at least two tests and the number of single-round tests; and / or, determine the process average processing response attribute in the process function performance attributes based on the processing response time corresponding to some of the function modules to be tested in the process to be tested under at least two tests and the number of single-round tests.

[0115] Based on the above device, optionally, the test result determination module 340 includes:

[0116] A test pass rate determination unit, configured to determine the test pass rates of all the processes to be tested;

[0117] A test result determination unit is used to determine a test result of the system to be tested based on the test pass rate and the process function performance attribute.

[0118] Based on the above device, optionally, the test result determination unit includes:

[0119] A historical processing information determination unit is used to obtain historical text consumption attributes and historical processing response attributes corresponding to each process to be tested, as well as historical process test pass attributes, obtained during a previous round of functional testing of the system to be tested based on test requirement information;

[0120] a test result determination first unit, configured to determine that the test result of the system to be tested is a test failure if the test pass rate is less than the historical process test pass attribute;

[0121] a second test result determination unit, configured to determine that a test result of the system to be tested is a test failure if the process average text consumption attribute in the process function performance attribute is greater than the historical text consumption attribute;

[0122] The test result determining third unit is used to determine that the test result of the system to be tested is a test failure if the process processing response attribute in the process function performance attribute is greater than the historical text consumption attribute.

[0123] Based on the above device, optionally, the test requirement information includes the number of single-round tests, and the number of single-round tests includes at least two tests. The device further includes:

[0124] a module test result determination first unit configured to determine, for each functional module to be tested in the process to be tested, an average module text consumption attribute corresponding to the functional module to be tested based on the text consumption corresponding to the functional module to be tested under at least two tests and the number of single-round tests when the test result is inconsistent with the preset result, and determine that the test result of the functional module to be tested is a test failure if the average module text consumption attribute is greater than the historical text consumption of the functional module to be tested;

[0125] The module test result determines the second unit, which is used to determine the module processing response attributes corresponding to the functional module to be tested based on the processing response time corresponding to the functional module to be tested under at least two tests and the number of single-round tests, and when the module processing response attributes are greater than the historical processing response time of the functional module to be tested, determine that the test result of the functional module to be tested is a test failure.

[0126] On the basis of the above device, optionally, the device further includes:

[0127] The function maintenance unit is used to generate an early warning prompt information when the test result is inconsistent with the preset result, so as to perform function maintenance on the system to be tested based on the early warning prompt information.

[0128] On the basis of the above device, optionally, the device further includes:

[0129] a text standard deviation determining unit, configured to determine the text standard deviation of the functional module to be tested based on the text consumption corresponding to the functional module to be tested and the module average text consumption attribute under at least two tests when the test result is consistent with the preset result;

[0130] a function performance attribute duration standard deviation determining unit, configured to determine a duration standard deviation of the function module to be tested based on the processing response duration corresponding to the function module to be tested and the processing response attributes of the module under at least two tests;

[0131] A determining unit is configured to determine a system function performance attribute of the system to be tested based on the text standard deviation and / or the duration standard deviation.

[0132] The system function testing device provided by the embodiment of the present invention can execute the system function testing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0133] Example 4

[0134] Figure 5 Schematic diagram of an electronic device for implementing a method for testing system functions according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided for example only and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0135] like Figure 5 As shown, electronic device 10 includes at least one processor 11 and memory, such as read-only memory 12 and random access memory 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer programs stored in read-only memory 12 or loaded from storage unit 18 into random access memory 13. Random access memory 13 can also store various programs and data required for the operation of electronic device 10. Processor 11, read-only memory 12, and random access memory 13 are interconnected via bus 14. An input / output (I / O) interface 15 is also connected to bus 14.

[0136] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0137] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as methods for testing system functionality.

[0138] In some embodiments, the method for testing system functionality can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via read-only memory 12 and / or communication unit 19. When the computer program is loaded into random access memory 13 and executed by processor 11, one or more steps of the method for testing system functionality described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the method for testing system functionality in any other appropriate manner (e.g., via firmware).

[0139] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0143] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0144] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0145] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication unit 19, or installed from the storage unit 18, or installed from the read-only memory 12. When the computer program is executed by the processor 11, the above-mentioned functions defined in the method of the embodiment of the present invention are performed.

[0146] An embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by a processor, the computer program implements the method for testing system functions provided by any embodiment of the present invention.

[0147] During implementation, the computer program product may be written in one or more programming languages ​​or a combination thereof to perform the operations of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0149] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for testing system functions, characterized in that: include: In response to an event that a functional test is performed on the system to be tested based on test requirement information in the current round, at least one process to be tested is determined based on the test requirement information; wherein the process to be tested includes at least one test node, and the test node is used to test the functional module to be tested in the system to be tested; the test requirement information includes the number of tests in a single round, and the number of tests in a single round includes at least two tests; For some test nodes in the process to be tested, determining test input information of the functional module to be tested corresponding to the test node, and transferring the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back a processing result; Upon receiving a processing result based on feedback from the functional module to be tested, determining the text consumption amount and processing response time generated from triggering the functional module to be tested to receiving the processing result; wherein the text consumption amount is used to represent the number of text characters processed; For each of the processes to be tested, determining a process function performance attribute corresponding to the process to be tested based on the text consumption and / or the processing response time corresponding to some of the functional modules to be tested in the process to be tested, and determining a test result of the system to be tested based on the process function performance attribute; The method further comprises: When the test result is consistent with the preset result, the text standard deviation of the functional module to be tested is determined based on the text consumption corresponding to the functional module to be tested under at least two tests and the module average text consumption attribute; wherein the module average text consumption attribute is used to represent the average amount of text consumed by the functional module to be tested; Determining a standard deviation of the duration of the functional module to be tested based on the processing response duration corresponding to the functional module to be tested under at least two tests and the average processing response attribute of the module; wherein the average processing response attribute of the module is determined based on the processing response duration corresponding to the functional module to be tested under at least two tests and the number of single-round tests; Based on the text standard deviation and / or the duration standard deviation, a system function performance attribute of the system to be tested is determined.

2. The method according to claim 1, characterized in that The step of transmitting the test input information to the function module to be tested, so that the function module to be tested processes the test input information and feeds back a processing result, includes: The test input information is sent to the functional module to be tested, so that when it is determined based on the functional module to be tested that there is an associated functional module that depends on it in the process to be tested, the processing result obtained by processing the test input information based on the functional module to be tested is used as the test input information of the associated functional module, and is sent to the associated functional module, so that the associated functional module processes the received test input information and feeds back the processing result.

3. The method according to claim 1, characterized in that After receiving the processing result based on the feedback from the functional module to be tested, the method further includes: If the processing result is a processing failure result, the test operation on the process to be tested to which the functional module to be tested belongs is stopped.

4. The method according to claim 1, wherein The determining of the process function performance attribute corresponding to the process to be tested based on the text consumption and / or the processing response time corresponding to some of the functional modules to be tested in the process to be tested includes: For each of the processes to be tested, determining the process average text consumption attribute in the process functional performance attributes corresponding to the process to be tested based on the text consumption corresponding to some of the functional modules to be tested in the process to be tested under at least two tests and the number of single-round tests; and / or Based on the processing response time corresponding to some of the functional modules to be tested in the process to be tested under at least two tests and the number of single-round tests, the process processing response attribute in the process functional performance attribute is determined.

5. The method according to claim 1, wherein The determining of a test result of the system to be tested based on the process function performance attribute includes: Determine the test pass rate of all the processes to be tested; A test result of the system to be tested is determined based on the test pass rate and the process function performance attribute.

6. The method according to claim 5, characterized in that Determining a test result of the system to be tested based on the test pass rate and the process function performance attribute includes: Acquire historical text consumption attributes, historical processing response attributes, and historical process test pass attributes corresponding to each process to be tested, obtained during a previous round of functional testing of the system to be tested based on test requirement information; If the test pass rate is less than the historical process test pass attribute, determining that the test result of the system to be tested is a test failure; If the process average text consumption attribute in the process function performance attribute is greater than the historical text consumption attribute, determining that the test result of the system to be tested is a test failure; If the process processing response attribute in the process function performance attribute is greater than the historical text consumption attribute, it is determined that the test result of the system to be tested is a test failure.

7. The method according to claim 1, characterized in that After determining a test result of the system to be tested based on the process function performance attribute, the method further includes: When the test result is inconsistent with the preset result, for each functional module to be tested in the process to be tested, based on the text consumption corresponding to the functional module to be tested under at least two tests and the number of single-round tests, the module average text consumption attribute corresponding to the functional module to be tested is determined, and when the module average text consumption attribute is greater than the historical text consumption of the functional module to be tested, the test result of the functional module to be tested is determined to be test failure; Based on the processing response time corresponding to the functional module to be tested under at least two tests and the number of single-round tests, the module processing response attributes corresponding to the functional module to be tested are determined, and when the module processing response attributes are greater than the historical processing response time of the functional module to be tested, the test result of the functional module to be tested is determined to be test failure.

8. The method according to claim 1 or 7, characterized in that After determining a test result of the system to be tested based on the process function performance attribute, the method further includes: When the test result is inconsistent with the preset result, an early warning prompt information is generated, so as to perform functional maintenance on the system to be tested based on the early warning prompt information.

9. A device for testing system functions, characterized in that: include: A module for determining a process to be tested, configured to determine, in response to an event in a current round in which a functional test is performed on a system to be tested based on test requirement information, at least one process to be tested based on the test requirement information; wherein the process to be tested includes at least one test node, and the test node is used to test a functional module to be tested in the system to be tested; the test requirement information includes the number of tests in a single round, and the number of tests in a single round includes at least two tests; a test input information determination module, configured to determine, for some test nodes in the process to be tested, the test input information of the functional module to be tested corresponding to the test node, and to transfer the test input information to the functional module to be tested, so that the functional module to be tested processes the test input information and feeds back a processing result; a text consumption determination module, configured to, upon receiving a processing result based on feedback from the functional module to be tested, determine the text consumption generated from triggering the functional module to be tested to receiving the processing result and the processing response time; wherein the text consumption is used to represent the number of text characters processed; a test result determination module, configured to determine, for each of the processes to be tested, based on the text consumption and / or the processing response time corresponding to some of the functional modules to be tested in the process to be tested, a process function performance attribute corresponding to the process to be tested, and determine a test result of the system to be tested based on the process function performance attribute; The device further comprises: A text standard deviation determining unit, configured to determine the text standard deviation of the functional module to be tested based on the text consumption corresponding to the functional module to be tested under at least two tests and the module average text consumption attribute when the test result is consistent with a preset result; wherein the module average text consumption attribute is used to represent the average amount of text consumed by the functional module to be tested; a functional performance attribute duration standard deviation determining unit, configured to determine a duration standard deviation of the functional module under test based on the processing response duration corresponding to the functional module under test under at least two tests and the module average processing response attribute; wherein the module average processing response attribute is determined based on the processing response duration corresponding to the functional module under test under at least two tests and the number of single-round tests; The functional performance attribute determining unit is configured to determine the system functional performance attribute of the system to be tested based on the text standard deviation and / or the duration standard deviation.

Images