Automatic monitoring method and device of test machine and server
Through automated monitoring methods and script deployment, the problems of low efficiency and human error of the test machine are solved, efficient and accurate test results and data monitoring are achieved, and the testing accuracy and efficiency are improved.
Patent Information
- Application Number
- CN202411423642.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-01-14
AI Technical Summary
The prior art is inefficient and prone to human errors in the automated monitoring of test machines, resulting in deviations in test data, affecting product quality, and unable to meet the complex and changing test environment needs.
By obtaining initial test data and test plan, using execution scripts to automatically deploy test cases, perform task testing and exception monitoring processing, generate monitoring reports, realize automated monitoring and retest processing, and improve test accuracy and efficiency.
It significantly improves the accuracy and testing efficiency of system testing, reduces human resources waste, improves software quality and hardware service life, and realizes accurate capture and rapid analysis of abnormal errors.
Smart Images
Figure CN119310512A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of automatic monitoring, and in particular to an automatic monitoring method, device and server for a test machine. Background Art
[0002] With the increasing application of wafer level reliability (WLR) testers, the user groups of product applications are increasing, and the demands are also increasing. For the increasingly complex test systems and the ever-changing test environments, how to maintain the stability and sustainable operation of the products is particularly important. At present, relevant technologies have proposed that in order to make the products run stably and sustainably under various steady-state and non-steady-state environments, the executors need to manually operate the tests and observe the fluctuations of various indicators in the test system, so as to accumulate a large amount of trial and error data for analysis and evaluation in long-term continuous testing. However, in this solution, the efficiency of manually deploying the test environment is low and it is easy to make human errors, which leads to deviations in the test data and seriously affects the product quality. Summary of the invention
[0003] In view of this, an object of the present invention is to provide an automated monitoring method, device and server for a test machine, which can significantly improve the system testing accuracy and testing efficiency.
[0004] In the first aspect, an embodiment of the present invention provides an automated monitoring method for a testing machine, the method comprising: obtaining initial test data, a test plan and an execution script, wherein the initial test data is a test result obtained by performing a stability test on a system to be tested under a no-load state, and the test plan comprises: a plurality of test cases, each test case comprising: a test number, a test condition, a monitoring indicator, an execution step, an expected result and a test status; using the test plan and the execution script, determining at least one target test environment and a target monitoring content corresponding to each test case; performing task test processing and abnormal monitoring processing on the system to be tested according to the target test environment and the initial test data, determining the target monitoring result, and generating a monitoring report based on the target monitoring result.
[0005] In one embodiment, the step of obtaining initial test data includes: obtaining initial software and hardware data of the system to be tested, initializing the initial test system according to test requirements, performing stability testing on the system environment of the initial test system, and determining the initial test environment and initial test data that have reached a stable state.
[0006] In one embodiment, each test case includes: a test number, test conditions, and monitoring indicators; using a test plan and an execution script, the steps of determining at least one target test environment and target monitoring content corresponding to each test case include: using the execution script to extract key fields of the test conditions in each test case, and deploying at least one target test environment for the test case under different test pressures; obtaining monitoring indicators, automatically updating the target monitoring content of the software and hardware under the test case, and automatically deploying the initial test system for each test case according to the test number.
[0007] In one embodiment, each test case includes: execution steps and test status; performing task test processing and exception monitoring processing on the system to be tested according to the target test environment and initial test data, and determining the target monitoring result step, including: performing task test processing on the system to be tested according to the execution steps according to the target test environment, and determining the task test data; performing an exception comparative analysis on the task test data and the initial test data through a preset evaluation model, and determining a first monitoring result, wherein the first monitoring result is used to evaluate the task processing capability of the system to be tested under the target test environment; performing exception monitoring processing on the exception error report generated by the system to be tested during the task test process, and determining a second monitoring result; determining the first monitoring result and the second monitoring result as the target monitoring result, and storing the target monitoring result as a temporary monitoring data file in a preset path.
[0008] In one embodiment, abnormal monitoring is performed for abnormal errors that occur in the task testing process of the system to be tested, and the steps of determining a second monitoring result include: when an abnormal error is detected during the task testing process, obtaining an abnormal screenshot corresponding to the test interface of the system to be tested, and obtaining the corresponding first monitoring result to form a monitoring report; sending the abnormal screenshot and the monitoring report to the user end in real time to obtain the abnormal severity level assessment result fed back by the user end for the abnormal error; when the abnormal severity level assessment result is higher than the preset level threshold, determining the second monitoring result by retesting the test scenario where the abnormal error occurs.
[0009] In one embodiment, the step of determining the second monitoring result by retesting the test scenario where the abnormal error occurs includes: obtaining the retest requirement fed back by the execution user end, and retesting the test scenario where the abnormal error occurs multiple times according to a preset number of retests to determine the results of each retest; determining the abnormal error analysis result by comparing the results of each retest, and determining the abnormal error analysis result and the abnormal screenshot as the second monitoring result.
[0010] In one embodiment, the step of generating a monitoring report based on the target monitoring results includes: performing format conversion processing on the target monitoring results through a preset data conversion model, determining the monitoring report, and after the format conversion is completed, deleting the temporary monitoring data file corresponding to the target monitoring results in the preset path.
[0011] In the second aspect, an embodiment of the present invention also provides an automatic monitoring device for a testing machine, the device comprising: a data acquisition module, which acquires initial test data, a test plan and an execution script, wherein the initial test data is the test result obtained by performing a stability test on the system to be tested under a no-load state, and the test plan comprises: multiple test cases, each test case comprises: a test number, test conditions, monitoring indicators, execution steps, expected results and test status; an environment construction module, which uses the test plan and the execution script to determine at least one target test environment and target monitoring content corresponding to each test case; an automatic monitoring module, which performs task testing processing and abnormal monitoring processing on the system to be tested according to the target test environment and the initial test data, determines the target monitoring results, and generates a monitoring report based on the target monitoring results.
[0012] In a third aspect, an embodiment of the present invention further provides a server, comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement any one of the methods provided in the first aspect.
[0013] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.
[0014] The embodiments of the present invention bring the following beneficial effects:
[0015] An embodiment of the present invention provides an automated monitoring method, device and server for a test machine. After acquiring initial test data, a test plan and an execution script, the method uses the test plan and the execution script to determine at least one target test environment and target monitoring content corresponding to each test case, and performs task testing and abnormal monitoring on the system to be tested based on the target test environment and the initial test data, determines the target monitoring results, and generates a monitoring report based on the target monitoring results. The embodiment of the present invention can significantly improve the system testing accuracy and testing efficiency.
[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 A schematic diagram of a flow chart of an automated monitoring method for a test machine provided by an embodiment of the present invention;
[0020] Figure 2 A schematic diagram of a specific flow chart of an automated monitoring method for a test machine provided by an embodiment of the present invention;
[0021] Figure 3 A schematic diagram of a specific process of abnormal monitoring processing provided by an embodiment of the present invention;
[0022] Figure 4 A schematic diagram of the structure of an automatic monitoring device for a testing machine provided by an embodiment of the present invention;
[0023] Figure 5 A schematic diagram of the structure of a server provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0025] At present, with the increasing application of WLR testers, the user groups of product applications are increasing, and the demand is also increasing. In the process of continuous development, for the increasingly complex test system and the changing test environment, how to maintain the stability and sustainable operation of the product is particularly important. In the actual mass production process, in order to make the product stable and sustainable in various steady-state and non-steady-state environments, it is necessary to continuously experiment and improve the product in the development stage. Therefore, a large number of test tasks based on the WLR tester system will be carried out before the product is released. The executors will manually operate the test and observe the fluctuations of various indicators of software and hardware, and then accumulate a large amount of trial and error data in long-term continuous testing for analysis and evaluation. However, considering that such tests have a long test cycle and many monitoring contents, including software robustness, smoothness of use, response time, coupling, security, memory usage, compatibility, ease of use, hardware CPU temperature, graphics card temperature, motherboard temperature, hard disk temperature, tester hardware module temperature, GPLB connection, etc., manual operation or a single tool alone cannot fully cover the monitoring content, nor can it achieve the purpose of continuous operation day and night. It is both a waste of manpower and inefficient.
[0026] In addition, the system environment of the WLR test machine is changeable, from steady state to unsteady state, from a clean, idle test system to an increase in load, pressure and other changes that destroy the stable state. The process of manually deploying the test environment is slow, and multiple frequent operations are prone to human errors, which directly lead to test data deviations and seriously affect product quality. The execution personnel cannot immediately know about abnormal errors and alarms in the test process, which leads to extended test time and seriously affects the progress of production and research and development. At the same time, after the test, the existing technology usually stores the test data in text form. This solution is not conducive to reading and data analysis. The existing test tools cannot meet the compatibility of all test systems and test scenarios, and the deployment also takes up resources. Therefore, in order to improve test efficiency, avoid unnecessary waste of human resources, obtain accurate data monitoring records, real-time abnormal warning notifications, and intuitive analysis reports, an efficient and dynamic WLR test machine automatic monitoring solution is very necessary.
[0027] Based on this, the automated monitoring method, device and server of the test machine provided by the present invention have more comprehensive and accurate automated monitoring content compared to manual testing and tools, can more fully capture problems and potential risks in the software operation process, can be compatible with different WLR test machine systems, discover test bottlenecks, and easily implement repeated execution of a large number of test cases to verify software stability and reliability and improve software quality. In addition, abnormal errors can also be accurately captured, thereby reducing analysis time and costs introduced by human errors. There is no need to reproduce problem scenarios multiple times, reducing the number of times the WLR test machine hardware is used and extending the hardware service life. The operating efficiency of automated testing is also significantly improved compared to manual testing, and a large number of test cases can be executed quickly, freeing up human resources and improving the overall work efficiency of the team.
[0028] See also Figure 1 The flowchart of an automated monitoring method for a testing machine shown in FIG. 1 mainly includes the following steps S102 to S106:
[0029] Step S102, obtaining initial test data, a test plan and an execution script, wherein the initial test data is the test result obtained by performing a stability test on the system to be tested in a no-load state, and the test plan includes: multiple test cases, each test case includes: a test number, a test condition, a monitoring indicator, an execution step, an expected result and a test status. In one embodiment, before monitoring begins, it is necessary to obtain various initial data of the software and hardware of the system to be tested, so that the system can be tested for stability in an idle state without any load, and the test result is used as the initial test data. In addition, the test plan and execution script can be obtained from an open source version control system (Subversion, SVN).
[0030] Step S104, using the test plan and the execution script, determine at least one target test environment and target monitoring content corresponding to each test case. In one embodiment, the key fields in the test plan can be automatically obtained by executing the script, so as to select the monitoring content and deploy the stress test environment (i.e., the target test environment), wherein the monitoring content includes: hardware monitoring content and software monitoring content. The software monitoring content includes robustness, fluency of use, response time, coupling, security, memory usage, compatibility, ease of use and network latency. The hardware monitoring content includes system CPU usage, system memory usage size, process CPU usage, process dedicated set memory size, number of threads, CPU temperature, graphics card temperature, motherboard temperature, hard disk temperature, temperature of the test machine hardware module (SMU), non-page buffer pool size, disk utilization and response time of each stage.
[0031] Step S106, according to the target test environment and the initial test data, the system to be tested is subjected to task testing and exception monitoring, the target monitoring results are determined, and a monitoring report is generated according to the target monitoring results. In one embodiment, the exception monitoring is used to monitor the exceptions that occur during the task test. If an exception is detected, the test report needs to be sent to the executor in real time, and the executor will evaluate the severity of the problem and whether to continue the test. In addition, if the test plan is not executed after the test, it is necessary to enable the retest function to continue to deploy a new stress test environment to continue the test until the test plan is executed, and the final target monitoring results are sent to the executor in the form of an html report.
[0032] The automated monitoring method, device and server for the test machine provided by the embodiments of the present invention can efficiently evaluate software quality, discover test bottlenecks, reduce waste of human resources, and significantly improve system test accuracy and test efficiency.
[0033] See also Figure 2 The specific flow chart of the automated monitoring method of a test machine is shown in the figure. The embodiment of the present invention also provides an implementation method in which a WLR test machine can automatically perform test tasks and efficiently and quickly evaluate software quality. For details, see (1) to (3) below:
[0034] (1) Obtain the initial software and hardware data of the system to be tested, initialize the initial test system according to the test requirements, perform stability test on the system environment of the initial test system, and determine the initial test environment and initial test data that have reached a stable state. For details, see (a) to (c) below:
[0035] (a) Before monitoring begins, it is necessary to obtain various initial data of the software and hardware of the system to be tested, so that the system can be tested for stability without any load and in an idle state. The test results are used as the initial test data, which is convenient for comparison with the data of subsequent variable test scenarios to evaluate whether the system can still maintain the expected functions under these conditions, as well as the stability and reliability of the system under these conditions. In addition, automated testing can obtain and record detailed information data of each process in the task manager at that time. The monitoring function of the WLR test machine system needs to configure parameters according to the actual test and usage, so that the system after the test can meet the test purpose. Among them, the parameter configuration only needs to be modified once before the test, and no modification is required for subsequent tests.
[0036] (b) To ensure that the initial test environment is clean and to reduce interference factors, the system, browser, and other caches, logs, and temporary data must be cleaned before testing, and the database and firewall status must be modified to ensure that the load can be gradually increased and variable control can be performed during the automated test.
[0037] (c) In order to verify the reliability and stability based on the software functional test, it is necessary to preheat the software. If there is no abnormality after the software runs for a period of time, it can be determined that the system environment is stable and the test environment is determined as the initial test environment.
[0038] (2) After extracting key fields of the test conditions in each test case by using the execution script and deploying at least one target test environment under different test pressures, the monitoring indicators are obtained, the target monitoring content of the software and hardware under the test case is automatically updated, and the initial test system is automatically deployed for each test case according to the test number, and the target test environment corresponding to each test case is determined, thereby completing the deployment of the test scenario. In one embodiment, each test case includes: test number, test condition, monitoring indicator, execution step, expected result and test status. After the key fields in the test case are automatically obtained by executing the monitoring script, the target monitoring content of the software and hardware under the test case can be automatically updated according to the key fields in the test condition. The test environment is deployed in the field, and one or more pressures are gradually increased in each test scenario to destroy the original stable environment. At the same time, the monitoring indicators in the test cases are captured for abnormal monitoring and processing. Among them, when performing abnormal monitoring and processing, the monitoring contents include: system usage fluency, response time, security, memory usage, compatibility, usability, network latency, system CPU usage, system memory usage size, process CPU usage, process dedicated set memory size, number of threads, CPU temperature, graphics card temperature, motherboard temperature, hard disk temperature, test machine hardware module (SMU) temperature, non-page buffer pool size, disk utilization and response time of each stage, etc.
[0039] (3) Performing task testing on the system to be tested according to the execution steps in the target test environment, determining the task test data, and performing abnormal comparison and analysis on the task test data and the initial test data through a preset evaluation model, to evaluate whether the system can still maintain the expected functions under different conditions, as well as the stability and reliability of the system under these conditions, and determining the first monitoring result, wherein the first monitoring result is used to evaluate the task processing capability of the system to be tested under the target test environment.
[0040] (4) For abnormal errors reported by the system under test during the task test, the step of performing abnormal monitoring and processing and determining a second monitoring result specifically includes the following (A) to (B):
[0041] (A) When an abnormal error is detected during the task test, an abnormal screenshot corresponding to the test interface of the system to be tested is obtained, and the corresponding first monitoring result is obtained to form a monitoring report, and the abnormal screenshot and the monitoring report are sent to the user end in real time to obtain the abnormal severity level assessment result fed back by the user end for the abnormal error. In one embodiment, the test cases can be executed in sequence according to the numbering sequence of the test numbers contained in each test case. If a monitoring abnormality occurs during the test process, including: software interface freezes, response time is too long, pop-up window errors, dump file generation, occasional inability to open or close, process CPU usage is too high, memory usage is too high, hardware temperature is too high, etc., it is necessary to take a screenshot of the test interface, generate a temporary monitoring data file in json format, store it in a preset path, and finally convert it into an html test report, which is sent to the customer in real time by email through attachments and hyperlinks.
[0042] (B) When the abnormal severity level assessment result is higher than the preset level threshold, a second monitoring result is determined by retesting the test scenario where the abnormal error occurs. In one embodiment, see Figure 3 A specific flow chart of abnormal monitoring and processing is shown. After obtaining the number of retests fed back by the executing user end, the test scenario where the abnormal error occurs is retested multiple times according to the number of retests to determine the results of each retest, and then the abnormal error analysis result is determined by comparing the results of each retest, and the abnormal error analysis result and the abnormal screenshot are determined as the second monitoring result. In other words, if the abnormal problem is more serious and the root cause cannot be fully understood by only one test data, it is necessary to perform repeated observations and multiple operations through retesting. During retesting, the test of this scenario is executed multiple times according to the preset number of tests, and the test data of each time is recorded in a folder for easy comparison and observation until the executor solves the problem, otherwise the retest process continues.
[0043] (5) The first monitoring result and the second monitoring result are determined as the target monitoring result, and the target monitoring result is stored as a temporary monitoring data file in a preset path. In one embodiment, the target monitoring result is formatted and processed by a preset data conversion model to determine a monitoring report. After the format conversion is completed, the temporary monitoring data file corresponding to the target monitoring result in the preset path is deleted, and the test environment of the WLR test machine system is restored. That is to say, after the test is completed, due to the long test cycle and the large amount of monitoring content, the data records in text form cannot enable the executor to quickly observe the subtle impact fluctuations, and it is also particularly time-consuming to find the key points of abnormalities from a large amount of data. Therefore, in order for users to understand the test process more intuitively, a large amount of temporary data in json format is converted into a customized html report, and all temporary monitoring data files are deleted after the conversion. This monitoring report can include a pie chart of the overall execution results of the use case, a green circle for the test pass, and a bar chart of the serious status. The execution process of each test case is displayed in a process log, and the hardware monitoring indicators are displayed through the smoothness of the line chart to show the stability of the hardware system operation. Therefore, the monitoring report in the form of a web page has strong compatibility and is easy for users to read and understand.
[0044] Furthermore, the data interdependence between test cases during the testing process requires that the test cases be executed in a certain order. If there is data sharing in the dependency relationship, the operation of one test case directly affects the test results of other test cases, which may lead to unstable test results and difficult-to-debug problems, increase the coupling between test cases, and make management and maintenance difficult.
[0045] Therefore, the present invention needs to execute the test cases in sequence according to the numbering sequence of the test numbers contained in each test case. If a monitoring abnormality occurs during the test process, and in order to make each link of the automated testing process easier to operate and maintain, a teardown function is added when performing test case management. After each execution of a test case is completed, the configuration file and the test case are cleaned up by the Teardown function. After the data cleaning of this test case is completed, the data cleaning of the next test case is performed, so that the execution of this test case does not affect other tests, and after the next test case starts, the simulation data required for the test is automatically created through the Setup function to ensure the independence of each use case, thereby reducing the coupling between the use cases and making the automated testing process more flexible and scalable.
[0046] In summary, the present invention is based on the script monitoring method of the WLR test machine, which is fully automated and does not require human intervention after the script deployment is completed. The script monitoring data can be more accurate and the test efficiency can be higher. The execution user end is notified in real time by email, and the stress test environment is dynamically deployed, which can make dynamic monitoring more flexible. In addition, the teardown and setup functions are used to clean up and create simulation data to realize dynamic test case management, which can make each test scenario independent of each other and test multiple scenarios.
[0047] For the automated monitoring method of the test machine provided in the above embodiment, the embodiment of the present invention provides an automated monitoring device for the test machine, see Figure 4 The structure diagram of an automatic monitoring device of a testing machine shown in FIG. 1 includes the following parts:
[0048] The data acquisition module 402 acquires initial test data, a test plan, and an execution script, wherein the initial test data is the test result obtained by performing a stability test on the system under test in a no-load state, and the test plan includes: a plurality of test cases, each of which includes: a test number, a test condition, a monitoring indicator, an execution step, an expected result, and a test status;
[0049] The environment construction module 404 determines at least one target test environment and target monitoring content corresponding to each test case by using the test plan and the execution script;
[0050] The automatic monitoring module 406 performs task testing and abnormal monitoring on the system under test according to the target testing environment and the initial testing data, determines the target monitoring result, and generates a monitoring report according to the target monitoring result.
[0051] The automated monitoring device for the test machine provided in the embodiment of the present application can significantly improve the system test accuracy and test efficiency.
[0052] In one embodiment, when performing the step of acquiring initial test data, the data acquisition module 402 is also used to: acquire the initial software and hardware data of the system to be tested, initialize the initial test system according to the test requirements, perform stability testing on the system environment of the initial test system, and determine the initial test environment and initial test data that have reached a stable state.
[0053] In one embodiment, when using the test plan and execution script to determine at least one target test environment and target monitoring content corresponding to each test case, the above-mentioned environment construction module 404 is also used to: use the execution script to extract the key fields of the test conditions in each test case, and deploy at least one target test environment for the test case under different test pressures; obtain monitoring indicators, automatically update the target monitoring content of the software and hardware under the test case, and automatically deploy the initial test system for each test case according to the test number.
[0054] In one embodiment, when performing task testing and abnormality monitoring processing on the system to be tested according to the target test environment and initial test data to determine the target monitoring result, the above-mentioned automatic monitoring module 406 is also used to: perform task testing processing on the system to be tested according to the execution steps according to the target test environment to determine the task test data; perform abnormality comparative analysis on the task test data and the initial test data through a preset evaluation model to determine a first monitoring result, wherein the first monitoring result is used to evaluate the task processing capability of the system to be tested under the target test environment; perform abnormality monitoring processing on abnormal errors generated by the system to be tested during the task test to determine a second monitoring result; determine the first monitoring result and the second monitoring result as the target monitoring result, and store the target monitoring result as a temporary monitoring data file to a preset path.
[0055] In one embodiment, when performing abnormal monitoring and processing for abnormal errors occurring in the system to be tested during the task test and determining the second monitoring result, the above-mentioned automatic monitoring module 406 is also used to: when an abnormal error is detected during the task test, obtain an abnormal screenshot corresponding to the test interface of the system to be tested, and obtain the corresponding first monitoring result to form a monitoring report; send the abnormal screenshot and the monitoring report to the user end in real time to obtain the abnormal severity level assessment result fed back by the user end for the abnormal error; when the abnormal severity level assessment result is higher than the preset level threshold, determine the second monitoring result by retesting the test scenario where the abnormal error occurs.
[0056] In one embodiment, when performing the step of determining the second monitoring result by retesting the test scenario where an abnormal error occurs, the automatic monitoring module 406 is also used to: obtain the retest requirement fed back by the execution user end, and retest the test scenario where the abnormal error occurs multiple times according to a preset number of retests to determine the results of each retest; determine the abnormal error analysis result by comparing the results of each retest, and determine the abnormal error analysis result and the abnormal screenshot as the second monitoring result.
[0057] In one embodiment, when performing the step of generating a monitoring report based on the target monitoring results, the above-mentioned automatic monitoring module 406 is also used to: perform format conversion processing on the target monitoring results through a preset data conversion model, determine the monitoring report, and after the format conversion is completed, delete the temporary monitoring data file corresponding to the target monitoring results in the preset path.
[0058] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.
[0059] An embodiment of the present invention provides a server. Specifically, the server includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned embodiments.
[0060] Figure 5 A structural diagram of a server provided in an embodiment of the present invention, the server 100 includes: a processor 50, a memory 51, a bus 52 and a communication interface 53, wherein the processor 50, the communication interface 53 and the memory 51 are connected via the bus 52; the processor 50 is used to execute an executable module stored in the memory 51, such as a computer program.
[0061] The memory 51 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.
[0062] The bus 52 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0063] Among them, the memory 51 is used to store programs, and the processor 50 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 50 or implemented by the processor 50.
[0064] The processor 50 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 50. The above processor 50 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 51, and the processor 50 reads the information in the memory 51 and completes the steps of the above method in combination with its hardware.
[0065] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.
[0066] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0067] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. An automated monitoring method for a testing machine, characterized in that: The method comprises: Acquire initial test data, a test plan and an execution script, wherein the initial test data is a test result obtained by performing a stability test on the system to be tested under a no-load state, and the test plan includes a plurality of test cases; Determine at least one target test environment and target monitoring content corresponding to each test case by using the test plan and the execution script; According to the target test environment and the target monitoring content, task test processing and abnormal monitoring processing are performed on the system to be tested, target monitoring results are determined, and a monitoring report is generated according to the target monitoring results.
2. The automated monitoring method for a testing machine according to claim 1, characterized in that: The step of obtaining initial test data includes: The initial software and hardware data of the system to be tested are obtained, the initial test system is initialized according to the test requirements, a stability test is performed on the system environment of the initial test system, and the initial test environment and the initial test data that have reached a stable state are determined.
3. The automated monitoring method for a testing machine according to claim 1, characterized in that: Each test case includes: a test number, a test condition, and a monitoring indicator; the step of using the test plan and the execution script to determine at least one target test environment and target monitoring content corresponding to each test case includes: Extracting key fields of the test conditions in each of the test cases using the execution script, and deploying at least one target test environment for the test case under different test pressures; Obtain the monitoring indicators and automatically update the target monitoring content of the software and hardware under the test case; An initial test system is automatically deployed for each test case according to the test number.
4. The automated monitoring method for a testing machine according to claim 1, characterized in that: Each test case includes: execution steps and test status; the step of performing task test processing and abnormal monitoring processing on the system to be tested according to the target test environment and the target monitoring content, and determining the target monitoring result includes: Performing task test processing on the system to be tested according to the target test environment and the execution steps to determine task test data; By using a preset evaluation model, performing an abnormal comparison analysis on the task test data and the initial test data, and determining a first monitoring result, wherein the first monitoring result is used to evaluate the task processing capability of the system under test under the target test environment; Performing abnormal monitoring and processing on abnormal errors reported by the system under test during the task test to determine a second monitoring result; The first monitoring result and the second monitoring result are determined as the target monitoring result, and the target monitoring result is stored as a temporary monitoring data file in a preset path.
5. The automated monitoring method for a testing machine according to claim 4, characterized in that: The step of performing abnormal monitoring processing on the abnormal error reported by the system under test during the task test and determining the second monitoring result includes: When an abnormal error is detected during the task test, an abnormal screenshot corresponding to the test interface of the system to be tested is obtained, and a corresponding first monitoring result is obtained to form a monitoring report; The abnormal screenshot and the monitoring report are sent to the user terminal in real time to obtain the abnormal severity level assessment result fed back by the user terminal for the abnormal error report; When the abnormal severity level assessment result is higher than a preset level threshold, the second monitoring result is determined by retesting the test scenario where the abnormal error occurs.
6. The automated monitoring method for a testing machine according to claim 5, characterized in that: The step of determining the second monitoring result by retesting the test scenario where an abnormal error occurs includes: Obtain the retest requirements fed back by the execution user, and retest the test scenarios with abnormal errors multiple times according to the preset number of retests to determine the results of each retest; By comparing the retest results, the abnormal error analysis result is determined, and the abnormal error analysis result and the abnormal screenshot are determined as the second monitoring result.
7. The automated monitoring method for a testing machine according to claim 1, characterized in that: The step of generating a monitoring report according to the target monitoring result includes: The target monitoring result is format converted by a preset data conversion model to determine a monitoring report, and after the format conversion is completed, the temporary monitoring data file corresponding to the target monitoring result in the preset path is deleted.
8. An automatic monitoring device for a testing machine, characterized in that: The device comprises: A data acquisition module, which acquires initial test data, a test plan and an execution script, wherein the initial test data is a test result obtained by performing a stability test on the system to be tested under a no-load state, and the test plan includes a plurality of test cases; An environment deployment module, using the test plan and the execution script, determines at least one target test environment and target monitoring content corresponding to each of the test cases; The automatic monitoring module performs task testing and abnormal monitoring on the system to be tested according to the target testing environment and the target monitoring content, determines the target monitoring result, and generates a monitoring report according to the target monitoring result.
9. A server, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.