Test result determination method and test result determination device
By generating a predetermined model through cluster analysis of historical test logs, the inefficiency caused by unclear test results in software testing is solved, achieving the effect of rapid judgment and improved testing efficiency.
Patent Information
- Application Number
- CN202111050746.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-08
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In current software testing, when test results are unclear, testers need to spend a lot of manpower to investigate the cause, resulting in low testing efficiency.
By performing cluster analysis on historical test logs to generate a predetermined model, the model is used to determine whether real-time test logs are abnormal or normal, and corresponding prompts are generated to help testers quickly filter out unclear test results that require attention.
It improved the judgment of testers, reduced manpower consumption, and increased testing efficiency and software development speed.
Smart Images

Figure CN113778874B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of software testing, and in particular to a test result determination method and a test result determination device. Background Art
[0002] There are various software systems on the market, and new software systems are emerging in different fields. From the completion of development to the online operation, a new software system must pass a strict testing process. Only software systems that pass the test can be put online. If the test fails, the software system needs to be redeveloped.
[0003] Taking the software system of a commercial bank as an example, the testing of the software system usually includes interface testing, interface testing, etc. When performing interface testing, the tester needs to open the interface of the software system being tested, enter the test parameters and submit them, and then determine whether the test results returned by the interface are normal. When performing interface testing, the tester needs to send the agreed test parameters to the specified interface to determine whether the test results returned by the interface are normal. In the above testing process, the tester performs the test from the user's perspective, so the tester cannot clearly see the response process of the background program of the software system when testing the test parameters, and can only determine whether the background program executing the test parameters in the current software system is running normally based on the test results.
[0004] In the actual testing process, after the test parameters are input for testing, the tested software system will always return some unclear test results. For example, when the tested software system fails to query transaction data, it returns the test result of "no data"; when the tested software system does not query transaction data, it also returns the test result of "no data". When the tester encounters the test result of "no data", the tester is not clear whether the background program of the software system corresponding to the test result fails to query transaction data or does not query transaction data. Then, the test result of "no data" is an unclear test result. The tester needs to record the unclear test result and then communicate with the developer about the specific reason for the unclear test result. If necessary, the developer needs to conduct in-depth tracking of the background program corresponding to the test parameter in the tested software system and conduct in-depth analysis of the test environment and test parameters to draw a conclusion and determine whether the unclear test result is normal. At present, with the increasing popularity of automated testing, if the software system being tested returns a large number of unclear test results during testing, and each unclear test result requires the corresponding developers and testers to determine whether the unclear test result is normal according to the above process, such a testing process will not only consume too much manpower cost, but also lead to low testing efficiency. Therefore, there is an urgent need for a test result determination method that can assist testers in quickly determining whether the unclear test result is normal when the test result is unclear, avoid consuming a lot of manpower to troubleshoot the unclear test result, and improve testing efficiency. Summary of the invention
[0005] The embodiments of the present application provide a test result determination method and a test result determination device, which relate to the field of software testing and can solve the problem of low test efficiency in existing software testing.
[0006] To achieve the above objectives, the embodiments of the present application provide the following technical solutions:
[0007] In a first aspect, a test result discrimination method is provided. The test result discrimination method includes the following steps: first, when the software system under test receives the real-time test parameters for testing, the test result output by the software system is obtained; then, when it is determined that the test result is an ambiguous result, the real-time test log generated by the software system for testing the real-time test parameters is obtained; finally, the real-time test log is input into a predetermined model, and when the real-time test log is determined to be abnormal or normal according to the judgment result output by the predetermined model, a prompt message is generated to indicate that the test result is abnormal when the real-time test log is abnormal, or a prompt message is generated to indicate that the test result is normal when the real-time test log is normal; wherein the predetermined model includes: a model generated by clustering analysis of the historical test logs of the software system with abnormal and normal as cluster centers, and the historical test log is a test log generated after the software system tests the received historical test parameters. In the above test result discrimination method, first, after inputting the real-time test parameters into the tested software system, the test result output by the software system is obtained, and then when it is determined that the test result is an unclear test result, the real-time test log generated by the software system for testing the real-time test parameters is obtained, and the real-time test log is input into the predetermined model, and the real-time test log is determined to be normal or abnormal according to the judgment result output by the predetermined model, and a prompt message indicating that the test result is normal or abnormal is generated. In addition, since the predetermined model is generated by clustering the historical test logs with normal and abnormal as the cluster center, and the historical test logs in the software system record the operation of the software system, and store a large number of normal or abnormal historical test logs, the normal historical test logs in the software system are mostly very similar, and the abnormal historical test logs are very different from the normal historical test logs, so when clustering, it can be ensured that the normal ones in the historical test logs are divided into one cluster and the abnormal ones are divided into one cluster, and the cluster model generated by clustering the historical test logs with normal and abnormal as the cluster center is obtained, that is, the predetermined model. After the real-time test log is input into the predetermined model, it can be known through calculation whether the real-time test log is close to a normal cluster in the historical test log or an abnormal cluster in the historical test log. The judgment result output by the predetermined model determines whether the real-time test log is normal or abnormal, thereby generating a prompt message indicating that the test result is normal when the real-time test log is normal, and generating a prompt message indicating that the test result is abnormal when the real-time test log is abnormal.Such steps can assist testers to quickly screen out which unclear test results need to be paid attention to (that is, which unclear test results can be regarded as abnormal test results) and which unclear test results do not need to be paid attention to (that is, which unclear test results can be regarded as normal test results) when a large number of unclear test results appear, thereby improving the testers' ability to judge the test results. At the same time, when the testers feed back the abnormal test results to the developers, the developers can focus on those abnormal test results and avoid wasting time on some normal test results. In addition, such a test result determination method also enables the testers' test process to more accurately reflect the problems existing in the current software system, and the developers can also make targeted modifications to the problems existing in the current software system, thereby improving the efficiency of software testing and the speed of software development. Moreover, the predetermined model can be reused many times after being generated, and the testers can regenerate the predetermined model according to a certain period, which also makes the entire test result determination method simple and has high utilization rate and good reusability.
[0008] Optionally, before the real-time test log is input into the predetermined model, it includes: reading the historical test log generated by the software system in the database, and performing cluster analysis on the historical test log with abnormal and normal as cluster centers to generate the predetermined model; or, after the software system receives the historical test parameters for testing, generating the historical test log, performing cluster analysis on the historical test log with abnormal and normal as cluster centers to generate the predetermined model. The above optional scheme provides two methods for obtaining the historical test log. If the software system is a software system that has been put into operation, the historical operation log generated by the software system can be directly read in the database. Since the tester simulates the user to test the software system, the historical operation log is equivalent to the historical test log, which proves that the software system that has been put into operation can read the historical test log generated by the software system in the database; if the software system is a newly added software system that has not been put into operation, the test parameters input by the tester before can be obtained, and the software system tests the test parameters and generates a test log. The test log is the historical test log, and then the historical test log can be obtained through the log acquisition tool.
[0009] Optionally, before clustering the historical test logs with abnormal and normal as cluster centers to generate a predetermined model, it includes: preprocessing the acquired historical test logs to generate a first log text, and the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning. Feature extraction is performed on the first log text after word segmentation to generate a first log feature, and the feature extraction includes extracting one or more of the following log features: word frequency and inverse word frequency. Clustering the historical test logs with abnormal and normal as cluster centers to generate a predetermined model includes: clustering the first log feature with abnormal and normal as cluster centers to generate a predetermined model. In this optional solution, the historical test logs are first preprocessed and then feature extracted, which can ensure that the acquired first log feature can more accurately reflect the characteristics of the historical test logs, thereby making the cluster analysis of the first log feature with abnormal and normal as cluster centers to generate a predetermined model more accurate and representative.
[0010] Optionally, the real-time test logs can be obtained through the data collection tool logstash when the software system tests the real-time test parameters.
[0011] Optionally, after obtaining the real-time test log generated by the software system testing the real-time test parameters, it also includes: preprocessing the real-time test log to generate a second log text, and the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning. After word segmentation, feature extraction is performed on the second log text to generate a second log feature. The feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency. Inputting the real-time test log into a predetermined model includes: inputting the second log feature into a predetermined model. In this optional scheme, the real-time test log can be preprocessed, and then feature extraction is performed, and the extracted second log feature is input into the predetermined model, so that the judgment result output by the predetermined model is more accurate.
[0012] Optionally, the real-time test parameters include test parameters of an interface test, or the real-time test parameters include test parameters of an interface test.
[0013] In a second aspect, a test result discrimination device is provided, comprising: a test result judgment module, a real-time log collection module, an analysis and prediction module, and a result display module. The test result judgment module is used to obtain the test result output by the software system after the tested software system receives the real-time test parameter for testing, and to determine whether the test result is clear; the real-time log collection module is used to obtain the real-time test log generated by the software system for testing the real-time test parameter when the test result judgment module determines that the test result is an unclear result; the analysis and prediction module is used to input the real-time test log obtained by the real-time log collection module into a predetermined model, and determine whether the real-time test log is abnormal or normal according to the judgment result output by the predetermined model; wherein the predetermined model includes: a model generated by clustering analysis of the historical test logs of the software system with abnormality and normality as cluster centers, wherein the historical test log is a test log generated by the software system after testing the received historical test parameters; the result display module is used to generate a prompt message when the analysis and prediction module determines that the real-time test log is abnormal or normal according to the judgment result output by the predetermined model, and the prompt message is used to prompt that the test result is abnormal when the real-time test log is abnormal, or the prompt message is used to prompt that the test result is normal when the real-time test log is normal.
[0014] Optionally, the test result identification device also includes a historical log collection module and a cluster training module. The historical log collection module is used to read the historical test logs generated by the software system in the database; the cluster training module is used to perform cluster analysis on the historical test logs obtained by the historical log collection module with abnormal and normal as cluster centers to generate a predetermined model. Alternatively, the historical log collection module is used to obtain the historical test logs generated after the software system receives the historical test parameters for testing; the cluster training module is used to perform cluster analysis on the historical test logs obtained by the historical log collection module with abnormal and normal as cluster centers to generate a predetermined model.
[0015] Optionally, the test result determination device also includes a first log preprocessing module. The first log preprocessing module is used to preprocess the historical test logs acquired by the historical log collection module to generate a first log text, and the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning; feature extraction is performed on the first log text after word segmentation to generate a first log feature, and the feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency; a clustering training module is specifically used to perform cluster analysis on the first log feature generated by the first log preprocessing module with abnormal and normal as cluster centers to generate a predetermined model.
[0016] Optionally, the real-time log collection module is specifically used to obtain real-time test logs through the data collection tool logstash when the software system tests the real-time test parameters.
[0017] Optionally, the test result identification device also includes a second log preprocessing module. The second log preprocessing module is used to preprocess the real-time test log obtained by the real-time log collection module to generate a second log text. The preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning. After word segmentation, feature extraction is performed on the second log text to generate a second log feature. The feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency. The analysis and prediction module is specifically used to input the second log feature generated by the second log preprocessing module into a predetermined model.
[0018] Optionally, the real-time test parameters include test parameters of an interface test, or the real-time test parameters include test parameters of an interface test.
[0019] In a third aspect, a test result determination device is provided, comprising a memory and a processor. The memory is used to store computer-executable instructions, and the processor is connected to the memory via a bus. When the test result determination device is running, the processor executes the computer-executable instructions stored in the memory, so that the test result determination device performs the test result determination method as described in any one of the first aspects above.
[0020] In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising computer execution instructions, and when the computer execution instructions are executed on a computer, the computer executes the test result determination method as described in any one of the above-mentioned first aspects.
[0021] Among them, the technical effects brought about by any possible implementation method of the second to fourth aspects can refer to the technical effects brought about by the implementation method of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 A schematic diagram of the hardware structure of a test result determination device provided in an embodiment of the present application;
[0023] Figure 2 A schematic diagram of the hardware structure of a test result determination device provided in another embodiment of the present application;
[0024] Figure 3 A flowchart of a test result determination method provided for another embodiment of the present application;
[0025] Figure 4 A specific flow chart of a test result determination method provided for another embodiment of the present application;
[0026] Figure 5 A schematic diagram of the structure of a test result determination device provided in yet another embodiment of the present application;
[0027] Figure 6 A schematic diagram of the specific structure of a test result determination device provided in yet another embodiment of the present application. DETAILED DESCRIPTION
[0028] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.
[0029] Unless otherwise defined, all scientific and technological terms used herein have the same meanings as those known to those of ordinary skill in the art. In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. In addition, in the embodiments of the present application, the words "first", "second", etc. do not limit the quantity and order.
[0030] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0031] The technical terms in the embodiments of the present application are described as follows:
[0032] Log: A software system generates event records during operation, which are called logs. Each line of the log records the date, time, user, action, and other related operations. The log provided in the embodiments of the present application mainly refers to the log generated by the software system when testing test data (i.e., when the operation process is testing). In particular, in the embodiments of the present application, the test log (including real-time test logs or historical test logs) can be obtained or collected through the data collection tool logstash.
[0033] Logstash: is a lightweight log collection and processing tool that can collect scattered and diverse logs, perform custom processing, and then transfer them to a specified location, such as a server or file. In an embodiment of the present application, the test log (including real-time test logs or historical test logs) is acquired or collected by the data acquisition tool logstash, and the acquired test log (including real-time test logs or historical test logs) can also be pre-processed, and then the pre-processed test log (including real-time test logs or historical test logs) is saved to the corresponding server or file, and the embodiment of the present application does not limit the form of the server or file.
[0034] Cluster analysis: Cluster analysis, also known as group analysis, is a multivariate statistical analysis method that classifies sampled data based on the principle of "birds of a feather flock together". The object of cluster analysis is a large amount of sampled data, and it is required to be reasonably classified according to the characteristics of the sampled data. The cluster analysis process is a process of classifying sampled data into different classes (clusters), and the objects in the same cluster have great similarities, while the objects in different clusters have great differences. The goal of cluster analysis is to classify the sampled data on the basis of similarity and obtain a clustering model.
[0035] K-means clustering algorithm: K-means clustering algorithm is an iterative clustering analysis algorithm, and its steps are to randomly select K objects as the initial cluster centers (that is, K seed cluster centers), and then calculate the distance between each object and the K seed cluster centers, and assign each object to the seed cluster center closest to it. In addition, the above steps will be repeated until a certain termination condition is met, and the termination condition can be any of the following: no (or the minimum number) objects are reassigned to different seed cluster centers; no (or the minimum number) seed cluster centers change again; the error sum of squares is locally minimized. After the termination condition is met, the clustering model is obtained, and the K seed cluster centers that meet the termination condition are the final cluster centers. When applied to the embodiments of the present application, cluster analysis is mainly used to cluster historical test logs, and the normal or abnormal historical test logs are used as two cluster centers for cluster analysis.
[0036] There are various software systems on the market, and new software systems are emerging in different fields. From the completion of development to the online operation, a new software system must pass a strict testing process. Only software systems that pass the test can be put online. If the test fails, the software system needs to be redeveloped.
[0037] Taking the software system of a commercial bank as an example, the testing of the software system usually includes interface testing, interface testing, etc. When conducting interface testing, the tester needs to open the interface of the software system being tested, enter the test parameters and submit them, and then determine whether the test results returned by the interface are normal. When conducting interface testing, the tester needs to send the agreed test parameters to the specified interface to determine whether the test results returned by the interface are normal. It cannot be ignored that in the design process of the software system, in order to avoid unfriendly prompts that confuse users, and the software system itself will hide technical details, when users use the software system to perform various operations, the operation results returned to the user by the software system interface are often modified, and the operation results will not display the execution process and error information of the background program of the software system. For example, when a user queries transaction data, if the background program of the software system fails to query the transaction data, the user will be returned an operation result of "no data"; when the number of users is too large and the background server is blocked, the user will be returned an operation result of "please try again later". There are many such modified operation results. Although such modified operation results can bring a better experience to users, it is difficult for testers to understand the running status of the background program of the software system through the modified operation results. Because testers test the software system from the user's perspective, the test parameters entered by the testers are equivalent to the operations performed by the users, and the test results generated by the software system are equivalent to the operation results obtained by the users. Therefore, when testers enter test parameters into the software system for testing, they will get the unclear test results as described above. Testers cannot judge whether the unclear test results are normal or abnormal. Testers will record the unclear test results and communicate with developers about the reasons for the unclear test results. If necessary, developers are required to conduct in-depth tracking of the background programs corresponding to the test parameters in the tested software system and conduct in-depth analysis of the test environment and test parameters to draw conclusions and determine whether the unclear test results are normal. At present, with the increasing popularity of automated testing, if the software system being tested returns a large number of unclear test results during testing, and each unclear test result appears, the corresponding developers and testers are required to determine whether the unclear test result is normal according to the above process. Such a testing process will not only consume too much manpower costs, but also lead to low testing efficiency.
[0038] To solve the above-mentioned problem, an embodiment of the present application provides a test result judgment method, which can judge whether the real-time test log is normal or abnormal based on a predetermined model generated by clustering analysis of historical test logs with abnormal and normal as cluster centers when unclear test results appear during the testing of real-time test parameters by the tested software system, and generate prompt information to indicate that the test result is abnormal when the real-time test log is abnormal, or generate prompt information to indicate that the test result is normal when the real-time test log is normal, thereby realizing rapid judgment on whether the test result is normal or abnormal, and then deciding how to handle the background program of the software system corresponding to the test result.
[0039] The above-mentioned test result determination method can be applied to a test result determination device, which can be a device, or a chip in a device, or a system on chip in the device.
[0040] Optionally, the device may be a physical machine, such as a desktop computer, also known as a desktop or desktop computer (desktopcomputer), a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA) and other terminal devices.
[0041] Optionally, the test result determination device may also implement the functions to be implemented by the test result determination device through a virtual machine (VM) deployed on a physical machine.
[0042] For ease of understanding, the structure of the test result determination device in the embodiment of the present application is described below.
[0043] Figure 1 FIG. 1 is a schematic diagram showing the hardware structure of the test result determination device provided in the embodiment of the present application. Figure 1 As shown, the test result determination device 10 includes a processor 101, a memory 102, a communication interface 103, and a bus 104. The processor 101, the memory 102, and the communication interface 103 may be connected via the bus 104.
[0044] The processor 101 may be a central processing unit (CPU), a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a programmable logic device (PLD), or any combination thereof. The processor may also be any other device having a processing function, such as a circuit, a device, or a software module. The processor 101 may also include multiple CPUs, and the processor 101 may be a single-CPU processor or a multi-CPU processor. The processor here may refer to one or more devices, circuits, or processing cores for processing data (such as computer program instructions).
[0045] The memory 102 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, and the embodiments of the present application do not impose any restrictions on this. The memory 102 can be independent or integrated with the processor 101. Among them, the memory 102 is used to store computer program code, and the computer program code includes computer execution instructions, etc. When one or more processors 101 execute instructions, the test result discrimination device 10 can execute the above-mentioned test result discrimination method.
[0046] The bus 104 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 1Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0047] It should be pointed out that Figure 1 The structure shown does not constitute a limitation on the test result determination device. Figure 1 In addition to the components shown, the test result determination device may include more or fewer components than shown, or a combination of certain components, or a different arrangement of components.
[0048] Figure 2 FIG. 1 shows a hardware structure diagram of a test result determination device provided in an embodiment of the present application. Figure 2 As shown, the test result determination device 20 may include a processor 201 and a communication interface 202. The processor 201 is coupled to the communication interface 202. The function of the processor 201 may refer to the functional description of the above processor 101. In addition, the processor 201 also has a storage function, and the functional description of the above memory 102 may refer to it.
[0049] The communication interface 202 is used to provide data to the processor 201. The communication interface 202 can be an internal interface of the test result determination device 20, or an external interface of the test result determination device 20 (equivalent to Figure 1 The communication interface 103 in FIG.
[0050] It should be pointed out that Figure 1 (or Figure 2 ) does not constitute a limitation on the test result determination device, except Figure 1 (or Figure 2 ) In addition to the components shown in the figure, the test result determination device may include more or fewer components than those shown in the figure, or a combination of certain components, or a different arrangement of components.
[0051] The test result determination method provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0052] Reference Figure 3 As shown, Figure 3 The flowchart of the test result determination method provided in the embodiment of the present application includes the following three steps:
[0053] S301. After the tested software system receives the real-time test parameters for testing, the test results output by the software system are obtained.
[0054] In step S301, the tester first needs to input real-time test parameters to the software system being tested, and the real-time test parameters can be test parameters corresponding to the interface test. Interface testing is also called user interface (UI) testing, or UI testing for short, and is a test performed on the display interface of the software system. The purpose of performing interface testing is to check whether the interface display of the software system is normal, whether the content displayed on the interface is appropriate, etc. Usually, interface testing includes testing whether the content displayed on the software system interface meets the requirements, testing whether the interface display of the software system is user-friendly, etc.
[0055] For example, when testing whether the content displayed on the software system interface meets the requirements, the tester will trigger the relevant jump button in the interface (that is, the system software receives the input real-time test parameters in response to the tester's triggering button operation) to check whether the software system jumps to the corresponding interface normally when the button is triggered, and whether the software system will suddenly have style confusion problems when jumping; or, the tester will enter the real-time test parameters in the input box in the display interface of the software system to determine whether the real-time test parameters displayed on the interface are clear and readable.
[0056] For example, when testing whether the interface display of a software system is user-friendly, the tester will stand in the user's perspective and enter the test parameters in the input box in the display interface of the software system, paying attention to whether the input box will match the spaces where the user mistakenly operates, whether there will be prompt messages when the input content is inappropriate, and whether the prompt messages are friendly when input errors occur.
[0057] In step S301, the real-time test parameter can also be a test parameter corresponding to the interface test. Most of the current software systems will set up multiple subsystems, and the subsystems will cooperate and interact with each other through the interface inside the software system to achieve the normal operation of the software system; at the same time, the software system will also interact with other software systems through the external interface of the software system to provide users with more convenient and faster services. The interface inside the software system includes the interactive interface between the functions of the software system. For example, when performing data calculation, the data calculation function will transmit the calculation result to the result display function through the interface between the data calculation function and the result display function to realize data calculation and calculation result display; the interface inside the software system also includes the interactive interface between modules and modules. For example, many software systems require that data release operations can only be performed in the software system after logging in, which means that the data release module must obtain the login confirmation information returned by the login module through the interface between the data release module and the login module when performing the data release operation, so as to perform the data release operation. The external interface of a software system includes the interface opened by one software system to another software system. For example, when searching, resource information on websites or servers other than the search website will be searched. However, this does not mean that the current search website can directly obtain the databases of other websites or servers. It is just because other websites or servers provide an external interface to the current search website. The current search website obtains resource information on other websites or servers through the external interface, thereby realizing interaction between websites or servers.
[0058] Whether it is the above-mentioned interface test or interface test, in fact, the tester needs to input the real-time test parameters into the software system being tested, wait for the software system being tested to test the real-time test parameters, and then the test result judgment device obtains the test results output by the software system for testing.
[0059] S302. When it is determined that the test result is an ambiguous result, obtain a real-time test log generated by the software system testing the real-time test parameters.
[0060] In step S301, the test result determination device obtains the test result output by the software system when testing the real-time test parameters. In step S302, the test result determination device needs to perform further determination on the test result.
[0061] Taking the registration function test of the tested software system as an example, the tester will simulate the user to make a registration request to the tested software system. First, the tester enters various real-time test parameters (such as mobile phone number and password, etc.) required for registration in the display interface of the tested software system. After the display interface of the tested software system receives the real-time test parameters input by the user, the real-time test parameters are transmitted to the background program through the interface between the display interface and the background program. The background program calls the verification module to verify whether the real-time test parameters input by the tester are qualified (such as whether the mobile phone number is accurate, whether the password setting complies with the rules, etc.). When the verification is passed, the background program will transmit the real-time test parameters to the database through the interface between the background program and the database, and the database will store the real-time test parameters. Moreover, when the verification is passed, the background program will also transmit the verification result to the display interface through the interface between the display interface and the background program, so that the tested software system outputs the test result of "registration successful" to complete the entire registration process. In the above registration request process, if the mobile phone number in the real-time test parameters entered by the tester is incorrect, the tested software system outputs the test result of "the mobile phone number is incorrect"; if the real-time test parameters entered by the tester complete the registration process, the tested software system outputs the test result of "registration successful". The above two results are clear test results. However, the tested software system does not only output clear test results. For example, when the interface between the background program and the database of the tested software system fails, it will output the test result of "please try again later"; when the current network status of the tester is unstable, the tested software system will also output the test result of "please try again later". This test result of "please try again later" is an unclear test result.
[0062] When the tester inputs real-time test parameters to determine whether the test result is an ambiguous test result, the tester can make a manual judgment, and then the test result judgment device records the ambiguous test result; or, the test result judgment device sets a click button in the software system being tested, and when the click button is clicked, it indicates confirmation that the test result is an ambiguous test result. Therefore, after the tester manually determines that the test result is an ambiguous test result, the test result judgment device can determine that the test result is an ambiguous result by clicking this button; or, the test result judgment device can directly set a judgment condition, and the judgment condition can judge a clear test result, then the corresponding other test results are ambiguous test results.
[0063] When the test result determination device determines that the test result is an ambiguous result, the test result determination device can obtain the real-time test log generated by the tested software system for testing the real-time test parameters through the data acquisition tool logstash. The test log (in the following example, it can be a historical test log or a real-time test log) can include but is not limited to the following information: log recording time, device identifier (English: identifier, ID), and log recording content.
[0064] S303. Input the real-time test log into a predetermined model, and generate a prompt message when determining whether the real-time test log is abnormal or normal according to the judgment result output by the predetermined model.
[0065] Specifically, the predetermined model can be a model generated by clustering analysis of the historical test logs of the tested software system with abnormal and normal as cluster centers. The historical test log is a test log generated after the tested software system tests the received historical test parameters, or the historical test log is a historical test log generated by reading the tested software system in the database. The historical test log can represent the operation of the current software system, and these historical test logs are clustered by the K-means clustering algorithm, and the K value is set to 2 (that is, 2 seed cluster centers are set for cluster analysis). Since the historical test logs in the software system store a large number of normal or abnormal historical test logs, the normal historical test logs in the software system are mostly very similar, and the abnormal historical test logs are very different from the normal historical test logs. Therefore, during the cluster analysis, it can be ensured that the normal ones in the historical test logs are divided into one cluster, and the abnormal ones are divided into one cluster, and a cluster model generated by clustering analysis of the historical test logs with normal and abnormal as cluster centers is obtained, that is, the predetermined model.
[0066] The real-time test log obtained in step S302 is input into the predetermined model, and the test result determination device calculates the distance a between the real-time test log and a cluster in the predetermined model representing normal historical test logs, and calculates the distance b between the real-time test log and a cluster in the predetermined model representing abnormal historical test logs, and determines whether distance a is smaller or distance b is smaller. If distance a is smaller, it means that the distance between the real-time test log at this time and the cluster in the predetermined model representing normal historical test logs is small, and the test result determination device can determine that the real-time test log is normal, and generate prompt information indicating that the test result is normal; if distance b is smaller, it means that the distance between the real-time test log at this time and the cluster in the predetermined model representing abnormal historical test logs is small, and the test result determination device can determine that the real-time test log is abnormal, and generate prompt information indicating that the test result is abnormal.
[0067] The test result discrimination method provided by the embodiment of the present application is as follows: first, after inputting the real-time test parameters into the tested software system, the test result output by the software system is obtained, and then when it is determined that the test result is an ambiguous test result, the real-time test log generated by the software system for testing the real-time test parameters is obtained, and the real-time test log is input into the predetermined model, and the real-time test log is determined to be normal or abnormal according to the judgment result output by the predetermined model, and a prompt message indicating that the test result is normal or abnormal is generated. In addition, since the predetermined model is generated by clustering the historical test logs with normal and abnormal as the cluster centers, and the historical test logs in the software system record the operation of the software system and store a large number of normal or abnormal historical test logs, the normal historical test logs in the software system are mostly very similar, and the abnormal historical test logs are very different from the normal historical test logs, so when clustering the historical test logs, it can be ensured that the normal ones are divided into one cluster and the abnormal ones are divided into another cluster, and the cluster model generated by clustering the historical test logs with normal and abnormal as the cluster centers is obtained, that is, the predetermined model. Then, after the real-time test log is input into the predetermined model, it can be known by calculation whether the real-time test log is close to a normal cluster in the historical test log or an abnormal cluster in the historical test log, and the judgment result output by the predetermined model determines whether the real-time test log is normal or abnormal, thereby generating a prompt message indicating that the test result is normal when the real-time test log is normal, and generating a prompt message indicating that the test result is abnormal when the real-time test log is abnormal. Such a step can assist the tester to quickly screen out which unclear test results need to be paid attention to (that is, which unclear test results can be regarded as abnormal test results) and which unclear test results do not need to be paid attention to (that is, which unclear test results can be regarded as normal test results) when a large number of unclear test results appear, thereby improving the tester's ability to judge the test results. At the same time, when the tester feeds back the abnormal test results to the developer, it also enables the developer to focus on those abnormal test results, avoiding wasting time on some normal test results. In addition, this test result determination method also enables the tester's test process to more accurately reflect the problems existing in the current software system, and the developer can also make targeted modifications to the problems existing in the current software system, which improves the efficiency of software testing and the speed of software development. Moreover, the predetermined model can be reused many times after it is generated, and the tester can regenerate the predetermined model according to a certain period, which also makes the entire test result determination method simple and has a high usage rate and good reusability.
[0068] Reference Figure 4 As shown, another embodiment of the present application provides a test result determination method, which specifically includes the following process:
[0069] S401. Obtain historical test logs of the software system.
[0070] The acquisition of historical test logs can be done by reading historical test logs generated by the software system in the database. When the software system is a software system that has been put into operation, various historical operation logs generated by user operations will exist in the software system that has been put into operation, and since the tester also tests the software system from the user's perspective during the test process, the various historical operation logs generated by user operations in the software system that has been put into operation can be considered as historical test logs, which will be retained in the database of the software system, and the test result determination device can directly obtain the historical test logs in the database.
[0071] Alternatively, after the software system receives the historical test parameters for testing, a historical test log is generated. Specifically, if the software system is a newly added software system and has not been put into operation, then the software system does not have a historical operation log generated by user operations. Then the test log generated by the previous test of the software system can be obtained and used as the historical test log. In other words, the historical test parameters are the test parameters entered by the tester for the previous test. After the test parameters entered by the tester enter the software system, the software system will perform a test and generate a test log. The test log is the historical test log, and then the test result determination device can obtain the historical test log through the log acquisition tool.
[0072] S402. Preprocess the acquired historical test log to generate a first log text, perform feature extraction on the first log text after word segmentation, and generate a first log feature.
[0073] Specifically, in step S402, the test result determination device preprocesses the historical test logs obtained in the above step S401, and the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning.
[0074] Among them, time slice alignment means aligning the time sequence of historical test logs. Considering that historical test logs are a kind of time series data, if the time sequence of the historical test logs obtained is disordered, the subsequent processing logic of the historical test logs will be affected. For example, in the case of multi-threaded concurrent operations, some historical test logs generated with later occurrence times will be ranked before other historical test logs. In this case, all historical test logs can be reordered according to the time in the historical test logs through time slice alignment; for example, when the server clocks where multiple historical test logs are located are not synchronized, the time of the obtained historical test log can be added or subtracted by the difference between the server clock where the current historical test log is located and the normal clock through time slice alignment, and then reordered according to the time in the historical test log.
[0075] Missing value processing refers to operations such as deleting or filling missing values in historical test logs. Missing value processing specifically includes first determining whether there are missing values in the current historical test log, and determining whether the missing values in the historical test log belong to missing completely at random (MCAR), missing at random (MAR), or missing not at random (MNAR). After determining the type of missing values, it is also necessary to comprehensively consider factors such as the reasons for missing data and the number of existing historical test logs to determine how to handle the missing values. For example, if the missing value is not an important value, the missing value may be deleted directly; for example, if the missing value has an obvious regularity with the values in other historical test logs except the missing value, then the missing value can be determined based on this regularity and data filling can be performed; for example, if the missing value represents a certain characteristic value in the historical test log, and there is an average value for the characteristic value in other historical test logs except the missing value, then this average value can also be filled in the missing value of the historical test log with the missing value.
[0076] Sensitive field cleaning means converting some sensitive private data into some meaningless characters or directly deleting sensitive private data. For example, in the historical test log, some fields representing user identities are directly deleted, or the fields representing user identities are converted into meaningless characters such as 123 that will not conflict with other content.
[0077] After the above preprocessing is completed, the first log text can be obtained.
[0078] Next, the first log text is segmented and then feature extracted to generate a first log feature. The feature extraction includes extracting one or more of the following log features: word frequency and inverse word frequency.
[0079] Among them, the word segmentation of a sentence in the first log text means extracting some nouns, verbs, etc. in the sentence, and then directly displaying these words after extraction, instead of displaying the complete sentence, and then calculating the word frequency or inverse word frequency features of all words, where the word frequency represents the frequency of occurrence of a certain word, and the inverse word frequency represents the importance of a word in the entire text, because the smaller the frequency of a word, the greater the inverse word frequency of this word, which proves that this word is the difference between the current document and other documents. These word frequency and inverse word frequency features are the first log features.
[0080] For example, in the sentence "Today is sunny, I am happily going on an outing today", after word segmentation, it means "today", "yes", "sunny", "I", "today", "happily", "go", "outing", and a total of 8 words appear, of which "today" appears twice, "yes", "sunny", "I", "happily", "go", and "outing" appear once each. The calculation rule of word frequency is: the word frequency of a word = the number of times the word appears in a certain category ÷ the number of all entries in the category. Then the word frequency of "today" is 0.25, and the word frequencies of "yes", "sunny", "I", "happily", "go", and "outing" are all 0.125. The calculation rule of inverse word frequency is: the inverse word frequency of a word = log (the total number of documents in the corpus ÷ (the number of documents containing the word + 1)). Among them, since we do not list the documents in which the sentence "Today is sunny, I am happily going on an outing today" appears and the total number of documents in the corpus where the current document is located, we do not calculate the inverse word frequency in detail, only for example.
[0081] S403. Perform cluster analysis on the first log feature with abnormality and normality as cluster centers to generate a predetermined model.
[0082] Specifically, the first log feature of step S402 is generated by preprocessing and feature extraction of the historical test log obtained in step S401. Since the historical test log is a historical test log generated after the tested software system tests the received historical test parameters, or the historical test log is a historical test log generated by the tested software system read in the database, a large number of normal or abnormal historical test logs are stored in the historical test log. These normal or abnormal historical test logs represent the operation status of the current software system. Therefore, the first log feature can also represent the operation status of the current software system. Some of the first log features represent that the historical test log is normal, and some represent that the historical test log is normal. Then, cluster analysis is performed on the first log feature using the K-means clustering algorithm, and the K value is set to 2 (representing setting 2 seed clustering centers for cluster analysis). Since most normal historical test logs in the software system have great similarities, and abnormal historical test logs are very different from normal historical test logs, the basic requirement of cluster analysis is that objects in a cluster have great similarities, while objects in different clusters have great differences. Therefore, the clustering model generated by cluster analysis of the first log feature is that one cluster represents the normal first log feature, and the other cluster represents the abnormal first log feature. These two cluster clustering models are the predetermined models mentioned above.
[0083] Among them, the test result discrimination device can be set to retrain the predetermined model every certain period of time, for example, the test result discrimination device can be set to re-execute steps S401 to S403 every 10 days, and re-acquire the historical test log by incremental update (in the incremental update method, the historical test log can include all the historical test logs previously acquired, and the newly acquired historical test logs are added at intervals of 10 days), and then pre-processing and feature extraction are performed, and then cluster analysis is performed to generate a predetermined model, which can reflect the results of long-term cluster analysis of the historical test log; of course, the test result discrimination device can be set to re-execute steps S401 to S403 every 15 days, and acquire the historical test log by real-time update (in the real-time update method, the historical test log can only include all the historical test logs within 15 days, and the historical test logs re-acquired at intervals of 15 days), and then pre-processing and feature extraction are performed, and then cluster analysis is performed to generate a predetermined model, which can reflect the results of the phased (i.e., only within the most recent 15 days) cluster analysis. The above-mentioned interval time and update method of the embodiment of the present application are not limited.
[0084] After the above-mentioned predetermined model is trained, the predetermined model can be used to judge the test results output by the software system.
[0085] S404. After the tested software system receives the real-time test parameters for testing, the test results output by the software system are obtained.
[0086] Specifically, the test result determination method provided in the embodiment of the present application is described by taking the registration function test as an example.
[0087] The tester will simulate the user to make a registration request to the software system being tested. First, the tester enters various real-time test parameters (such as mobile phone number and password, etc.) required for registration in the display interface of the software system being tested. After the display interface of the software system receives the real-time test parameters input by the user, it transmits the real-time test parameters to the background program through the interface between the display interface and the background program. However, at this time, the interface between the display interface and the background program has an error. The software system will modify the interface error problem between the display interface and the background program, and output a test result of "Please try again later" to the user. The test result determination device obtains the test result of "Please try again later" output by the software system.
[0088] S405. When it is determined that the test result is an ambiguous result, obtain a real-time test log generated by the software system testing the real-time test parameters.
[0089] In step S405, the test result judging device will judge the test result of "Please try again later" obtained in step S404, wherein the test result judging device detects, by querying the development function description document of the software system, that the test result of "Please try again later" may correspond to problems such as an interface error between the display interface and the background program, an interface error between the background program and the database, and unstable network conditions. The test result judging device cannot clearly determine which problem "Please try again later" corresponds to, and therefore the test result judging device will determine that the test result of "Please try again later" is an unclear result.
[0090] Next, the test result determination device obtains the real-time test log generated when the software system tests the real-time test parameters through the data collection tool logstash.
[0091] S406. Preprocess the acquired real-time test log to generate a second log text, perform feature extraction on the second log text after word segmentation, and generate a second log feature.
[0092] In step S406, the real-time test log obtained in the above step S405 is preprocessed, and the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning. Then, the second log text is segmented and feature extraction is performed to generate a second log feature. The feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency. The preprocessing and feature extraction here are similar to the above step S402, except that the historical test log in step S402 is replaced with a real-time test log, which will not be repeated here.
[0093] S407. Input the second log feature into a predetermined model, and generate a prompt message when determining whether the real-time test log is abnormal or normal according to the judgment result output by the predetermined model.
[0094] The second log feature generated in step S406 is input into the predetermined model generated in step S403, and the distance c between the second log feature and a cluster representing the normal first log feature in the predetermined model is calculated, and the distance d between the second log feature and a cluster representing the abnormal first log feature in the predetermined model is calculated, and it is determined whether the distance c is smaller or the distance d is smaller. If the distance c is smaller, it means that the distance between the second log feature at this time and the cluster representing the normal first log feature in the predetermined model is small, and the second log feature can be determined to be normal, so the real-time test log corresponding to the second log feature is also normal, and the test result determination device can generate a prompt message indicating that the test result is normal; if the distance d is smaller, it means that the distance between the second log feature at this time and the cluster representing the abnormal first log feature in the predetermined model is small, and the second log feature can be determined to be abnormal, so the real-time test log corresponding to the second log feature is also abnormal, and the test result determination device can generate a prompt message indicating that the test result is abnormal.
[0095] The above test result discrimination method shows in more detail that the test result discrimination device performs preprocessing and feature extraction on the real-time test log to generate the second log feature, and also performs preprocessing and feature extraction on the historical test log, and then cluster analysis is performed on the first log feature generated by preprocessing and feature extraction of the historical test log to generate a predetermined model. Then, the second log feature is input into the predetermined model, and the second log feature is determined to be close to a cluster representing the normal first log feature in the predetermined model or close to a cluster representing the abnormal first log feature according to the output judgment result of the predetermined model, and a prompt message is generated. If the second log feature is close to a cluster representing the normal first log feature in the predetermined model, it means that the second log feature is normal, and the corresponding real-time test log is also normal, and a prompt message indicating that the test result corresponding to the real-time test log is normal is generated; if the second log feature is close to a cluster representing the abnormal first log feature in the predetermined model, it means that the second log feature is abnormal, and the corresponding real-time test log is also abnormal, and a prompt message indicating that the test result corresponding to the real-time test log is abnormal is generated. Such a method of using log features for judgment and using log features for clustering will be more accurate and can more accurately judge whether the real-time test log is normal or abnormal.
[0096] The above mainly introduces the scheme provided by the embodiment of the present application from the perspective of the method. In order to realize the various functions in the above method, the test result discrimination device also includes hardware structures and / or computer software corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in conjunction with the methods of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function in the above method is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different hardware structures and / or computer software to implement the methods described in the embodiments of the present application for each specific application, but such implementation should not be considered to exceed the scope of the present application.
[0097] The embodiment of the present application can divide the functional modules of the test result discrimination device according to the above method example. For example, each method can be divided into each functional module for execution, or two or more methods can be divided into one functional module for execution. The above functional modules can be implemented in the form of hardware or in the form of computer software. Optionally, the division of functional modules in the embodiment of the present application is schematic, which is only a logical function division, and there may be other division methods in actual implementation. The scheme provided in the embodiment of the present application will be introduced from the perspective of functional modules below.
[0098] The embodiment of the present application provides a test result determination device, referring to Figure 5As shown, Figure 5 The structure diagram of the test result determination device provided in the embodiment of the present application is shown in FIG. Figure 5 As shown, the test result determination device 50 includes a test result determination module 501 , a real-time log collection module 502 , an analysis and prediction module 503 and a result display module 504 .
[0099] The test result judgment module 501 is used to obtain the test result output by the software system after the tested software system receives the real-time test parameters for testing, and to determine whether the test result is clear; the real-time log collection module 502 is used to obtain the real-time test log generated by the software system for testing the real-time test parameters when the test result judgment module 501 determines that the test result is an unclear result; the analysis and prediction module 503 is used to input the real-time test log obtained by the real-time log collection module 502 into a predetermined model, and determine whether the real-time test log is abnormal or normal according to the judgment result output by the predetermined model; wherein the predetermined model includes: a model generated by clustering analysis of the historical test logs of the software system with abnormal and normal as cluster centers, and the historical test log is a test log generated after the software system tests the received historical test parameters; the result display module 504 is used to generate a prompt message when the analysis and prediction module 503 determines that the real-time test log is abnormal or normal according to the judgment result output by the predetermined model, the prompt message is used to prompt that the test result is abnormal when the real-time test log is abnormal, or the prompt message is used to prompt that the test result is normal when the real-time test log is normal.
[0100] Optional, see Figure 6 As shown, the test result determination device 50 also includes a historical log collection module 505 and a cluster training module 506; the historical log collection module 505 is used to read the historical test logs generated by the software system in the database; the cluster training module 506 is used to perform cluster analysis on the historical test logs obtained by the historical log collection module 505 with abnormal and normal as cluster centers to generate a predetermined model; or, the historical log collection module 505 is used to obtain the historical test logs generated after the software system receives the historical test parameters for testing; the cluster training module 506 is used to perform cluster analysis on the historical test logs obtained by the historical log collection module 505 with abnormal and normal as cluster centers to generate a predetermined model.
[0101] Optional, see Figure 6As shown, the test result determination device 50 also includes a first log preprocessing module 507; the first log preprocessing module 507 is used to preprocess the historical test log obtained by the historical log collection module 505 to generate a first log text, and the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning; feature extraction is performed on the first log text after word segmentation to generate a first log feature, and the feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency; the clustering training module 506 is specifically used to perform cluster analysis on the first log feature generated by the first log preprocessing module 507 with abnormal and normal as cluster centers to generate a predetermined model.
[0102] Optionally, the real-time log collection module 502 is specifically used to obtain real-time test logs through the data collection tool logstash when the software system tests the real-time test parameters.
[0103] Optional, see Figure 6 As shown, the test result determination device 50 also includes a second log preprocessing module 508; the second log preprocessing module 508 is used to preprocess the real-time test log obtained by the real-time log collection module 502 to generate a second log text, and the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning; feature extraction is performed on the second log text after word segmentation to generate a second log feature, and the feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency; the analysis and prediction module 503 is specifically used to input the second log feature generated by the second log preprocessing module 508 into a predetermined model.
[0104] Optionally, the real-time test parameters include test parameters of an interface test, or the real-time test parameters include test parameters of an interface test.
[0105] In actual implementation, the test result judgment module 501, the real-time log collection module 502, the analysis and prediction module 503, the result display module 504, the historical log collection module 505, the cluster training module 506, the first log preprocessing module 507 and the second log preprocessing module 508 can be composed of Figure 1 The processor 101 shown in the figure calls the program code in the memory 102 to implement the Figure 2 The processor 201 shown calls the program code stored in the processor 201 to implement. The specific execution process can be referred to Figure 3 Figure 4 The description of the method part in the method embodiment shown will not be repeated here.
[0106] Those skilled in the art should be aware that in one or more of the above examples, the functional modules described in this application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functional modules can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.
[0107] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functional modules can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0108] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the functional modules is only a logical function division, and there may be other division methods in actual implementation. For example, multiple functional modules can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection between devices can be realized through some interfaces, and the indirect coupling or communication connection of the devices can be realized in electrical, mechanical or other forms. The functional modules described as separate components may or may not be physically separated, and the components displayed as functional modules may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed in multiple different places. Some or all of the functional modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
[0109] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk.
[0110] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, which includes computer execution instructions. When the computer execution instructions are executed on a computer, the computer executes the test result determination method provided in the above embodiment.
[0111] The embodiment of the present application also provides a computer program, which can be directly loaded into a memory and contains software code. After being loaded and executed by a computer, the computer program can implement the test result determination method provided in the above embodiment.
[0112] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A test result determination method, characterized in that: include: After the software system under test receives the real-time test parameters for testing, the test results output by the software system are obtained; The test result is normal or abnormal or unclear; When it is determined that the test result is an ambiguous result, obtaining a real-time test log generated by the software system testing the real-time test parameters; Input the real-time test log into a predetermined model, and generate prompt information when determining whether the real-time test log is abnormal or normal according to the judgment result output by the predetermined model; The prompt information is used to prompt that the test result is abnormal when the real-time test log is abnormal, or the prompt information is used to prompt that the test result is normal when the real-time test log is normal; The predetermined model includes: a model generated by clustering analysis of the historical test logs of the software system with abnormal and normal as cluster centers, and the historical test logs are test logs generated after the software system tests the received historical test parameters.
2. The test result determination method according to claim 1, characterized in that: Before inputting the real-time test log into a predetermined model, the method includes: Reading the historical test log generated by the software system in a database, and performing cluster analysis on the historical test log with abnormality and normality as cluster centers to generate the predetermined model; Alternatively, after the software system receives the historical test parameters for testing, the historical test log is generated, and cluster analysis is performed on the historical test log with abnormality and normality as cluster centers to generate the predetermined model.
3. The test result determination method according to claim 2, characterized in that: Before performing cluster analysis on the historical test logs with abnormal and normal as cluster centers to generate the predetermined model, the method includes: Preprocessing the acquired historical test log to generate a first log text, wherein the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning; After word segmentation, feature extraction is performed on the first log text to generate a first log feature, wherein the feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency; The generating the predetermined model by clustering analysis on the historical test logs with abnormality and normality as cluster centers includes: generating the predetermined model by clustering analysis on the first log features with abnormality and normality as cluster centers.
4. The test result determination method according to claim 1, characterized in that: The obtaining of the real-time test log generated by the software system when testing the real-time test parameters includes: obtaining the real-time test log by using a data acquisition tool logstash when the software system tests the real-time test parameters.
5. The test result determination method according to claim 1, characterized in that: After obtaining the real-time test log generated by the software system testing the real-time test parameters, the method further includes: Preprocessing the real-time test log to generate a second log text, wherein the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning; After word segmentation, feature extraction is performed on the second log text to generate second log features, wherein the feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency; The step of inputting the real-time test log into a predetermined model includes: inputting the second log feature into a predetermined model.
6. The test result determination method according to claim 1, characterized in that: The real-time test parameters include test parameters of an interface test, or the real-time test parameters include test parameters of an interface test.
7. A test result determination device, characterized in that: include: Test result judgment module, real-time log collection module, analysis and prediction module, and result display module; The test result judgment module is used to obtain the test result output by the software system after the tested software system receives the real-time test parameters for testing, and judge whether the test result is clear; the test result is normal, abnormal or unclear; The real-time log collection module is used to obtain the real-time test log generated by the software system testing the real-time test parameters when the test result judgment module determines that the test result is an ambiguous result; The analysis and prediction module is used to input the real-time test log acquired by the real-time log collection module into a predetermined model, and determine whether the real-time test log is abnormal or normal according to the judgment result output by the predetermined model; wherein the predetermined model includes: a model generated by clustering analysis of the historical test logs of the software system with abnormal and normal as cluster centers, and the historical test logs are test logs generated after the software system tests the received historical test parameters; The result display module is used to generate a prompt message when the analysis and prediction module determines that the real-time test log is abnormal or normal according to the judgment result output by the predetermined model. The prompt message is used to prompt that the test result is abnormal when the real-time test log is abnormal, or the prompt message is used to prompt that the test result is normal when the real-time test log is normal.
8. The test result determination device according to claim 7, characterized in that: The test result determination device also includes a history log collection module and a cluster training module; The historical log collection module is used to read the historical test log generated by the software system in the database; the cluster training module is used to perform cluster analysis on the historical test log acquired by the historical log collection module with abnormal and normal as cluster centers to generate the predetermined model; Alternatively, the historical log collection module is used to obtain the historical test log after the software system receives historical test parameters for testing; the clustering training module is used to perform clustering analysis on the historical test log obtained by the historical log collection module with abnormal and normal as cluster centers to generate the predetermined model.
9. The test result determination device according to claim 8, characterized in that: The test result determination device also includes a first log preprocessing module; The first log preprocessing module is used to preprocess the historical test log acquired by the historical log collection module to generate a first log text, wherein the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning; After word segmentation, feature extraction is performed on the first log text to generate a first log feature, wherein the feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency; The cluster training module is specifically used to perform cluster analysis on the first log features generated by the first log preprocessing module with abnormality and normality as cluster centers to generate the predetermined model.
10. The test result determination device according to claim 7, characterized in that: The real-time log collection module is specifically used to obtain the real-time test log through the data collection tool logstash when the software system tests the real-time test parameters.
11. The test result determination device according to claim 7, characterized in that: The test result determination device also includes a second log preprocessing module; The second log preprocessing module is used to preprocess the real-time test log acquired by the real-time log collection module to generate a second log text, wherein the preprocessing includes at least one or more of the following: time slice alignment, missing value processing, and sensitive field cleaning; After word segmentation, feature extraction is performed on the second log text to generate second log features, wherein the feature extraction includes extracting one or more of the following log features: word frequency, inverse word frequency; The analysis and prediction module is specifically used to input the second log features generated by the second log preprocessing module into the predetermined model.
12. The test result determination device according to claim 7, characterized in that: The real-time test parameters include test parameters of an interface test, or the real-time test parameters include test parameters of an interface test.
13. A test result determination device, characterized in that: It includes a memory and a processor; the memory is used to store computer-executable instructions, and the processor is connected to the memory via a bus; When the test result determination device is running, the processor executes the computer-executable instructions stored in the memory, so that the test result determination device performs the test result determination method according to any one of claims 1 to 6.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer-executable instructions. When the computer-executable instructions are executed on a computer, the computer is enabled to execute the test result determination method according to any one of claims 1 to 6.
