Artificial Intelligence-Based Test Report Analysis Method, Device, Equipment and Medium

Abstract

This application is applicable to the field of automated testing technologies, and particularly relates to a method, device, equipment, and medium for analyzing test reports based on artificial intelligence. The method includes obtaining a root cause set, a training set, and a second test report set constructed based on a first test report set, extracting the error key points of each test report in the second test report set, matching each error key point with the root cause set to obtain a root cause field, using the root cause field as the annotation for the corresponding test report, generating test data vectors based on the annotated test reports and the corresponding error key points to form a test set, performing K-nearest neighbor similarity analysis on the training set and the test set, determining the K value corresponding to the maximum similarity as the target number, extracting the error key points of the test report to be analyzed, and determining the root causes that match the target number from the root cause set as the analysis result of the test report to be analyzed, thereby realizing the automated analysis of test reports and effectively improving the test development efficiency.

Description

Technical Field

[0001] This application is applicable to the field of automated testing technology, and particularly relates to a method, device, equipment and medium for analyzing test reports based on artificial intelligence. Background Art

[0002] Currently, in the early stage of the test operation process, testers will plan and write a test case set for the business scenario, and arrange the test case context to simulate the business scenario. After the test cases are executed, generally, the test report will be obtained from the tool platform, and the results of the test case execution will be obtained from the test report for analysis. This kind of original operation that requires testers to analyze often needs to be based on the information at the time of the original error, for example, error logs, error stack information, assertion data, and even response data, so as to accurately analyze and obtain the approximate root cause of the error. And this initial analysis will consume a lot of time. However, for general complex software systems, the generation of system exceptions is often an uncontrollable small "sandpile experiment". The prior art does not have a technology that can give an error analysis result based on the test case execution report, which limits the test efficiency of developers and causes waste of resources. Therefore, how to perform automated predictive analysis on test reports to improve test development efficiency has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, the embodiments of this application provide a method, device, equipment and medium for analyzing test reports based on artificial intelligence to solve the problem of reducing manual participation to achieve prediction tasks and improve efficiency and prediction accuracy.

[0004] In the first aspect, the embodiments of this application provide a method for analyzing test reports based on artificial intelligence, and the test report analysis method includes:

[0005] Obtain a root cause set and a training set constructed based on a first test report set, where the training set includes at least one training data vector, and the training data vector is formed by a test report with a labeled root cause field;

[0006] Obtain a second test report set, and extract the error key points of each test report in the second test report set, and match each error key point with the root cause set to obtain the root cause field corresponding to the error key point;

[0007] Use the root cause field as the label of the corresponding test report, and generate a test data vector based on the labeled test report and the corresponding error key point to obtain a test set;

[0008] Perform K-nearest neighbor similarity analysis on the training set and the test set, and determine that the K value corresponding to the maximum similarity is the target number;

[0009] Extract the key error points of the test report to be analyzed, and determine that the root causes matching the target number from the set of root causes are the analysis results of the test report to be analyzed.

[0010] In one embodiment, the set of root causes and the training set constructed based on the first test report set specifically include:

[0011] Obtain the first test report set, and extract the key error points of each test report in the first test report set;

[0012] Collect the root cause fields of all key error points manually, and map all the root cause fields to the corresponding key error points to form a set of root causes;

[0013] Use the root cause field as the annotation of the corresponding test report, obtain the annotated test report, vectorize the annotated test report and the corresponding key error points, and determine that the vectorization result is a training data vector in the training set.

[0014] In one embodiment, extracting the key error points of each test report in the first test report set includes:

[0015] Extract the first error information of each test report in the first test report set;

[0016] Perform noise reduction on the first error information, and determine that the noise reduction result is the key error point.

[0017] In one embodiment, collecting the root cause fields of all key error points manually and mapping all the root cause fields to the corresponding key error points to form a set of root causes includes:

[0018] Extract the first type of key error points that are the first classification among all key error points;

[0019] Collect the first type of root cause fields of the first type of key error points manually, map the first type of key error points to the corresponding first type of root cause fields to form a first mapping set, and determine that the first mapping set is the set of root causes.

[0020] In one embodiment, collecting the root cause fields of all key error points manually and mapping all the root cause fields to the corresponding key error points to form a set of root causes includes:

[0021] Extract the second type of key error points that are the second classification among all key error points;

[0022] Collect the second type of root cause fields of the second type of key error points manually, encode the second type of key error points using the raise statement, map the encoding result to the corresponding second type of root cause fields to form a second mapping set, and determine that the second mapping set is the set of root causes.

[0023] In one embodiment, collect the root cause fields of all error reporting key points manually, and map all the root cause fields to the corresponding error reporting key points to form a root cause set, including:

[0024] Extract the third type of error reporting key points that are of the third classification among all error reporting key points;

[0025] Collect the third type of root cause fields of the third type of error reporting key points manually, and perform semantic analysis on the third type of error reporting key points;

[0026] Map the semantic analysis results to the corresponding third type of root cause fields to form a third mapping set, and determine the third mapping set as the root cause set.

[0027] In one embodiment, extracting the error reporting key points of each test report in the second test report set includes:

[0028] Extract the second error information of each test report in the second test report set;

[0029] Perform noise reduction on the second error information, and determine the noise reduction result as the error reporting key point.

[0030] In a second aspect, an embodiment of the present application provides a test report analysis device based on artificial intelligence. The test report analysis device includes:

[0031] A benchmark acquisition module, configured to acquire a root cause set and a training set constructed based on a first test report set. The training set includes at least one training data vector, and the training data vector is formed by a test report with a labeled root cause field;

[0032] A root cause analysis module, configured to acquire a second test report set, extract the error reporting key points of each test report in the second test report set, and match each error reporting key point with the root cause set to obtain the root cause field corresponding to the error reporting key point;

[0033] A test set generation module, configured to use the root cause field as the annotation of the corresponding test report, and generate a test data vector according to the annotated test report and the corresponding error reporting key point to obtain a test set;

[0034] A similarity analysis module, configured to perform K-nearest neighbor similarity analysis on the training set and the test set, and determine the K value corresponding to the maximum similarity as the target number;

[0035] An analysis result determination module, configured to extract the error reporting key points of the test report to be analyzed, and determine the root causes that match the target number from the root cause set as the analysis result of the test report to be analyzed.

[0036] In one embodiment, the root cause set and the training set constructed based on the first test report set in the reference acquisition module specifically include:

[0037] A key point extraction unit, configured to obtain the first test report set and extract the error key points of each test report in the first test report set;

[0038] A root cause set formation unit, configured to collect the root cause fields of all error key points manually, and map all the root cause fields to the corresponding error key points to form a root cause set;

[0039] A training set formation unit, configured to use the root cause field as the annotation of the corresponding test report to obtain the annotated test report, vectorize the annotated test report and the corresponding error key points, and determine that the vectorization result is a training data vector in the training set.

[0040] In one embodiment, the key point extraction unit includes:

[0041] A first information extraction subunit, configured to extract the first error information of each test report in the first test report set;

[0042] A first key point determination subunit, configured to denoise the first error information and determine that the denoising result is the error key point.

[0043] In one embodiment, the root cause set formation unit includes:

[0044] A first classification subunit, configured to extract the first type of error key points that are of the first classification among all error key points;

[0045] A first set determination subunit, configured to collect the first type of root cause fields of the first type of error key points manually, map the first type of error key points to the corresponding first type of root cause fields to form a first mapping set, and determine that the first mapping set is the root cause set.

[0046] In one embodiment, the root cause set formation unit includes:

[0047] A second classification subunit, configured to extract the second type of error key points that are of the second classification among all error key points;

[0048] An encoding subunit, configured to collect the second type of root cause fields of the second type of error key points manually and encode the second type of error key points using the raise statement;

[0049] A second set determination subunit, configured to map the encoding result to the corresponding second type of root cause fields to form a second mapping set, and determine that the second mapping set is the root cause set.

[0050] In one embodiment, the root cause set forming unit includes:

[0051] A third classification subunit, configured to extract third - type error - reporting key points that are of the third classification among all error - reporting key points;

[0052] A semantic analysis subunit, configured to collect third - type root cause fields of the artificial for the third - type error - reporting key points, and perform semantic analysis on the third - type error - reporting key points;

[0053] A third set determination subunit, configured to map the semantic analysis result to the corresponding third - type root cause field to form a third mapping set, and determine the third mapping set as the root cause set.

[0054] In one embodiment, the root cause analysis module includes:

[0055] A second information extraction subunit, configured to extract second error - reporting information of each test report in the second test report set;

[0056] A second key point determination subunit, configured to denoise the second error - reporting information, and determine the denoising result as the error - reporting key point.

[0057] In a third aspect, an embodiment of the present application provides a computer device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the test report analysis method described in the first aspect is implemented.

[0058] In a fourth aspect, an embodiment of the present application provides a computer - readable storage medium, which stores a computer program. When the computer program is executed by a processor, the test report analysis method described in the first aspect is implemented.

[0059] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: The present application obtains a root cause set and a training set constructed based on a first test report set, obtains a second test report set, extracts error - reporting key points of each test report in the second test report set, matches each error - reporting key point with the root cause set to obtain the root cause field corresponding to the error - reporting key point, uses the root cause field as the annotation of the corresponding test report, generates a test data vector according to the annotated test report and the corresponding error - reporting key point to obtain a test set, performs K - nearest neighbor similarity analysis on the training set and the test set, determines that the K value corresponding to the maximum similarity is the target number, extracts the error - reporting key points of the test report to be analyzed, and determines that the root causes matching the target number from the root cause set are the analysis results of the test report to be analyzed, thereby realizing the automated analysis of the test report and effectively improving the test development efficiency. Description of the Drawings

[0060] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for use in the embodiments or the description of the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0061] Figure 1 It is a schematic diagram of an application environment of a method for analyzing test reports based on artificial intelligence provided in Embodiment 1 of the present application;

[0062] Figure 2 It is a schematic flowchart of a method for analyzing test reports based on artificial intelligence provided in Embodiment 2 of the present application;

[0063] Figure 3 It is a schematic flowchart of a method for analyzing test reports based on artificial intelligence provided in Embodiment 3 of the present application;

[0064] Figure 4 It is a schematic structural diagram of a device for analyzing test reports based on artificial intelligence provided in Embodiment 4 of the present application;

[0065] Figure 5 It is a schematic structural diagram of a computer device provided in Embodiment 5 of the present application. Detailed implementation manners

[0066] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are proposed to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0067] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0068] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0069] As used in the specification of this application and the appended claims, the term "if" may be construed contextually as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrases "if determined" or "if [the described condition or event] is detected" may be construed contextually to mean "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0070] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are used only for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0071] The reference to "one embodiment" or "some embodiments" etc. described in the specification of this application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0072] The embodiments of this application may acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use the knowledge to obtain the best results.

[0073] Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, robotics, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0074] It should be understood that the magnitudes of the sequence numbers of the steps in the following embodiments do not mean the order of execution is prior or posterior, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0075] To illustrate the technical solution of this application, the following specific embodiments are used for illustration.

[0076] A method for analyzing test reports based on artificial intelligence provided in the first embodiment of this application can be applied in an application environment such as Figure 1 where the client communicates with the server. The client includes, but is not limited to, computer devices such as palm computers, desktop computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, cloud computer devices, personal digital assistants (PDAs), etc. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0077] See Figure 2 , which is a schematic flowchart of a method for analyzing test reports based on artificial intelligence provided in the second embodiment of this application. The above method for analyzing test reports based on artificial intelligence is applied to Figure 1 the server in Figure 2 . The computer device corresponding to the server is connected to a corresponding database to obtain corresponding data in the database. The above computer device can also be connected to a corresponding client, and the client is operated by a user. The user can provide a corresponding test report to the server through the client. As

[0078] shown in

[0079] Step S201, obtain a root cause set and a training set constructed based on the first test report set.

[0080] In this application, the first test report set is a test report obtained after a user tests a system to be tested using corresponding test cases. The test report generally contains corresponding test results, test durations, test process information, etc. In addition, the test report also contains context information of the test cases used when an error occurs, etc., which helps to determine the cause of the error.

[0081] The first test report is analyzed to determine the error key points in each test report and the root cause fields corresponding to the error key points. Among them, using the root cause fields to label the test reports forms a set of training data vectors, that is, a training set is formed. The training set includes at least one training data vector, and the training data vector is formed by the test reports labeled with root cause fields.

[0082] Optionally, the root cause set and the training set constructed based on the first test report set specifically include:

[0083] Obtain the first test report set, and extract the error key points of each test report in the first test report set;

[0084] Collect the root cause fields of all error key points manually, and map all the root cause fields to the corresponding error key points to form a root cause set;

[0085] Use the root cause field as the annotation of the corresponding test report, obtain the annotated test report, vectorize the annotated test report and the corresponding error key points, and determine that the vectorization result is a training data vector in the training set.

[0086] Among them, when in use, for the error key points and the root cause field input items corresponding to the test report, after displaying the error key points to the user, the user inputs corresponding data in the root cause field input item to form a root cause field, that is, a mapping relationship is formed between the error key point and the root cause field, and the set of this mapping relationship is the root cause set.

[0087] Specifically, vectorize the test report marked with the root cause field together with the error key points, and use its vectorization result, that is, a vector, as one in the training set. Of course, in other implementation manners, other contents that can characterize the root cause field, the error key point, and the features of the test report can also be vectorized. For example, only use the context information of the test case corresponding to the error key point in the test report.

[0088] Optionally, extracting the error key points of each test report in the first test report set includes:

[0089] Extract the first error information of each test report in the first test report set;

[0090] Denoise the first error information, and determine that the denoising result is the error key point.

[0091] Among them, the error information in the test report can be extracted by using artificial intelligence methods. For example, by using feature recognition methods, locate the start position and end position of the error information in the test report, and extract the content between the start position and the end position as the error information.

[0092] After obtaining the error information, denoise the error information, remove unnecessary words such as modal particles and auxiliary words, and finally leave the error key points, that is, error keywords, error key sentences, etc.

[0093] Optionally, collecting the root cause fields of all error key points manually and mapping all the root cause fields to the corresponding error key points to form a root cause set includes:

[0094] Extract the first - type error - reporting key points that are of the first classification among all error - reporting key points;

[0095] Collect the first - type root - cause fields of the first - type error - reporting key points by humans, map the first - type error - reporting key points to the corresponding first - type root - cause fields to form a first mapping set, and determine the first mapping set as the root - cause set.

[0096] Among them, for the root - cause set, the root - cause types corresponding to different types of error - reporting key points are also different. For example, for common error - reporting key points, their corresponding keywords, key sentences, etc. are fixed and there is no ambiguity. Therefore, directly map these error - reporting key points to the root - cause fields to obtain the first mapping set.

[0097] This first mapping set can be expressed as a whitelist. In the subsequent matching process, the corresponding root - cause can be obtained without excessive coding, semantic analysis, etc., thus improving the efficiency of determining the root - cause.

[0098] Optionally, collect the root - cause fields of all error - reporting key points by humans, and map all the root - cause fields to the corresponding error - reporting key points to form a root - cause set, including:

[0099] Extract the second - type error - reporting key points that are of the second classification among all error - reporting key points;

[0100] Collect the second - type root - cause fields of the second - type error - reporting key points by humans, use the raise statement to code the second - type error - reporting key points, map the coding results to the corresponding second - type root - cause fields to form a second mapping set, and determine the second mapping set as the root - cause set.

[0101] Among them, for the root - cause set, the root - cause types corresponding to different types of error - reporting key points are also different. For example, for error - reporting key points in different business domains, their corresponding keywords, key sentences, etc. are not fixed but are related to the business domain. By using the method of coding with the raise statement, the root - causes corresponding to different keywords or sentences in different domains can be classified. Therefore, these error - reporting key points need to be coded and then mapped to the root - cause fields to obtain the second mapping set.

[0102] This second mapping set can be expressed as a list that needs to be coded for matching. In the subsequent matching process, the quantity to be matched needs to be coded, and then the coding results are matched, which can improve the matching accuracy to a certain extent.

[0103] Optionally, collect the root - cause fields of all error - reporting key points by humans, and map all the root - cause fields to the corresponding error - reporting key points to form a root - cause set, including:

[0104] Extract the third - type error - reporting key points that are of the third classification among all error - reporting key points;

[0105] Collect the third type of root cause fields of the third type of error reporting key points manually, and perform semantic analysis on the third type of error reporting key points;

[0106] Map the semantic analysis results to the corresponding third type of root cause fields to form a third mapping set, and determine the third mapping set as the root cause set.

[0107] Among them, in terms of the root cause set, the root cause types corresponding to different types of error reporting key points are also different. For example, for other or some defined error reporting key points, their corresponding keywords, key sentences, etc. are more irregular and have no pattern to follow, and semantic analysis needs to be performed on them to obtain the semantic analysis results, that is, to represent the true meaning of the error reporting key points. Therefore, map the semantic analysis results corresponding to this part of the error reporting key points to the root cause fields to obtain the third mapping set.

[0108] This third mapping set can be expressed as a list that requires semantic analysis for matching. In the subsequent matching process, use semantic analysis to analyze the quantity to be matched to match the corresponding root cause, which can improve the matching accuracy to a certain extent.

[0109] Step S202, obtain the second test report set, extract the error reporting key points of each test report in the second test report set, and match each error reporting key point with the root cause set to obtain the root cause field corresponding to the error reporting key point.

[0110] In this application, the second test report set is the test report obtained after the user uses the corresponding test cases to test the above-mentioned system to be tested. The test report generally contains the corresponding test results, test duration, test process information, etc. In addition, the test report also contains the context information of the test cases used when an error occurs, etc., which helps to judge the cause of the error.

[0111] Similarly, the error reporting key points are the keywords, key sentences, etc. that characterize the errors in the test report. Matching the error reporting key points with the above-mentioned root cause set can determine the root cause field corresponding to each error reporting key point, that is, determine the cause corresponding to each error.

[0112] Optionally, extracting the error reporting key points of each test report in the second test report set includes:

[0113] Extract the second error information of each test report in the second test report set;

[0114] Perform noise reduction on the second error information, and determine the noise reduction result as the error reporting key point.

[0115] Among them, the error messages in the test report can be extracted by using artificial intelligence. For example, by using feature recognition, the starting position and ending position of the error message in the test report are located, and the content between the starting position and the ending position is extracted as the error message.

[0116] After obtaining the error message, noise reduction is performed on the error message to remove unnecessary expressions such as modal particles and auxiliary words, and finally the key error points, namely error keywords, key error statements, etc., are left.

[0117] Matching can be performed by using the similarity of words or sentences. During the matching process, a 100% hit can be used as a condition to obtain the most accurate result.

[0118] Step S203: Use the root cause field as the annotation for the corresponding test report. According to the annotated test report and the corresponding error key points, generate a test data vector to obtain a test set.

[0119] In this application, when obtaining the root cause field corresponding to the error key point of the test report in the second test report set, use this root cause field as the annotation of the test report to form an annotated test report. Vectorize the annotated test report and the corresponding error key points, and the result of the vectorization is a test data vector, and then a test set is obtained.

[0120] To increase the adaptability of the training set and the test set in subsequent analysis, the process and parameters for generating the test data vector are the same as those for generating the training data vector. Specifically, vectorize the test report annotated with the root cause field together with the error key points, and use the vectorization result, which is also a vector, as one in the training set. Of course, in other embodiments, other content that can represent the characteristics of the root cause field, error key points, and test report can also be vectorized. For example, only use the context information of the test case corresponding to the error key point in the test report.

[0121] Step S204: Perform K-nearest neighbor similarity analysis on the training set and the test set, and determine that the K value corresponding to the maximum similarity is the target number.

[0122] In this application, K-nearest neighbor is (K-Nearest Neighbor, KNN). The purpose is to find the K records in the training set that are closest to the new data, and then determine the category of the new data according to the main classification of the K records. This algorithm involves three main factors: the training set, the measurement of distance or similarity, and the size of K.

[0123] For any K value, the following are performed respectively:

[0124] 1) Calculate the distance: Given a test object, calculate its distance from each object in the training set;

[0125] 2) Find neighbors: Identify the K training objects with the closest distance as the neighbors of the test object.

[0126] 3) Calculate similarity: Calculate the similarity between the final test object and the training objects, and retain the similarity results for all K - value cases. Among them, the similarity result can be the average value of the overall similarity.

[0127] 4) Find the maximum: That is, determine the K value corresponding to the maximum similarity result.

[0128] Among them, the closer the distance should mean the greater the possibility that these two points belong to the same category. Distance measurement includes Euclidean distance, cosine of the included angle, etc. In this application, for the vectors of text, it is more appropriate to use cosine to calculate similarity than Euclidean distance.

[0129] Step S205: Extract the error - reporting key points of the test report to be analyzed, and determine that the root causes that match the target number from the root - cause set are the analysis results of the test report to be analyzed.

[0130] In this application, for the test report to be analyzed, the error - reporting key points of the test report to be analyzed are extracted in the same way as those of the test reports in the first test report set and the second test report set.

[0131] The root causes corresponding to the error - reporting key points are matched from the root - cause set by a matching method. The number of matched root causes is the number corresponding to the above - mentioned K value. For example, when the K value is 3, correspondingly, the test report to be analyzed will finally match 3 root causes, thus adaptively giving a certain number of root causes instead of generating root causes in a manually set way, ensuring the objectivity of root - cause generation and improving the accuracy.

[0132] In the embodiment of this application, by obtaining the root - cause set and the training set constructed based on the first test report set, obtaining the second test report set, extracting the error - reporting key points of each test report in the second test report set, matching each error - reporting key point with the root - cause set to obtain the root - cause fields corresponding to the error - reporting key points, using the root - cause fields as the annotations of the corresponding test reports, generating test data vectors according to the annotated test reports and the corresponding error - reporting key points to obtain a test set, performing K - nearest - neighbor similarity analysis on the training set and the test set, determining that the K value corresponding to the maximum similarity is the target number, extracting the error - reporting key points of the test report to be analyzed, and determining that the root causes that match the target number from the root - cause set are the analysis results of the test report to be analyzed, thereby realizing the automated analysis of test reports and effectively improving the test development efficiency.

[0133] See Figure 3, which is a schematic flowchart of a method for analyzing test reports based on artificial intelligence provided in the third embodiment of the present application. As Figure 3 shown, the method for analyzing test reports based on artificial intelligence may include the following steps:

[0134] Step S301, obtain a training set, a first mapping set, a second mapping set, and a third mapping set constructed based on the first test report set.

[0135] In the present application, with respect to the root cause set, the root cause types corresponding to different types of error key points are also different. For example, for common error key points, the corresponding keywords, key sentences, etc. are fixed and there is no ambiguity. Therefore, these error key points are directly corresponded to the root cause fields to obtain the first mapping set. Another example is that for error key points in different business domains, although the corresponding keywords, key sentences, etc. are not fixed, they are related to the business domain. By using the raise statement for encoding, different keywords or sentences corresponding to different domains can be classified according to their roots. Therefore, these error key points need to be encoded and then corresponded to the root cause fields to obtain the second mapping set. For another example, for other or some defined error key points, the corresponding keywords, key sentences, etc. are more unfixed and there is no regular pattern to follow. Semantic analysis needs to be performed on them to obtain the semantic analysis result, that is, to represent the true meaning of the error key point. Therefore, the semantic analysis results corresponding to these error key points are corresponded to the root cause fields to obtain the third mapping set.

[0136] The above three mapping sets are jointly used as the root cause set for subsequent matching work, which helps to improve efficiency and accuracy.

[0137] Step S302, obtain the second test report set, and extract the error key points of each test report in the second test report set.

[0138] Among them, part of the content of step S302 is the same as that of step S202 above. For the description of step S202, reference can be made, and it will not be elaborated here.

[0139] Step S303, match each error key point with the first mapping set to obtain the root cause field corresponding to the error key point.

[0140] In the present application, this first mapping set can be expressed as a whitelist, and the corresponding root cause can be obtained without excessive encoding, semantic analysis, etc. in the subsequent matching process, thus improving the efficiency of determining the root cause.

[0141] Step S304, use the raise statement to encode the error key points that are not matched with the first mapping set, and match the encoding result with the second mapping set to obtain the root cause field corresponding to the error key point.

[0142] In this application, this second mapping set can be expressed as a list that needs to be encoded for matching. During the subsequent matching process, the quantity to be matched needs to be encoded, and then the encoded result is matched, which can improve the matching accuracy to a certain extent.

[0143] Step S305: Perform semantic analysis on the error reporting key points that are not matched with the first mapping set and the second mapping set, and match the semantic analysis result with the third mapping set to obtain the root cause field corresponding to the error reporting key point.

[0144] In this application, this third mapping set can be expressed as a list that needs to be matched by semantic analysis. During the subsequent matching process, semantic analysis is used to analyze the quantity to be matched to match the corresponding root cause, which can improve the matching accuracy to a certain extent.

[0145] Step S306: Use the root cause field as the annotation of the corresponding test report. According to the annotated test report and the corresponding error reporting key point, generate a test data vector to obtain a test set.

[0146] Step S307: Perform K-nearest neighbor similarity analysis on the training set and the test set, and determine that the K value corresponding to the maximum similarity is the target number.

[0147] Step S308: Extract the error reporting key points of the test report to be analyzed, and determine that the root cause that matches the target number from the root cause set is the analysis result of the test report to be analyzed.

[0148] Among them, the content of Step S306 to Step S308 is the same as that of the above-mentioned Step S203 to Step S205. For the description, reference can be made to Step S203 to Step S205, and details are not repeated here.

[0149] In an embodiment of the present application, a training set, a first mapping set, a second mapping set, and a third mapping set are obtained based on a first test report set. A second test report set is obtained, and the error-causing key points of each test report in the second test report set are extracted. Each error-causing key point is matched with the first mapping set to obtain the root cause field corresponding to the error-causing key point. The raise statement is used to encode the error-causing key points that do not match the first mapping set. The encoding result is matched with the second mapping set to obtain the root cause field corresponding to the error-causing key point. Semantic analysis is performed on the error-causing key points that do not match the first mapping set and the second mapping set. The semantic analysis result is matched with the third mapping set to obtain the root cause field corresponding to the error-causing key point. The root cause field is used as the annotation of the corresponding test report. According to the annotated test report and the corresponding error-causing key points, test data vectors are generated to obtain a test set. The training set and the test set are subjected to K-nearest neighbor similarity analysis to determine that the K value corresponding to the maximum similarity is the target number. The error-causing key points of the test report to be analyzed are extracted, and the root causes that match the target number from the root cause set are determined as the analysis result of the test report to be analyzed, thereby realizing the automated analysis of the test report, effectively improving the test development efficiency. The above three mapping sets are jointly used as the root cause set for subsequent matching work, which helps to improve the efficiency and accuracy.

[0150] A test report analysis method based on artificial intelligence corresponding to the above embodiment Figure 4 shows a structural block diagram of a test report analysis device based on artificial intelligence provided in Embodiment 4 of the present application. The above test report analysis device is applied to Figure 1 the server in. The computer device corresponding to the server is connected to a corresponding database to obtain corresponding data in the database. The above computer device can also be connected to a corresponding client. The client is operated by a user, and the user can provide a corresponding test report to the server through the client. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown.

[0151] See Figure 4 , the test report analysis device includes:

[0152] A benchmark acquisition module 41, configured to obtain a root cause set and a training set constructed based on a first test report set. The training set includes at least one training data vector, and the training data vector is formed by a test report with an annotated root cause field;

[0153] A root cause analysis module 42, configured to obtain a second test report set, extract the error-causing key points of each test report in the second test report set, and match each error-causing key point with the root cause set to obtain the root cause field corresponding to the error-causing key point;

[0154] The test set generation module 43 is used to use the root cause field as the annotation of the corresponding test report, generate a test data vector according to the annotated test report and the corresponding error reporting key points, and obtain a test set;

[0155] The similarity analysis module 44 is used to perform K-nearest neighbor similarity analysis on the training set and the test set, and determine that the K value corresponding to the maximum similarity is the target number;

[0156] The analysis result determination module 45 is used to extract the error reporting key points of the test report to be analyzed, and determine that the root cause that matches the target number from the root cause set is the analysis result of the test report to be analyzed.

[0157] Optionally, the root cause set and the training set constructed by the benchmark acquisition module 41 specifically include:

[0158] The key point extraction unit is used to obtain the first test report set and extract the error reporting key points of each test report in the first test report set;

[0159] The root cause set formation unit is used to collect the root cause fields of all error reporting key points manually, and map all the root cause fields to the corresponding error reporting key points to form a root cause set;

[0160] The training set formation unit is used to use the root cause field as the annotation of the corresponding test report, obtain the annotated test report, vectorize the annotated test report and the corresponding error reporting key points, and determine that the vectorization result is a training data vector in the training set.

[0161] Optionally, the key point extraction unit includes:

[0162] The first information extraction subunit is used to extract the first error information of each test report in the first test report set;

[0163] The first key point determination subunit is used to denoise the first error information and determine that the denoising result is the error reporting key point.

[0164] Optionally, the root cause set formation unit includes:

[0165] The first classification subunit is used to extract the first type of error reporting key points that are of the first classification among all error reporting key points;

[0166] The first set determination subunit is used to collect the first type of root cause fields of the first type of error reporting key points manually, map the first type of error reporting key points to the corresponding first type of root cause fields to form a first mapping set, and determine that the first mapping set is the root cause set.

[0167] Optionally, the root cause set formation unit includes:

[0168] A second classification subunit, configured to extract the second type of error key points belonging to the second classification from all the error key points;

[0169] An encoding subunit, configured to collect the second type of root cause fields of the second type of error key points manually, and use the raise statement to encode the second type of error key points;

[0170] A second set determination subunit, configured to map the encoding result and the corresponding second type of root cause fields to form a second mapping set, and determine the second mapping set as the root cause set.

[0171] Optionally, the root cause set formation unit includes:

[0172] A third classification subunit, configured to extract the third type of error key points belonging to the third classification from all the error key points;

[0173] A semantic analysis subunit, configured to collect the third type of root cause fields of the third type of error key points manually, and perform semantic analysis on the third type of error key points;

[0174] A third set determination subunit, configured to map the semantic analysis result and the corresponding third type of root cause fields to form a third mapping set, and determine the third mapping set as the root cause set.

[0175] Optionally, the root cause analysis module 42 includes:

[0176] A second information extraction subunit, configured to extract the second error information of each test report in the second test report set;

[0177] A second key point determination subunit, configured to denoise the second error information and determine the denoising result as the error key point.

[0178] It should be noted that for the information interaction, execution process, etc. among the above modules, since they are based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details are not described herein again.

[0179] Figure 5 This is a schematic structural diagram of a computer device provided in Embodiment 5 of the present application. As Figure 5 shown, the computer device of this embodiment includes: at least one processor ( Figure 5 only one is shown in

[0180] ), a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, it implements the steps in any of the above method embodiments of the artificial intelligence-based test report analysis method. Figure 5The above are merely examples of computer devices and do not constitute a limitation on computer devices. A computer device may include more or fewer components than shown in the figure, or combine certain components, or have different components. For example, it may also include a network interface, a display screen, an input device, etc.

[0181] The so-called processor may be a CPU, or it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc.

[0182] The memory includes a readable storage medium, internal memory, etc. Among them, the internal memory may be the memory of the computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be the hard disk of the computer device, or in other embodiments, it may also be an external storage device of the computer device. For example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the memory may also include both the internal storage unit of the computer device and the external storage device. The memory is used to store the operating system, application programs, boot loaders, data, and other programs, such as the program code of computer programs. The memory may also be used to temporarily store data that has been output or is to be output.

[0183] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above device can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0184] All or part of the processes in the above method embodiments of this application can also be completed by a computer program product. When the computer program product runs on a computer device, the computer device can be made to execute the steps in the above method embodiments.

[0185] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0186] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0187] In the embodiments provided in this application, it should be understood that the disclosed device / computer equipment and method can be implemented in other ways. For example, the device / computer equipment embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0188] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0189] The above embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. An artificial intelligence-based test report analysis method, characterized in that The described test report analysis method includes: Obtaining a root cause set and a training set constructed based on a first test report set, where the training set includes at least one training data vector, and the training data vector is formed by a test report with a labeled root cause field; Obtaining a second test report set, and extracting the error key points of each test report in the second test report set, matching each error key point with the root cause set to obtain the root cause field corresponding to the error key point; Using the root cause field as the annotation of the corresponding test report, generating a test data vector based on the annotated test report and the corresponding error key point to obtain a test set; Performing K-nearest neighbor similarity analysis on the training set and the test set, and determining that the K value corresponding to the maximum similarity is the target number; Extracting the error key points of the test report to be analyzed, and determining that the root causes that match the target number from the root cause set are the analysis results of the test report to be analyzed; The root cause set and the training set constructed based on the first test report set specifically include: Obtaining a first test report set, and extracting the error key points of each test report in the first test report set; Collecting the root cause fields of all error key points manually, and mapping all the root cause fields to the corresponding error key points to form a root cause set; Using the root cause field as the annotation of the corresponding test report to obtain an annotated test report, vectorizing the annotated test report and the corresponding error key point, and determining that the vectorization result is a training data vector in the training set; The extraction of the error key points of each test report in the first test report set includes: Extracting the first error information of each test report in the first test report set; Denosing the first error information to determine that the denoising result is the error key point.

2. The test report analysis method according to claim 1, wherein The collection of the root cause fields of all error key points manually, and the mapping of all the root cause fields to the corresponding error key points to form a root cause set includes: Extracting the first type of error key points that are of the first classification among all error key points; Collecting the first type of root cause fields of the first type of error key points manually, mapping the first type of error key points to the corresponding first type of root cause fields to form a first mapping set, and determining that the first mapping set is the root cause set.

3. The test report analysis method according to claim 1, wherein The collection of the root cause fields of all error key points manually, and the mapping of all the root cause fields to the corresponding error key points to form a root cause set includes: Extracting the second type of error key points that are of the second classification among all error key points; Collecting the second type of root cause fields of the second type of error key points manually, encoding the second type of error key points using the raise statement, mapping the encoding result to the corresponding second type of root cause fields to form a second mapping set, and determining that the second mapping set is the root cause set.

4. The test report analysis method according to claim 1, characterized in that, The collection of the root cause fields of all error key points manually, and the mapping of all the root cause fields to the corresponding error key points to form a root cause set includes: Extracting the third type of error key points that are of the third classification among all error key points; Collecting the third type of root cause fields of the third type of error key points manually, and performing semantic analysis on the third type of error key points; Map the semantic analysis results to the corresponding third - type root cause fields to form a third mapping set, and determine the third mapping set as the root cause set.

5. The method for analyzing a test report according to any one of claims 1 to 4, characterized in that, The extraction of the error - reporting key points of each test report in the second test report set includes: Extract the second error - reporting information of each test report in the second test report set; Reduce the noise of the second error - reporting information, and determine the noise - reduction result as the error - reporting key point.

6. An artificial intelligence-based test report analysis device, characterized in that, The test report analysis device includes: A benchmark acquisition module, configured to acquire a root cause set and a training set constructed based on a first test report set. The training set includes at least one training data vector, and the training data vector is formed by a test report with labeled root cause fields; A root cause analysis module, configured to acquire a second test report set, extract the error - reporting key points of each test report in the second test report set, and match each error - reporting key point with the root cause set to obtain the root cause field corresponding to the error - reporting key point; A test set generation module, configured to use the root cause field as the annotation of the corresponding test report, generate a test data vector based on the annotated test report and the corresponding error - reporting key point, and obtain a test set; A similarity analysis module, configured to perform K - nearest neighbor similarity analysis on the training set and the test set, and determine the K value corresponding to the maximum similarity as the target number; An analysis result determination module, configured to extract the error - reporting key points of the test report to be analyzed, and determine the root causes that match the target number from the root cause set as the analysis result of the test report to be analyzed; The benchmark acquisition module includes: A key point extraction unit, configured to acquire a first test report set and extract the error - reporting key points of each test report in the first test report set; A root cause set formation unit, configured to collect the root cause fields of all error - reporting key points manually, and map all the root cause fields to the corresponding error - reporting key points to form a root cause set; A training set formation unit, configured to use the root cause field as the annotation of the corresponding test report to obtain an annotated test report, vectorize the annotated test report and the corresponding error - reporting key point, and determine the vectorization result as a training data vector in the training set; The key point extraction unit includes: A first information extraction subunit, configured to extract the first error - reporting information of each test report in the first test report set; A first key point determination subunit, configured to reduce the noise of the first error - reporting information and determine the noise - reduction result as the error - reporting key point.

7. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the test report analysis method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the test report analysis method according to any one of claims 1 to 5.

Images