Error report analysis method, device and equipment for test case

Abstract

The invention provides a test case error analysis method, device and equipment, and relates to the technical field of data processing. The method comprises the following steps: acquiring an analysis request for a target test case sent by front-end equipment, and performing feature extraction on an error report picture to obtain picture features; performing feature extraction on the error report to obtain text features; according to the picture feature and the text feature, determining whether a target historical test case matched with both the picture feature and the text feature exists in a preset case analysis knowledge base; if the target historical test case exists, generating a first error analysis result of the target test case according to error analysis information of the target historical test case; and sending the first error analysis result of the target test case to front-end equipment, so that the front-end equipment displays the first error analysis result of the target test case. According to the method provided by the invention, the intelligence of error analysis is realized, and the analysis efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and equipment for error analysis of test cases. Background Art

[0002] In today's digital age, the scale and complexity of software systems are growing exponentially. Simultaneously, the number of test cases generated by automated testing to ensure software quality is also skyrocketing. Against this backdrop, analyzing automated test error reports presents significant challenges.

[0003] Currently, due to the lack of systematic and structured knowledge base support, existing solutions are unable to quickly call for effective information for reference and comparison when faced with new error problems, resulting in inefficient analysis process and difficulty in ensuring the accuracy of analysis results. Summary of the Invention

[0004] In response to the above-mentioned deficiencies in the prior art, the present application provides a test case error analysis method, device and equipment to solve the problems existing in the prior art.

[0005] The technical solutions adopted in the embodiments of this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a test case error analysis method, comprising:

[0007] Obtain an analysis request for a target test case sent by a front-end device, the analysis request including an error report and an error image of the target test case;

[0008] Performing feature extraction on the error picture to obtain picture features;

[0009] Performing feature extraction on the error report to obtain text features;

[0010] Determining, based on the image features and the text features, whether there is a target historical test case in a preset use case analysis knowledge base that matches both the image features and the text features;

[0011] If the target historical test case exists, generating a first error analysis result of the target test case according to the error analysis information of the target historical test case;

[0012] The first error analysis result of the target test case is sent to the front-end device, so that the front-end device displays the first error analysis result of the target test case.

[0013] Optionally, extracting features from the error image to obtain image features includes:

[0014] Using a preset feature extraction algorithm, extract features from the error image to obtain low-level features; the low-level features include: at least one of color histogram features, texture features, and shape features;

[0015] A preset feature extraction network is used to extract features from the error picture to obtain high-level semantic features; the picture features include: the low-level features and the high-level semantic features.

[0016] Optionally, extracting features from the error report to obtain text features includes:

[0017] Extracting text content from the error report to obtain text content of the error report;

[0018] Keyword extraction is performed on the error report text content to obtain the text features.

[0019] Optionally, the preset use case analysis knowledge base includes: error data tables of multiple historical test cases, and the error data table of each historical test case includes: an image feature table and a text feature table, wherein the image feature table stores: error image features of each historical test case and corresponding error analysis information, and the text feature table stores: error text features of each historical test case and corresponding error analysis information;

[0020] The determining, based on the image features and the text features, whether there is a target historical test case in a preset use case analysis knowledge base that matches both the image features and the text features includes:

[0021] Performing feature matching on the image feature tables of the multiple historical test cases according to the image features to obtain a first image matching result;

[0022] Performing feature matching on the text feature tables of the multiple historical test cases according to the text features to obtain a first text matching result;

[0023] The target historical test case in which both the image feature and the text feature match is determined according to the first image matching result and the first text matching result.

[0024] Optionally, the preset use case analysis knowledge base includes: a business data table, the business data table includes: business question and answer data of multiple historical test cases, and the business question and answer data of each historical test case includes: question text information of the corresponding business, answer information of the question of the corresponding business, and error picture features of the corresponding business;

[0025] The determining, based on the image features and the text features, whether there is a target historical test case in a preset use case analysis knowledge base that matches both the image features and the text features includes:

[0026] Matching error image features in the business question and answer data of the multiple historical test cases according to the image features to obtain a second image matching result;

[0027] Performing feature matching on question text information in the business question and answer data of the multiple historical test cases based on the text features to obtain a second text matching result;

[0028] According to the second image matching result and the second text matching result, the target historical test case in which both the image feature and the text feature match is determined.

[0029] Optionally, the method further includes:

[0030] If the target historical test case does not exist, performing error analysis using a preset large language model based on the text features to obtain a second error analysis result of the target test case;

[0031] The second error analysis result of the target test case is sent to the front-end device, so that the front-end device displays the second error analysis result of the target test case.

[0032] Optionally, the method further includes:

[0033] If a first feedback operation for the first error analysis result returned by the front-end device is received, the preset use case analysis knowledge base is updated according to the first feedback operation.

[0034] Optionally, the method further includes:

[0035] If a second feedback operation for the second error analysis result returned by the front-end device is received, the preset large language model is updated according to the second feedback operation.

[0036] In a second aspect, an embodiment of the present application provides a test case error analysis device, comprising:

[0037] An acquisition module is configured to acquire an analysis request for a target test case sent by a front-end device, wherein the analysis request includes an error report and an error image of the target test case;

[0038] A first extraction module is used to extract features from the error image to obtain image features;

[0039] A second extraction module is used to extract features from the error report to obtain text features;

[0040] A determination module, configured to determine, based on the image features and the text features, whether there is a target historical test case in a preset use case analysis knowledge base that matches both the image features and the text features;

[0041] a generating module configured to generate a first error analysis result of the target test case according to the error analysis information of the target historical test case if the target historical test case exists;

[0042] The sending module is used to send the first error analysis result of the target test case to the front-end device, so that the front-end device displays the first error analysis result of the target test case.

[0043] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to implement the error analysis method of the test case described in any of the above embodiments.

[0044] The beneficial effects of the present application are: the present application provides a test case error analysis method, including: obtaining an analysis request for a target test case sent by a front-end device, the analysis request including an error report and an error picture of the target test case; performing feature extraction on the error picture to obtain image features; performing feature extraction on the error report to obtain text features; determining, based on the image features and text features, whether there is a target historical test case in a preset use case analysis knowledge base that matches both the image features and the text features; if the target historical test case exists, generating a first error analysis result for the target test case based on the error analysis information of the target historical test case; sending the first error analysis result of the target test case to the front-end device, so that the front-end device displays the first error analysis result of the target test case.

[0045] Among them, a preset use case analysis knowledge base is used to automatically generate the first error analysis result of the target test case based on the error analysis information of the target historical test case, which realizes the intelligence of error analysis and improves the efficiency and accuracy of error analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 One of the flow charts of the error analysis method for the test case provided in the embodiment of the present application;

[0048] Figure 2 The second flowchart of the error analysis method for the test case provided in the embodiment of the present application;

[0049] Figure 3 Flowchart 3 of the error analysis method for the test case provided in the embodiment of the present application;

[0050] Figure 4 Flowchart 4 of the error analysis method for the test case provided in the embodiment of the present application;

[0051] Figure 5 Flowchart 5 of the error analysis method for the test case provided in the embodiment of the present application;

[0052] Figure 6 Flowchart 6 of the error analysis method for the test case provided in the embodiment of the present application;

[0053] Figure 7 A schematic diagram of the structure of the error analysis device for the test case provided in the embodiment of the present application;

[0054] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0056] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0057] In addition, the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, rather than to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. For those skilled in the art, the specific meanings of the above-mentioned terms in this application can be understood according to the specific circumstances.

[0058] With the increasing complexity of software systems and the rapid growth in the number of automated test cases, the number of error reports generated during the testing process has also increased exponentially. Traditional manual analysis methods rely on testers to manually review logs, stack trace information, and other relevant data. This is not only inefficient but also prone to misjudgments or omissions due to human factors, affecting the accuracy of problem location and the efficiency of repair.

[0059] At present, automated test error analysis mainly relies on simple error information capture tools. Although these tools can collect and display error data in real time, they still rely heavily on manual intervention in error cause analysis, classification, and location. Existing technologies lack intelligent analysis methods, resulting in a large amount of effective information not being efficiently utilized.

[0060] In addition, although existing automated testing error analysis tools have the ability to capture error information in real time, they still rely heavily on manual intervention in the in-depth analysis and precise location of error causes, making it difficult to achieve intelligent error analysis and unable to meet the needs of increasingly complex software testing scenarios.

[0061] Therefore, this application provides a test case error analysis method to achieve intelligent error classification, root cause analysis, and solution recommendation, thereby significantly improving analysis efficiency and accuracy, reducing manual reliance, and optimizing the software testing process. This method can be generated by any electronic device with computing and processing capabilities, such as a back-end server.

[0062] It should be noted that, in the absence of conflict, the features in the embodiments of the present application can be combined with each other. The following describes the error analysis method for the test case provided by the present application in detail through multiple examples with reference to the accompanying drawings.

[0063] Figure 1 One of the flow charts of the error analysis method for the test case provided in the embodiment of the present application is as follows: Figure 1 As shown, the method includes:

[0064] S101: Obtain an analysis request for a target test case sent by a front-end device.

[0065] First, an analysis request for the target test case sent by the front-end device is obtained, where the front-end device can be, for example, a device that executes the test case. When a test case fails, the failed test case needs to be analyzed. Therefore, the test case is used as the target test case and is automatically regarded as receiving an analysis request for the target test case to carry out automatic error analysis for the target test case.

[0066] Among them, the analysis request may include the error report and error picture of the target test case. The error report is a structured error description document that may include the identification information of the test case (name, version information, etc.), the execution environment (front-end device type, operating system and version), and error details (error type, error level, error description, error time, etc.); the error picture is a visual error evidence, such as a complete screenshot when the error occurs, or a screen recording showing the dynamic process of the error occurring.

[0067] Optionally, the user can also upload an analysis request for the target test case. Specifically, a service request entry page is constructed using HyperText Markup Language (HTML) and JS programming language, providing a user interaction interface for users to upload error reports and error images and submit analysis requests.

[0068] Among them, the user interaction interface adopts a responsive design and can adapt to different screen sizes and devices. The user interaction interface includes a file upload area, a submit button and a result display area. The file upload area can accept files in various formats, such as text files such as .txt, .pdf, .doc, and image files such as .jpg, .png. The file upload area supports dragging and dropping to upload files, and multiple files can be uploaded at the same time; the submit button is used to trigger an analysis request for the target test case. When the user clicks the submit button, it is deemed to have triggered an analysis request for the target test case. The submit button style can adopt a gradient color design to enhance visual appeal; the result display area is used to display the analysis results. The result display area uses a folding panel, which is initially folded and clicks to expand to display the analysis results.

[0069] The front-end device uses JavaScript to obtain the file selected by the user, converts the file data into a Base64-encoded string, packages it together with other necessary information such as the transaction code into JSON format data, and sends it to the back-end server so that the back-end server can perform error analysis on the target historical test case.

[0070] S102: Extract features from the error image to obtain image features.

[0071] Feature extraction of error images refers to the process of automatically identifying and extracting elements or attributes from error images that represent the essential characteristics and key information of the image. These features can cover multiple aspects, such as texture features in the image, such as whether the background texture around the error text is rough, smooth, or has a specific pattern; color features, such as the color distribution of the error area, whether bright red highlights the error, or other color combinations convey the error information; shape features, such as the shape of the error prompt box (rectangular, circular, etc.) and the shape of the error icon. These extracted image features play a key role in subsequent analysis, classification, recognition, and other tasks, such as determining the type of error and tracing the source of the error.

[0072] S103: Extract features from the error report to obtain text features.

[0073] Feature extraction of error reports refers to extracting valuable and representative information from error reports, such as specific error codes. Specific error codes can often be directly associated with specific error types or functional modules in the system. For example, in programming languages, different abnormal errors will have corresponding error code identifiers; error prompt information, the error prompt text in the error report, can provide an intuitive description of the cause of the error, such as "file does not exist" and "memory overflow", etc. These text contents are important components of extracting text features; related operation steps, recording the operation steps performed by the user or system before the error occurs, helps to reproduce the error scenario, and is one of the key features for determining the source of the error; system environment information, including operating system version, software version, hardware configuration, etc., this information can be used to determine whether the error is related to specific environmental factors.

[0074] By extracting features from error reports, these text features can be subsequently used for a variety of purposes, such as error classification and clustering to more efficiently manage and handle different types of errors, or using these features to achieve automatic diagnosis and prediction of errors.

[0075] S104. Determine, based on the image features and the text features, whether there is a target historical test case in the preset use case analysis knowledge base that matches both the image features and the text features.

[0076] The preset use case analysis knowledge base is a pre-built knowledge base used to store error analysis reports of historical error test cases, including text information such as the error type, cause, solution, and related error image feature information of the historical test cases. Multiple tables are set up in the preset use case analysis knowledge base, which are used to store information such as error type, cause, solution, and image features.

[0077] The preset use case analysis knowledge base also establishes an indexing mechanism to improve the efficiency of knowledge matching. For text information tables, a GIN index is established to accelerate full-text search. For image feature tables, a GiST index is established to accelerate high-dimensional vector similarity queries.

[0078] After obtaining the image features of the error image and the text features of the error report, we match these features with information in the pre-set case analysis knowledge base to identify possible causes of the error. For example, we can calculate the text similarity between historical cases in the knowledge base and the text features, as well as the image feature distance between historical cases and the image features. Combining the text similarity and image feature distance, we select the case with the highest overall match as the target historical test case.

[0079] S105 : If the target historical test case exists, generate a first error analysis result of the target test case according to the error analysis information of the target historical test case.

[0080] Then, based on the error analysis information of the target historical test case, a first error analysis result of the target test case is generated. For example, the error analysis information of the target historical test case is sorted and summarized, and adjusted according to the actual situation of the target test case as the first error analysis result of the target test case, or the error analysis information of the target historical test case is directly used as the first error analysis result of the target test case.

[0081] S106: Send the first error analysis result of the target test case to the front-end device, so that the front-end device displays the first error analysis result of the target test case.

[0082] After obtaining the first error analysis result, the first error analysis result of the target test case is sent to the front-end device, so that the front-end device displays the first error analysis result of the target test case, realizing instant visual feedback. The user can intuitively understand the error details without viewing the log file, and the analysis results are directly displayed on the front-end interface, saving the time of manually reviewing the report. The analysis results are displayed on the front-end interface and can also be directly associated with the original error scenario for display, which is convenient for users to debug and shortens the debugging cycle.

[0083] In summary, this embodiment provides a test case error analysis method, which can be applied to scenarios such as error diagnosis, remote debugging and problem troubleshooting of automated testing platforms, and exception reporting of quality monitoring systems. This method adopts a preset use case analysis knowledge base to automatically generate the first error analysis result of the target test case based on the error analysis information of the target historical test case, thereby realizing the intelligence of error analysis and improving the efficiency and accuracy of error analysis.

[0084] Figure 2 The second flow chart of the error analysis method for the test case provided in the embodiment of the present application is as follows: Figure 2 As shown, in one embodiment, the feature extraction of the error image in S102 to obtain the image features may include:

[0085] S201: Using a preset feature extraction algorithm, extract features from the error image to obtain low-level features.

[0086] The preset feature extraction algorithm can be, for example, an algorithm corresponding to the OpenCV image processing library or the Pillow image processing library. First, the OpenCV image processing library or the Pillow image processing library is used to preprocess the error image, including but not limited to resizing, contrast enhancement, denoising, grayscale, binarization and other operations. Then, based on the preprocessed image, at least one low-level feature of the image's color histogram features, texture features, and shape features is extracted.

[0087] S202: Using a preset feature extraction network, extract features from the error image to obtain high-level semantic features.

[0088] The preset feature extraction network, for example, can be a pre-trained VGG convolutional neural network model. This neural network model is used to extract features from the error image to obtain high-level semantic features. The resulting image features include color histogram features, texture features, shape features, and semantic features. These features are then similarly matched with features stored in the preset use case analysis knowledge base to determine the cause of the error.

[0089] Figure 3 The third flow chart of the error analysis method for the test case provided in the embodiment of the present application is as follows: Figure 3 As shown, in one embodiment, the feature extraction of the error report in S103 to obtain text features may include:

[0090] S301: Extract text content from the error report to obtain the text content of the error report.

[0091] Error reports can exist in various forms, such as log files, pop-up prompt messages, error record documents in a specific format, etc. Extracting the text content from error reports means separating the text information contained in these different forms of error message carriers.

[0092] For example, if the error message is recorded in a log file, appropriate text processing tools or programming code need to be used to read and extract from the log file the text description related to the error, error code, time when the error occurred, modules or functions involved, etc. Eventually, a set containing the complete text content of the error report is obtained, providing basic data for subsequent analysis and processing.

[0093] S302. Extract keywords from the text content of the error report to obtain text features.

[0094] After obtaining the text content of the error report, preprocess the text content. The preprocessing can include operations such as word segmentation, stop word removal, and词性标注 (positional tagging). Among them, word segmentation is to perform refined processing on the text content, successively carry out word segmentation operations to split continuous text into independent lexical units; stop word removal is to remove stop words such as "的", "了", "在" that have no actual semantic meaning; positional tagging is to assign a tag representing its grammatical category (positional) to each word, such as noun, verb, adjective, adverb, etc.

[0095] Then perform keyword extraction. Keywords are words or phrases that can accurately summarize and represent the main content and key information of the text. Through keyword extraction technology, the most representative and important words can be screened out from a large amount of error text.

[0096] Keyword extraction methods can be statistical methods (such as word frequency statistics, TF-IDF algorithm, etc.), natural language processing-based methods (such as positional tagging, named entity recognition, etc.). For example, in error text, phrases such as "database connection failed", "memory overflow", "function call error" may appear. Through keyword extraction technology, "database connection", "memory overflow", "function call", etc. can be extracted as keywords. These keywords constitute the features of the error report text. They can reflect important information such as the type of error, system components or operations involved, which helps to further analyze, classify and locate the cause of the error, as well as cluster and compare similar error situations.

[0097] It should be noted that the term "词性标注" in the original text is not a standard English expression. I have translated it as "positional tagging" for the purpose of maintaining the context, but it may be more accurately expressed as "词性 tagging" or a more specific term in the field of natural language processing. You can adjust it according to the actual situation.In one embodiment, the pre-set test case analysis knowledge base may include error data tables for multiple historical test cases. Before performing feature matching, two basic data tables must be established: an image feature table and a text feature table for historical test cases. The data stored in these tables is extracted from past test cases, with each historical test case corresponding to a set of image feature data and text feature data.

[0098] That is, the error data table of each historical test case includes an image feature table and a text feature table. The image feature table stores the error image features of each historical test case and the corresponding error analysis information. The text feature table stores the error text features of each historical test case and the corresponding error analysis information.

[0099] In this case, determining whether there is a target historical test case that matches both the image features and the text features in the preset use case analysis knowledge base according to the image features and the text features in S104 may include:

[0100] S401: Perform feature matching on image feature tables of multiple historical test cases based on image features to obtain a first image matching result.

[0101] The image features of the error image are compared with the feature vectors in the historical test case image feature table. Common matching algorithms include cosine similarity calculation and Euclidean distance measurement. By calculating the similarity score, historical test cases with scores above the preset threshold are screened out to form the first image matching result. For example, historical test cases with a similarity score of 0.8 or above are considered to match the image to be tested in terms of image features and are included in the first image matching result.

[0102] S402: Perform feature matching on text feature tables of multiple historical test cases based on text features to obtain a first text matching result.

[0103] The text features of the error report are compared to the feature vectors in the historical test case text feature table for similarity. Similarly, matching historical test cases are selected based on a preset threshold to obtain a first text matching result. For example, if the text features of the error report have a high semantic similarity to the text in the historical test case, the corresponding historical test case will be selected into the first text matching result.

[0104] S403: Determine a target historical test case whose image features and text features match according to the first image matching result and the first text matching result.

[0105] Cross-compare the first image matching result and the first text matching result to screen out historical test cases that match both image features and text features. These cases are the target historical test cases. For example, when testing an app with image and text editing functions, only historical test cases that meet both the image editing operation features and the corresponding function description text features will be identified as the target test cases that best meet the current test requirements. These target historical test cases can provide a reference for the current test, helping testers to design test plans more comprehensively and predict possible problems, thereby improving the quality and efficiency of the test.

[0106] In another embodiment, the preset use case analysis knowledge base may include a business data table, which includes business question and answer data of multiple historical test cases. The business question and answer data of each historical test case includes question text information of the corresponding business, question answer information of the corresponding business, and error picture features of the corresponding business.

[0107] In this case, determining whether there is a target historical test case that matches both the image features and the text features in the preset use case analysis knowledge base according to the image features and the text features in S104 may include:

[0108] S501. Match error image features in business question and answer data of multiple historical test cases according to image features to obtain a second image matching result.

[0109] The image features of the error image are compared and analyzed with those of the error images in the business Q&A data. Common matching algorithms, such as cosine similarity and Euclidean distance, are used to quantify the similarity between the two. A reasonable similarity threshold is set, and historical test cases with scores above this threshold are filtered out to form the second image matching result set. For example, if the similarity score reaches 0.75 or above, the corresponding test case will be included in the second image matching result set. These test cases have a high degree of similarity in image features with the error image.

[0110] S502: Perform feature matching on question text information in the business question and answer data of multiple historical test cases based on text features to obtain a second text matching result.

[0111] The similarity between the text features of the error text and the question text information of the business question and answer data is calculated, and the matching historical test cases are filtered out according to the preset threshold to obtain the question answer information of the business corresponding to the question text information, and the obtained question answer information is used as the second text matching result.

[0112] S503: Determine a target historical test case where both the image features and the text features match according to the second image matching result and the second text matching result.

[0113] The second image matching result and the second text matching result are cross-compared to screen out historical test cases that match both image features and text features. These cases are the target historical test cases. For example, when testing a medical business Q&A platform, only historical test cases that simultaneously meet the requirements of abnormal error image display interface elements and the problem text description matches the abnormal function usage in the historical test cases will be identified as the target test cases that best meet the current test requirements. These target historical test cases can provide testers with a wealth of references to help them deeply analyze the root causes of the problems, develop comprehensive test plans, and predict possible problems in advance, thereby significantly improving the testing quality and efficiency of the business Q&A system and optimizing system performance.

[0114] Figure 6 The sixth flow chart of the error analysis method for the test case provided in the embodiment of the present application is as follows: Figure 6 As shown, the method of the present application also includes:

[0115] S601: If the target historical test case does not exist, perform error analysis using a preset large language model based on text features to obtain a second error analysis result of the target test case.

[0116] The pre-trained large language model uses a Transformer-based language model architecture, employing a self-attention mechanism to capture long-range dependencies in text. The model's output layer uses a softmax classifier to output a probability distribution of error causes. Alternatively, the model can employ a Transformer-based encoder-decoder architecture, employing a multi-head attention mechanism to capture long-range dependencies in text. Furthermore, the model is pre-trained using test case error analysis reports and fine-tuned using annotated historical error analysis report data to improve the model's ability to analyze test case errors.

[0117] If S104 determines that there is no target historical test case in the preset use case analysis knowledge base that matches both the image features and the text features based on the image features and the text features, the text features are preprocessed and input into a preset large language model for error analysis. The model outputs a probability distribution of the error causes, and the result with the highest probability is selected as the second error analysis result for the target test case. The preprocessing of the text features may include, for example, word segmentation, stop word removal, and part-of-speech tagging.

[0118] S602: Send the second error analysis result of the target test case to the front-end device, so that the front-end device displays the second error analysis result of the target test case.

[0119] The second error analysis result obtained by the preset large language model is returned to the front-end device in JSON format. The second error analysis result includes information such as a description of the error cause and possible solutions. The front-end device can then display the analysis result in the form of a card in the result display area.

[0120] In this embodiment, the preset large language model makes up for the limitations of the preset use case analysis knowledge base. When there are no historical test cases that match both the current image features and text features in the preset knowledge base, the large language model can obtain a second error analysis result, avoiding interruption of the analysis process due to insufficient knowledge base coverage, and improving the coverage and automation level of error analysis.

[0121] In one embodiment, the method of the present application further includes: if a first feedback operation is received from the front-end device regarding the first error analysis result, updating the preset use case analysis knowledge base according to the first feedback operation. The first feedback operation is a feedback operation performed by a user regarding the first error analysis result. For example, if the user believes that the accuracy of the first error analysis result does not meet the requirements, the user performs a corresponding "update knowledge base" operation on the front-end device, regards this operation as the first feedback operation, and updates the preset use case analysis knowledge base in response to this operation.

[0122] Similarly, if a second feedback operation regarding the second error analysis result is received from the front-end device, the preset large language model is updated according to the second feedback operation. For example, the parameters of the preset large language model can be adjusted to make the output result of the preset large language model more accurate.

[0123] In summary, the present invention provides a method for analyzing test case errors, which has the following advantages:

[0124] 1. It greatly improves the efficiency of test case error analysis, reduces the time cost of manual analysis, and effectively solves the problems of low efficiency and low accuracy of manual analysis in existing technologies.

[0125] 2. By combining historical error cases in the preset use case analysis knowledge base with the natural language processing capabilities of the large language model, the accuracy of error cause analysis is improved. In addition, comprehensive analysis can be performed in conjunction with error images, further improving analysis accuracy.

[0126] The following continues to explain the apparatus, device and storage medium for executing the error analysis method of the test case provided in any of the above embodiments of the present application. Its specific implementation process and the technical effects produced are the same as those of the corresponding method embodiments mentioned above. For the sake of brief description, for the parts not mentioned in the following embodiments, please refer to the corresponding content in the method embodiments.

[0127] Figure 7A schematic diagram of the structure of the error analysis device for the test case provided in the embodiment of the present application is shown as follows: Figure 7 As shown, the present application also provides a test case error analysis device, including:

[0128] The acquisition module 10 is used to acquire an analysis request for a target test case sent by a front-end device, where the analysis request includes an error report and an error picture of the target test case.

[0129] The first extraction module 20 is used to extract features from the error-reporting image to obtain image features.

[0130] The second extraction module 30 is used to extract features from the error report to obtain text features.

[0131] The determination module 40 is used to determine whether there is a target historical test case that matches both the image features and the text features in the preset use case analysis knowledge base based on the image features and the text features.

[0132] The generating module 50 is configured to generate a first error analysis result of the target test case according to the error analysis information of the target historical test case if the target historical test case exists.

[0133] The sending module 60 is configured to send the first error analysis result of the target test case to the front-end device, so that the front-end device displays the first error analysis result of the target test case.

[0134] Optionally, the first extraction module 20 is also used to use a preset feature extraction algorithm to perform feature extraction on the error picture to obtain low-level features; the low-level features include: at least one feature of color histogram features, texture features, and shape features; use a preset feature extraction network to perform feature extraction on the error picture to obtain high-level semantic features; the picture features include: low-level features and high-level semantic features.

[0135] Optionally, the second extraction module 30 is further configured to extract text content from the error report to obtain text content of the error report; and extract keywords from the text content of the error report to obtain text features.

[0136] Optionally, the preset use case analysis knowledge base includes: error data tables of multiple historical test cases, and the error data table of each historical test case includes: an image feature table, and a text feature table, wherein the image feature table stores: the error image features of each historical test case, and corresponding error analysis information, and the text feature table stores: the error text features of each historical test case, and corresponding error analysis information; the determination module 40 is also used to perform feature matching on the image feature tables of multiple historical test cases based on the image features to obtain a first image matching result; perform feature matching on the text feature tables of multiple historical test cases based on the text features to obtain a first text matching result; and determine the target historical test case whose image features and text features match based on the first image matching result and the first text matching result.

[0137] Optionally, the preset use case analysis knowledge base includes: a business data table, the business data table includes: business question and answer data of multiple historical test cases, the business question and answer data of each historical test case includes: question text information of the corresponding business, question answer information of the corresponding business, and error picture features of the corresponding business; the determination module 40 is also used to match the error picture features in the business question and answer data of multiple historical test cases according to the picture features to obtain a second picture matching result; perform feature matching on the question text information in the business question and answer data of multiple historical test cases according to the text features to obtain a second text matching result; determine the target historical test case in which both the picture features and the text features match according to the second picture matching result and the second text matching result.

[0138] Optionally, the generation module 50 is further configured to, if the target historical test case does not exist, perform error analysis using a preset large language model based on text features to obtain a second error analysis result of the target test case.

[0139] The sending module 60 is further configured to send the second error analysis result of the target test case to the front-end device, so that the front-end device displays the second error analysis result of the target test case.

[0140] Optionally, the apparatus of the present application further includes an updating module for updating a preset use case analysis knowledge base according to the first feedback operation if a first feedback operation for a first error analysis result is received from a front-end device.

[0141] Optionally, the updating module is further configured to update the preset large language model according to the second feedback operation if a second feedback operation for the second error analysis result returned by the front-end device is received.

[0142] The above-mentioned device is used to execute the method provided in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0143] The above modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0144] In one embodiment, if Figure 8 As shown, the present application also provides an electronic device, including a processor 100, a storage medium 200 and a bus 300, wherein the storage medium stores program instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the program instructions to implement the error analysis method of the test case described in any of the above embodiments.

[0145] Optionally, the present application further provides a readable storage medium having program instructions stored thereon, and the program instructions, when executed by a processor, implement the error analysis method for the test case described in any of the above embodiments.

[0146] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0147] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0148] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0149] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (English: Read-Only Memory, abbreviated: ROM), a random access memory (English: Random Access Memory, abbreviated: RAM), a disk or an optical disk, and other media that can store program code.

[0150] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A test case error analysis method, characterized in that: include: Obtain an analysis request for a target test case sent by a front-end device, the analysis request including an error report and an error image of the target test case; Performing feature extraction on the error picture to obtain picture features; Performing feature extraction on the error report to obtain text features; Determining, based on the image features and the text features, whether there is a target historical test case in a preset use case analysis knowledge base that matches both the image features and the text features; If the target historical test case exists, generating a first error analysis result of the target test case according to the error analysis information of the target historical test case; The first error analysis result of the target test case is sent to the front-end device, so that the front-end device displays the first error analysis result of the target test case.

2. The method according to claim 1, characterized in that The feature extraction of the error picture to obtain picture features includes: Using a preset feature extraction algorithm, extract features from the error image to obtain low-level features; the low-level features include: at least one of color histogram features, texture features, and shape features; A preset feature extraction network is used to extract features from the error picture to obtain high-level semantic features; the picture features include: the low-level features and the high-level semantic features.

3. The method according to claim 1, characterized in that The feature extraction of the error report to obtain text features includes: Extracting text content from the error report to obtain text content of the error report; Keyword extraction is performed on the error report text content to obtain the text features.

4. The method according to claim 1, wherein The preset use case analysis knowledge base includes: error data tables of multiple historical test cases, and the error data table of each historical test case includes: an image feature table and a text feature table, wherein the image feature table stores: error image features of each historical test case and corresponding error analysis information, and the text feature table stores: error text features of each historical test case and corresponding error analysis information; The determining, based on the image features and the text features, whether there is a target historical test case in a preset use case analysis knowledge base that matches both the image features and the text features includes: Performing feature matching on the image feature tables of the multiple historical test cases according to the image features to obtain a first image matching result; Performing feature matching on the text feature tables of the multiple historical test cases according to the text features to obtain a first text matching result; The target historical test case in which both the image feature and the text feature match is determined according to the first image matching result and the first text matching result.

5. The method according to claim 1, characterized in that The preset use case analysis knowledge base includes: a business data table, the business data table includes: business question and answer data of multiple historical test cases, and the business question and answer data of each historical test case includes: question text information of the corresponding business, answer information of the corresponding business question, and error picture features of the corresponding business; The determining, based on the image features and the text features, whether there is a target historical test case in a preset use case analysis knowledge base that matches both the image features and the text features includes: Matching error image features in the business question and answer data of the multiple historical test cases according to the image features to obtain a second image matching result; Performing feature matching on question text information in the business question and answer data of the multiple historical test cases based on the text features to obtain a second text matching result; According to the second image matching result and the second text matching result, the target historical test case in which both the image feature and the text feature match is determined.

6. The method according to claim 1, characterized in that The method further comprises: If the target historical test case does not exist, performing error analysis using a preset large language model based on the text features to obtain a second error analysis result of the target test case; The second error analysis result of the target test case is sent to the front-end device, so that the front-end device displays the second error analysis result of the target test case.

7. The method according to claim 1, characterized in that The method further comprises: If a first feedback operation for the first error analysis result returned by the front-end device is received, the preset use case analysis knowledge base is updated according to the first feedback operation.

8. The method according to claim 6, characterized in that The method further comprises: If a second feedback operation for the second error analysis result returned by the front-end device is received, the preset large language model is updated according to the second feedback operation.

9. A test case error analysis device, characterized in that: include: An acquisition module is configured to acquire an analysis request for a target test case sent by a front-end device, wherein the analysis request includes an error report and an error image of the target test case; A first extraction module is used to extract features from the error image to obtain image features; A second extraction module is used to extract features from the error report to obtain text features; A determination module, configured to determine, based on the image features and the text features, whether there is a target historical test case in a preset use case analysis knowledge base that matches both the image features and the text features; a generating module configured to generate a first error analysis result of the target test case according to the error analysis information of the target historical test case if the target historical test case exists; The sending module is used to send the first error analysis result of the target test case to the front-end device, so that the front-end device displays the first error analysis result of the target test case.

10. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the program instructions to implement the error analysis method of the test case described in any one of claims 1 to 8.

Images