Test result analysis method and device, equipment and storage medium

Abstract

The invention discloses a test result analysis method and device, equipment and a storage medium, and relates to the technical field of software testing, and the test result analysis method comprises the following steps: determining unit test data according to use case feature information and a target test result; and inputting the unit test data to a target language generation model to obtain a target test analysis result. Training the initial language generation model through the data training set and the domain-related corpus to obtain a target language generation model, splitting the target test result, obtaining unit test data in combination with the case feature information, inputting the unit test data into the target language generation model to generate a target test analysis result, and analyzing the target test analysis result. The problem description is automatically generated, the workload of manual analysis is reduced, and the test efficiency is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of software testing technology, and in particular to a test result analysis method, device, equipment and storage medium. Background Art

[0002] In the process of software testing, the analysis of test results and the generation of problems are key links in ensuring software quality. However, most current test analysis tools can only analyze performance tests, focusing mainly on the calculation and comparison of performance indicators, and cannot meet the analysis requirements of the complex mapping relationship between test cases and function definitions in functional testing; and the generated analysis results are mainly based on indicators or simple comparison data, which cannot directly form a problem description that meets the needs of engineers and is easy to understand and operate; based on fixed rules or preset standard values ​​to analyze test results, lack the ability to dynamically understand test contexts and complex function definitions, it is difficult to cope with diverse test scenarios. Therefore, how to solve the problem that test analysis tools can only record test steps and results, and cannot perform intelligent analysis of failed items and generate engineers that can be directly used has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of this application is to provide a test result analysis method, device, equipment and storage medium, aiming to solve the technical problem that the test analysis tool can only record the test steps and results, but cannot perform intelligent analysis on the failed items and generate problem descriptions that can be directly used by engineers.

[0004] To achieve the above objectives, the present application proposes a test result analysis method, which includes:

[0005] Determine unit test data based on use case feature information and target test results;

[0006] The unit test data is input into the target language generation model to obtain the target test analysis result.

[0007] In one embodiment, before the step of inputting the unit test data into the target language generation model to obtain the target test analysis result, the step further includes:

[0008] Pre-train the initial language generation model according to domain-related corpus to obtain a pre-trained initial language generation model;

[0009] The pre-trained initial language generation model is optimized according to the data training set to obtain the target language generation model.

[0010] In one embodiment, the step of optimizing the pre-trained initial language generation model according to the data training set to obtain the target language generation model includes:

[0011] Optimize the pre-trained initial language generation model according to the data training set to determine the model optimization status;

[0012] When the model optimization state is an optimization completion state, an optimized language generation model is obtained;

[0013] The optimized language generation model is evaluated according to the data validation set to obtain a target language generation model.

[0014] In one embodiment, before the step of optimizing the pre-trained initial language generation model according to the data training set to obtain the target language generation model, the step further includes:

[0015] Obtain historical test data and defect description information corresponding to historical software defect data;

[0016] Preprocessing the historical test data and the defect description information to obtain preprocessed data;

[0017] Performing data expansion on the preprocessed data according to a data enhancement strategy and a semantic similarity strategy to obtain target expanded data;

[0018] A corresponding data validation set and data training set are determined according to the target extended data.

[0019] In one embodiment, the step of performing data expansion on the preprocessed data according to the data enhancement strategy and the semantic similarity strategy to obtain target expanded data includes:

[0020] Performing data labeling on the preprocessed data to obtain target labeling data;

[0021] Determine target enhancement data according to the data enhancement strategy and the target annotation data;

[0022] The target extended data is determined according to the semantic similarity strategy and the target enhanced data.

[0023] In one embodiment, the step of determining unit test data according to the use case feature information and the target test result includes:

[0024] Split the target test results according to the use case feature information to obtain target split data;

[0025] The target split data is screened according to a preset screening strategy to obtain unit test data.

[0026] In one embodiment, the step of inputting the unit test data into the target language generation model to obtain the target test analysis result includes:

[0027] Inputting the unit test data into the target language generation model to obtain target description information;

[0028] Determine a corresponding defect data list according to the target description information;

[0029] A target test analysis result is generated according to the defect data list.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a test result analysis device, the test result analysis device comprising:

[0031] A processing module, used for determining unit test data according to the use case feature information and the target test result;

[0032] The analysis module is used to input the unit test data into the target language generation model to obtain the target test analysis result.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a test result analysis device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the test result analysis method described above.

[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the test result analysis method described above are implemented.

[0035] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the test result analysis method described above are implemented.

[0036] This application determines unit test data according to use case feature information and target test results; inputs the unit test data into the target language generation model to obtain the target test analysis results. The initial language generation model is trained with a data training set and domain-related corpus to obtain a target language generation model, and then the target test results are split and combined with the use case feature information to obtain unit test data, and then the unit test data is input into the target language generation model to generate the target test analysis results, thereby realizing the automatic generation of problem descriptions, reducing the workload of manual analysis, and significantly improving test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0039] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the test result analysis method of the present application;

[0040] Figure 2 The overall flow diagram provided for the first embodiment of the test result analysis method of the present application;

[0041] Figure 3 A schematic diagram of a flow chart provided for the second embodiment of the test result analysis method of the present application;

[0042] Figure 4 This is a schematic diagram of the module structure of the test result analysis device according to an embodiment of the present application;

[0043] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the test result analysis method in the embodiment of the present application.

[0044] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0046] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0047] The main solution of the embodiment of the present application is: obtain the target language generation model according to the data training set, domain-related corpus and the initial language generation model; determine the unit test data according to the use case feature information and the target test results; input the unit test data into the target language generation model to obtain the target test analysis results.

[0048] In the process of software testing, the analysis of test results and the generation of problems are key links in ensuring software quality. However, most current test analysis tools can only analyze performance tests, focusing mainly on the calculation and comparison of performance indicators, and cannot meet the analysis requirements of the complex mapping relationship between test cases and function definitions in functional testing; and the generated analysis results are mainly based on indicators or simple comparison data, which cannot directly form a problem description that meets the needs of engineers and is easy to understand and operate; based on fixed rules or preset standard values ​​to analyze test results, lack the ability to dynamically understand test contexts and complex function definitions, it is difficult to cope with diverse test scenarios. Therefore, how to solve the problem that test analysis tools can only record test steps and results, and cannot perform intelligent analysis of failed items and generate engineers that can be directly used has become an urgent problem to be solved.

[0049] This application determines unit test data according to use case feature information and target test results; inputs the unit test data into the target language generation model to obtain the target test analysis results. The initial language generation model is trained with a data training set and domain-related corpus to obtain a target language generation model, and then the target test results are split and combined with the use case feature information to obtain unit test data, and then the unit test data is input into the target language generation model to generate the target test analysis results, thereby realizing the automatic generation of problem descriptions, reducing the workload of manual analysis, and significantly improving test efficiency.

[0050] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a test result analysis device capable of realizing the above functions, etc. The following takes the test result analysis device as the execution subject as an example to illustrate this embodiment and the following embodiments.

[0051] Based on this, the present application embodiment provides a test result analysis method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the test result analysis method of the present application.

[0052] In this embodiment, the test result analysis method includes steps S10 to S20:

[0053] Step S10, determining unit test data according to the use case feature information and the target test result;

[0054] It should be noted that this embodiment realizes automatic analysis of test results by combining the powerful semantic understanding ability of the large language model and the automated tool chain design, and has significant advantages in terms of scope of application, analysis efficiency, and intelligent capabilities. It can effectively make up for the shortcomings of existing test analysis tools and provide strong support for test automation analysis.

[0055] It can be understood that use case feature information refers to the key attributes that can uniquely identify and describe a test case, such as the test case identifier, test case name, and test type, etc. The target test result refers to the software test result to be analyzed, and the unit test data refers to the input data, expected output, and related environment configuration information used in the unit testing process.

[0056] In the specific implementation, the software test results to be analyzed are split according to the key attributes that can uniquely identify and describe a test case, and then the input data, expected output, and related environment configuration information used in the unit test process, namely, the unit test data, are determined.

[0057] In a feasible implementation, step S10 may include steps A11 to A12:

[0058] Step A11, splitting the target test result data according to the use case feature information to obtain target split data;

[0059] It can be understood that the target split data refers to the data after the data unit is split. According to the key attributes that can uniquely identify and describe a test case, the micro-units of the software test results to be analyzed are identified to determine the target split data.

[0060] Step A12: Screen the target split data according to a preset screening strategy to obtain unit test data.

[0061] It can be understood that the preset screening strategy refers to a preset strategy for screening failed item data.

[0062] In a specific implementation, the failed item data in the data after the data unit is split is screened according to a preset strategy for screening the failed item data, so as to obtain the unit test data.

[0063] Step S20, inputting the unit test data into the target language generation model to obtain a target test analysis result.

[0064] It can be understood that the target test analysis results refer to the analysis results of the software test results, including preconditions, trigger conditions, expected actions and actual actions.

[0065] In the specific implementation, the unit test data is submitted to a trained large model with strong natural language generation capabilities for processing, and then the analysis results of the software test results including preconditions, trigger conditions, expected actions and actual actions are obtained, namely the target test analysis results.

[0066] In a feasible implementation, step S20 may include steps A21 to A23:

[0067] Step A21, inputting the unit test data into the target language generation model to obtain target description information;

[0068] It can be understood that the target description information refers to the description of software quality problems. The unit test data is submitted to the trained large model with strong natural language generation capabilities for processing to obtain the software quality problem description output by the large model, namely the target description information.

[0069] Step A22, determining a corresponding defect data list according to the target description information;

[0070] It can be understood that the defect data list refers to the data list composed of software quality problem descriptions. The software quality problem descriptions output by the big model are analyzed, and then the software quality problem descriptions output by the big model are divided, and the corresponding data list, namely the defect data list, is generated.

[0071] Step A23, generating target test analysis results according to the defect data list.

[0072] In a specific implementation, the data list consisting of the software quality problem description is analyzed, and analysis results of the software test results including preconditions, trigger conditions, expected actions and actual actions, namely, target test analysis results, are generated.

[0073] It should be noted that if Figure 2 As shown, this embodiment is through S1. Data collection and preprocessing. S1.1 Original data collection: Extract the test data and problem description corresponding to the software quality problem from the software BUG management system. The extracted data includes but is not limited to: exception logs, error codes, test case results, stack information, user feedback, etc. Select data with complete test data and clear problem description for subsequent processing. S1.2 Training set preparation: Clean the original data, including removing invalid information, duplicate data, format errors, etc. According to the training requirements of the large model, unify the data format, structure the test data, and generate a standardized problem description. S1.3 Data annotation: Manually annotate the cleaned data set to clarify the correlation between the test data and the problem description, including the mapping relationship between the key elements in the test data (such as error triggering conditions, scope of impact) and the description. S1.4 Data expansion: Use data enhancement techniques, such as synonym replacement, semantic extension, etc. to enrich the diversity of problem descriptions, and use semantic similarity technology to ensure the consistency of the expanded data with the original corpus.

[0074] S2 Large model training. S2.1 Model selection and initialization: Select a large model with strong natural language generation capabilities. Load domain-related corpus (such as software test documents, technical blogs, etc.) on the basic model for pre-training to enhance the model's ability to understand test data. S2.2 Model fine-tuning: Use the cleaned and expanded data set to fine-tune the pre-trained model. Focus on optimizing the model's ability to extract and convert the semantics of test data to ensure the accuracy and readability of the generated problem description. S2.3 Performance verification: Use the validation set to evaluate the model performance, indicators include description accuracy, language fluency, semantic completeness, etc.

[0075] S3.1 Data unit splitting: Use the characteristic information of the test case to identify micro-units and split the data in the test results. S3.2 Problem data identification: Data containing Failure items may have software quality problems. Based on this feature, filter out the unit set to be further processed.

[0076] S4 Problem description generation. Traverse the test data unit set obtained in S3.2 and perform the following operations: S4.1 Call the large model for processing: Submit the unit test data to the large model for processing S4.2 The large model returns the problem description: After the large model processing is completed, the software quality problem description will be output.

[0077] S5: Generate a problem list. Collect the problem descriptions output in S4.2 and generate a software quality problem list.

[0078] This embodiment obtains a target language generation model based on a data training set, domain-related corpus, and an initial language generation model; determines unit test data based on use case feature information and target test results; and inputs the unit test data into the target language generation model to obtain target test analysis results. The initial language generation model is trained with a data training set and domain-related corpus to obtain a target language generation model, and then the target test results are split and combined with use case feature information to obtain unit test data, and then the unit test data is input into the target language generation model to generate target test analysis results, thereby realizing automatic generation of problem descriptions, reducing the workload of manual analysis, and significantly improving test efficiency.

[0079] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 Before step S20, the test result analysis method further includes steps S21 to S22:

[0080] Step S21, pre-training the initial language generation model according to the domain-related corpus to obtain a pre-trained initial language generation model;

[0081] It can be understood that domain-related corpus refers to a collection of representative and authoritative text materials in a specific professional field, and the initial language generation model refers to a large model with strong natural language generation capabilities.

[0082] In the specific implementation, domain-related corpus (such as software testing documents, technical blogs, etc.) is loaded on the basic model to pre-train a large model with strong natural language generation capabilities, enhance the model's ability to understand test data, and then obtain the pre-trained initial language generation model.

[0083] Step S22, optimizing the pre-trained initial language generation model according to the data training set to obtain a target language generation model.

[0084] It can be understood that the target language generation model refers to a large model with strong natural language generation capabilities after training.

[0085] In the specific implementation, the cleaned and expanded dataset is used to fine-tune the pre-trained initial language generation model. The focus is on optimizing the model's ability to extract and convert the semantics of the test data to ensure the accuracy and readability of the generated problem description. Finally, the target language generation model is obtained.

[0086] In a feasible implementation, step S22 may include steps A221 to A223:

[0087] Step A221, optimizing the pre-trained initial language generation model according to the data training set to determine the model optimization state;

[0088] It can be understood that the model optimization state includes an optimization completion state and an optimization failure state.

[0089] In a specific implementation, the pre-trained initial language generation model is optimized using the cleaned and expanded data set to determine the optimization state of the pre-trained initial language generation model, that is, to determine the model optimization state.

[0090] Step A222, when the model optimization state is the optimization completion state, obtaining an optimized language generation model;

[0091] It can be understood that the optimized language generation model refers to the optimized pre-trained large model. When the model optimization state is the optimization completion state, it indicates that the optimization of the pre-trained initial language generation model has been completed, and then the optimized pre-trained large model is obtained, that is, the optimized language generation model.

[0092] Step A223, evaluating the optimized language generation model according to the data verification set to obtain a target language generation model.

[0093] In the specific implementation, the optimized pre-trained large model is evaluated according to the data set used for large model verification, that is, the model performance is evaluated using the verification set, and the indicators include description accuracy, language fluency, semantic completeness, etc. Then, after passing the evaluation, the target language generation model is obtained.

[0094] In a feasible implementation manner, before step S22, steps B221 to B224 may also be included:

[0095] Step B221, obtaining historical test data and defect description information corresponding to historical software defect data;

[0096] It should be noted that historical software defect data refers to data with software quality problems extracted from the software BUG management system, historical test data refers to software test data corresponding to software quality problems, and defect description information refers to problem description data corresponding to software quality problems.

[0097] In the specific implementation, the test data and problem description corresponding to the software quality problem are extracted from the software bug management system. The extracted data includes but is not limited to: exception logs, error codes, test case results, stack information, user feedback, etc. The data with complete test data and clear problem description are selected for subsequent processing.

[0098] Step B222, preprocessing the historical test data and the defect description information to obtain preprocessed data;

[0099] It can be understood that preprocessed data refers to software test data and problem description data after data cleaning.

[0100] In the specific implementation, the original data is cleaned, including removing invalid information, duplicate data, format errors, etc. According to the training requirements of the large model, the data format is unified, the test data is structured, and a standardized problem description is generated to finally obtain the preprocessed data.

[0101] Step B223, performing data expansion on the preprocessed data according to the data enhancement strategy and the semantic similarity strategy to obtain target expanded data;

[0102] It can be understood that the data enhancement strategy refers to a pre-set strategy for data expansion, the semantic similarity strategy refers to a pre-set strategy for ensuring that the expanded data is consistent with the original corpus, and the target expanded data refers to the pre-processed data after data expansion.

[0103] In the specific implementation, data enhancement techniques, such as synonym replacement and semantic expansion, are used to enrich the diversity of problem descriptions, and semantic similarity technology is used to ensure the consistency of the expanded data with the original corpus to determine the preprocessed data after data expansion, that is, the target expanded data.

[0104] In a feasible implementation manner, step B223 may include steps C2231 to C2233:

[0105] Step C2231, performing data labeling on the preprocessed data to obtain target labeling data;

[0106] It can be understood that the target labeled data refers to the pre-processed data after labeling.

[0107] In the specific implementation, the original data is cleaned, including removing invalid information, duplicate data, format errors, etc. According to the training requirements of the large model, the data format is unified, the test data is structured, and a standardized problem description is generated to obtain the target labeled data.

[0108] Step C2232, determining target enhancement data according to the data enhancement strategy and the target annotation data;

[0109] It can be understood that the target enhanced data refers to the labeled data after data enhancement.

[0110] In a specific implementation, data expansion is performed on the annotated preprocessed data according to a pre-set strategy for data expansion, namely synonym replacement, semantic expansion, etc., to obtain data-enhanced annotated data, namely target enhanced data.

[0111] Step C2233, determining the target extended data according to the semantic similarity strategy and the target enhanced data.

[0112] It can be understood that the data augmented labeled data is processed according to a pre-set strategy for ensuring that the expanded data is consistent with the original corpus, thereby obtaining the target expanded data.

[0113] Step B224, determining the corresponding data verification set and data training set according to the target extended data.

[0114] It can be understood that the data training set refers to the data set used for large model training, and the data validation set refers to the data set used for large model validation.

[0115] In a specific implementation, the preprocessed data after data expansion is divided into data sets, thereby obtaining data sets for model training and model validation, namely, a data validation set and a data training set, respectively.

[0116] It should be noted that the large model training in this embodiment is specifically as follows: Model selection and initialization: Select a large model with strong natural language generation capabilities. Load domain-related corpus (such as software test documents, technical blogs, etc.) on the basic model for pre-training to enhance the model's ability to understand test data; Model fine-tuning: Use the cleaned and expanded data set to fine-tune the pre-trained model. Focus on optimizing the model's ability to extract and convert the semantics of test data to ensure the accuracy and readability of the generated problem description; Performance verification: Use the verification set to evaluate the model performance, and the indicators include description accuracy, language fluency, semantic completeness, etc.

[0117] This embodiment obtains a pre-trained initial language generation model by pre-training the initial language generation model according to domain-related corpus; and obtains a target language generation model by optimizing the pre-trained initial language generation model according to the data training set. The target language generation model is obtained by training and optimizing the initial language generation model with domain-related corpus and data training set, that is, by using the semantic understanding ability of the large model, it is possible to dynamically analyze complex function definitions and test reports, accurately attribute failure items and generate context-related problem descriptions, and significantly improve the intelligence of the analysis.

[0118] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the test result analysis method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0119] This application also provides a test result analysis device, please refer to Figure 4 , the test result analysis device comprises:

[0120] The processing module 10 is used to determine the unit test data according to the use case feature information and the target test result;

[0121] The analysis module 20 is used to input the unit test data into the target language generation model to obtain a target test analysis result.

[0122] Optionally, the analysis module 20 is further used for:

[0123] Pre-train the initial language generation model according to domain-related corpus to obtain a pre-trained initial language generation model;

[0124] The pre-trained initial language generation model is optimized according to the data training set to obtain the target language generation model.

[0125] Optionally, the analysis module 20 is further used for:

[0126] Optimize the pre-trained initial language generation model according to the data training set to determine the model optimization status;

[0127] When the model optimization state is an optimization completion state, an optimized language generation model is obtained;

[0128] The optimized language generation model is evaluated according to the data validation set to obtain a target language generation model.

[0129] Optionally, the analysis module 20 is further used for:

[0130] Obtain historical test data and defect description information corresponding to historical software defect data;

[0131] Preprocessing the historical test data and the defect description information to obtain preprocessed data;

[0132] Performing data expansion on the preprocessed data according to a data enhancement strategy and a semantic similarity strategy to obtain target expanded data;

[0133] A corresponding data validation set and data training set are determined according to the target extended data.

[0134] Optionally, the analysis module 20 is further used for:

[0135] Performing data labeling on the preprocessed data to obtain target labeling data;

[0136] Determine target enhancement data according to the data enhancement strategy and the target annotation data;

[0137] The target extended data is determined according to the semantic similarity strategy and the target enhanced data.

[0138] Optionally, the processing module 10 is further used for:

[0139] Split the target test results according to the use case feature information to obtain target split data;

[0140] The target split data is screened according to a preset screening strategy to obtain unit test data.

[0141] Optionally, the analysis module 20 is further used for:

[0142] Inputting the unit test data into the target language generation model to obtain target description information;

[0143] Determine a corresponding defect data list according to the target description information;

[0144] A target test analysis result is generated according to the defect data list.

[0145] The test result analysis device provided by the present application adopts the test result analysis method in the above embodiment, which can solve the technical problem that the test analysis tool can only record the test steps and results, but cannot perform intelligent analysis on the failed items and generate a problem description that can be directly used by engineers. Compared with the prior art, the beneficial effects of the test result analysis device provided by the present application are the same as the beneficial effects of the test result analysis method provided by the above embodiment, and the other technical features in the test result analysis device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0146] The present application provides a test result analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the test result analysis method in the above-mentioned embodiment 1.

[0147] Reference below Figure 5 , which shows a schematic diagram of the structure of a test result analysis device suitable for implementing the embodiment of the present application. The test result analysis device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The test result analysis device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0148] like Figure 5As shown, the test result analysis device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the test result analysis device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the test result analysis device to communicate with other devices wirelessly or wired to exchange data. Although the test result analysis device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0149] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0150] The test result analysis device provided by the present application adopts the test result analysis method in the above embodiment, which can solve the technical problem that the test analysis tool can only record the test steps and results, but cannot perform intelligent analysis on the failed items and generate a problem description that can be directly used by engineers. Compared with the prior art, the beneficial effects of the test result analysis device provided by the present application are the same as the beneficial effects of the test result analysis method provided by the above embodiment, and the other technical features in the test result analysis device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0151] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0152] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0153] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the test result analysis method in the above-mentioned embodiment.

[0154] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0155] The computer-readable storage medium may be included in the test result analysis device; or may exist independently without being assembled into the test result analysis device.

[0156] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the test result analysis device, the test result analysis device: determines the unit test data according to the use case feature information and the target test result; inputs the unit test data into the target language generation model to obtain the target test analysis result.

[0157] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0158] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0159] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0160] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned test result analysis method, and can solve the technical problem that the test analysis tool can only record the test steps and results, but cannot perform intelligent analysis on the failed items and generate problem descriptions that engineers can directly use. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the test result analysis method provided by the above-mentioned embodiment, and will not be repeated here.

[0161] The present application also provides a computer program product, including a computer program, which implements the steps of the test result analysis method as described above when executed by a processor.

[0162] The computer program product provided by the present application can solve the technical problem that the test analysis tool can only record the test steps and results, but cannot intelligently analyze the failed items and generate a problem description that engineers can directly use. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the test result analysis method provided by the above embodiment, and will not be elaborated here.

[0163] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A test result analysis method, characterized in that: The test result analysis method comprises: Determine unit test data based on use case feature information and target test results; The unit test data is input into the target language generation model to obtain the target test analysis result.

2. The method according to claim 1, characterized in that Before the step of inputting the unit test data into the target language generation model to obtain the target test analysis result, the step further includes: Pre-train the initial language generation model according to domain-related corpus to obtain a pre-trained initial language generation model; The pre-trained initial language generation model is optimized according to the data training set to obtain the target language generation model.

3. The method according to claim 2, characterized in that The step of optimizing the pre-trained initial language generation model according to the data training set to obtain the target language generation model includes: Optimize the pre-trained initial language generation model according to the data training set to determine the model optimization status; When the model optimization state is an optimization completion state, an optimized language generation model is obtained; The optimized language generation model is evaluated according to the data validation set to obtain a target language generation model.

4. The method according to claim 2, characterized in that Before the step of optimizing the pre-trained initial language generation model according to the data training set to obtain the target language generation model, the method further includes: Obtain historical test data and defect description information corresponding to historical software defect data; Preprocessing the historical test data and the defect description information to obtain preprocessed data; Performing data expansion on the preprocessed data according to a data enhancement strategy and a semantic similarity strategy to obtain target expanded data; A corresponding data validation set and data training set are determined according to the target extended data.

5. The method according to claim 4, characterized in that The step of performing data expansion on the preprocessed data according to the data enhancement strategy and the semantic similarity strategy to obtain target expanded data includes: Performing data labeling on the preprocessed data to obtain target labeling data; Determine target enhancement data according to the data enhancement strategy and the target annotation data; The target extended data is determined according to the semantic similarity strategy and the target enhanced data.

6. The method according to claim 1, characterized in that The step of determining unit test data according to the use case feature information and the target test result includes: Split the target test results according to the use case feature information to obtain target split data; The target split data is screened according to a preset screening strategy to obtain unit test data.

7. The method according to claim 1, characterized in that The step of inputting the unit test data into the target language generation model to obtain the target test analysis result includes: Inputting the unit test data into the target language generation model to obtain target description information; Determine a corresponding defect data list according to the target description information; A target test analysis result is generated according to the defect data list.

8. A test result analysis device, characterized in that: The device comprises: A processing module, used for determining unit test data according to the use case feature information and the target test result; The analysis module is used to input the unit test data into the target language generation model to obtain the target test analysis result.

9. A test result analysis device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the test result analysis method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the test result analysis method according to any one of claims 1 to 7 are implemented.

Images