Test result analysis method, device, equipment and storage medium

Abstract

The present application discloses a test result analysis method, apparatus, equipment and storage medium, relating to the field of software testing technology. The test result analysis method comprises: determining unit test data based on use case feature information and target test results; inputting the unit test data into a target language generation model to obtain a target test analysis result. The initial language generation model is trained with a data training set and domain-related corpus to obtain a target language generation model, and then the target test result is split and combined with the use case feature information to obtain unit test data. The unit test data is then input into the target language generation model to generate a target test analysis result, thereby realizing automatic generation of problem descriptions, reducing the workload of manual analysis, and significantly improving test efficiency.

Description

Technical Field

[0001] The present application relates to the field of software testing technology, and in particular to a test result analysis method, apparatus, device, and storage medium. Background Art

[0002] During the software testing process, analyzing test results and generating problems are key steps in ensuring software quality. However, most current test analysis tools are only capable of analyzing performance tests, focusing primarily on calculating and comparing performance indicators. They are unable to meet the analytical needs of functional testing, which requires complex mappings between test cases and function definitions. Furthermore, the generated analysis results are primarily based on indicators or simple comparison data, and cannot directly form easy-to-understand and actionable problem descriptions that meet engineers' needs. Analyzing test results based on fixed rules or preset standard values ​​lacks the ability to dynamically understand test context and complex function definitions, making it difficult to cope with diverse test scenarios. Therefore, solving the problem of test analysis tools only being able to record test steps and results, and failing to intelligently analyze failed items and generate solutions that engineers can directly use, has become an urgent issue. Summary of the Invention

[0003] The main purpose of this application is to provide a test result analysis method, device, equipment and storage medium, aiming to solve the technical problem that the test analysis tool can only record the test steps and results, but cannot perform intelligent analysis on the failed items and generate problem descriptions that engineers can directly use.

[0004] To achieve the above objectives, the present application proposes a test result analysis method, which includes:

[0005] Determine unit test data based on use case feature information and target test results;

[0006] The unit test data is input into the target language generation model to obtain the target test analysis result.

[0007] In one embodiment, before the step of inputting the unit test data into the target language generation model to obtain the target test analysis result, the method further includes:

[0008] Pre-train the initial language generation model based on domain-related corpus to obtain a pre-trained initial language generation model;

[0009] The pre-trained initial language generation model is optimized according to the data training set to obtain the target language generation model.

[0010] In one embodiment, the step of optimizing the pre-trained initial language generation model based on the data training set to obtain the target language generation model includes:

[0011] Optimize the pre-trained initial language generation model based on the data training set to determine the model optimization status;

[0012] When the model optimization state is an optimization completion state, obtaining an optimized language generation model;

[0013] The optimized language generation model is evaluated based on the data validation set to obtain a target language generation model.

[0014] In one embodiment, before the step of optimizing the pre-trained initial language generation model according to the data training set to obtain the target language generation model, the method further includes:

[0015] Obtain historical test data and defect description information corresponding to historical software defect data;

[0016] Preprocessing the historical test data and the defect description information to obtain preprocessed data;

[0017] Performing data expansion on the preprocessed data according to a data enhancement strategy and a semantic similarity strategy to obtain target expanded data;

[0018] A corresponding data validation set and data training set are determined according to the target extended data.

[0019] In one embodiment, the step of performing data expansion on the preprocessed data according to the data enhancement strategy and the semantic similarity strategy to obtain target expanded data includes:

[0020] Performing data labeling on the preprocessed data to obtain target labeled data;

[0021] Determining target enhancement data according to the data enhancement strategy and the target annotation data;

[0022] Target extended data is determined according to a semantic similarity strategy and the target enhanced data.

[0023] In one embodiment, the step of determining unit test data based on use case feature information and target test results includes:

[0024] Split the target test results according to the use case feature information to obtain target split data;

[0025] The target split data is screened according to a preset screening strategy to obtain unit test data.

[0026] In one embodiment, the step of inputting the unit test data into the target language generation model to obtain target test analysis results includes:

[0027] Inputting the unit test data into the target language generation model to obtain target description information;

[0028] Determine a corresponding defect data list according to the target description information;

[0029] Generate target test analysis results based on the defect data list.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a test result analysis device, which includes:

[0031] A processing module is used to determine unit test data based on use case feature information and target test results;

[0032] The analysis module is used to input the unit test data into the target language generation model to obtain the target test analysis results.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a test result analysis device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the test result analysis method described above.

[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. When the computer program is executed by the processor, the steps of the test result analysis method described above are implemented.

[0035] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the test result analysis method described above are implemented.

[0036] This application determines unit test data based on use case feature information and target test results; inputs the unit test data into the target language generation model to obtain target test analysis results. The initial language generation model is trained with a data training set and domain-related corpus to obtain a target language generation model. The target test results are then split and combined with the use case feature information to obtain unit test data. The unit test data is then input into the target language generation model to generate the target test analysis results. This achieves automatic generation of problem descriptions, reduces manual analysis workload, and significantly improves test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 A flow chart of the first embodiment of the test result analysis method of this application is provided;

[0040] Figure 2 This is a schematic diagram of the overall process provided in Example 1 of the test result analysis method of this application;

[0041] Figure 3 A flow chart illustrating the second embodiment of the test result analysis method of this application;

[0042] Figure 4 This is a schematic diagram of the module structure of the test result analysis device according to an embodiment of the present application;

[0043] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the test result analysis method in the embodiment of the present application.

[0044] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0046] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0047] The main solution of the embodiment of the present application is: obtain the target language generation model based on the data training set, domain-related corpus and initial language generation model; determine the unit test data based on the use case feature information and the target test results; input the unit test data into the target language generation model to obtain the target test analysis results.

[0048] During the software testing process, analyzing test results and generating problems are key steps in ensuring software quality. However, most current test analysis tools are only capable of analyzing performance tests, focusing primarily on calculating and comparing performance indicators. They are unable to meet the analytical needs of functional testing, which requires complex mappings between test cases and function definitions. Furthermore, the generated analysis results are primarily based on indicators or simple comparison data, and cannot directly form easy-to-understand and actionable problem descriptions that meet engineers' needs. Analyzing test results based on fixed rules or preset standard values ​​lacks the ability to dynamically understand test context and complex function definitions, making it difficult to cope with diverse test scenarios. Therefore, solving the problem of test analysis tools only being able to record test steps and results, and failing to intelligently analyze failed items and generate solutions that engineers can directly use, has become an urgent issue.

[0049] This application determines unit test data based on use case feature information and target test results; inputs the unit test data into the target language generation model to obtain target test analysis results. The initial language generation model is trained with a data training set and domain-related corpus to obtain a target language generation model. The target test results are then split and combined with the use case feature information to obtain unit test data. The unit test data is then input into the target language generation model to generate the target test analysis results. This achieves automatic generation of problem descriptions, reduces manual analysis workload, and significantly improves test efficiency.

[0050] It should be noted that the execution entity of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or a test result analysis device capable of performing the above functions. The following describes this embodiment and the following embodiments using the test result analysis device as the execution entity.

[0051] Based on this, the present invention provides a method for analyzing test results. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the test result analysis method of this application.

[0052] In this embodiment, the test result analysis method includes steps S10 to S20:

[0053] Step S10, determining unit test data according to the use case feature information and the target test result;

[0054] It should be noted that this embodiment realizes automatic analysis of test results by combining the powerful semantic understanding capabilities of the large language model and the design of an automated tool chain, achieving significant advantages in terms of scope of application, analysis efficiency, and intelligent capabilities. It can effectively make up for the shortcomings of existing test analysis tools and provide strong support for test automation analysis.

[0055] It can be understood that use case feature information refers to the key attributes that can uniquely identify and describe a test case, such as the test case identifier, test case name, and test type, etc. The target test result refers to the software test result to be analyzed, and the unit test data refers to the input data, expected output, and related environment configuration information used in the unit testing process.

[0056] In the specific implementation, the software test results to be analyzed are split according to the key attributes that can uniquely identify and describe a test case, and then the input data, expected output, and related environment configuration information used in the unit test process, namely the unit test data, are determined.

[0057] In a feasible implementation, step S10 may include steps A11 to A12:

[0058] Step A11, splitting the target test result data according to the use case feature information to obtain target split data;

[0059] It is understandable that the target split data refers to the data after the data unit is split. According to the key attributes that can uniquely identify and describe a test case, the micro-units of the software test results to be analyzed are identified to determine the target split data.

[0060] Step A12: screening the target split data according to a preset screening strategy to obtain unit test data.

[0061] It is understandable that the preset screening strategy refers to a pre-set strategy for screening failed item data.

[0062] In a specific implementation, the failed item data in the data after the data unit is split is screened according to a pre-set strategy for screening failed item data, thereby obtaining unit test data.

[0063] Step S20: input the unit test data into the target language generation model to obtain target test analysis results.

[0064] It can be understood that the target test analysis results refer to the analysis results of the software test results, including preconditions, trigger conditions, expected actions and actual actions.

[0065] In the specific implementation, the unit test data is submitted to a trained large model with strong natural language generation capabilities for processing, and then the analysis results of the software test results including preconditions, trigger conditions, expected actions and actual actions are obtained, namely the target test analysis results.

[0066] In a feasible implementation, step S20 may include steps A21 to A23:

[0067] Step A21, inputting the unit test data into the target language generation model to obtain target description information;

[0068] It can be understood that the target description information refers to the description of software quality problems. The unit test data is submitted to the trained large model with strong natural language generation capabilities for processing to obtain the software quality problem description output by the large model, namely the target description information.

[0069] Step A22, determining a corresponding defect data list according to the target description information;

[0070] It can be understood that the defect data list refers to the data list composed of software quality problem descriptions. The software quality problem descriptions output by the large model are analyzed, and then the software quality problem descriptions output by the large model are divided, and the corresponding data list, namely the defect data list, is generated.

[0071] Step A23: Generate target test analysis results based on the defect data list.

[0072] In a specific implementation, the data list consisting of the description of software quality issues is analyzed, and analysis results of software test results including preconditions, trigger conditions, expected actions and actual actions are generated, namely target test analysis results.

[0073] It should be noted that if Figure 2 As shown, this embodiment is through S1. Data collection and preprocessing. S1.1 Original data collection: Extract test data and problem descriptions corresponding to software quality problems from the software BUG management system. The extracted data include but are not limited to: exception logs, error codes, test case results, stack information, user feedback, etc. Select data with complete test data and clear problem descriptions for subsequent processing. S1.2 Training set preparation: Clean the original data, including removing invalid information, duplicate data, format errors, etc. According to the training requirements of the large model, unify the data format, structure the test data, and generate standardized problem descriptions. S1.3 Data annotation: Manually annotate the cleaned data set to clarify the correlation between the test data and the problem description, including the mapping relationship between key elements in the test data (such as error triggering conditions, scope of impact) and the description. S1.4 Data expansion: Use data enhancement techniques, such as synonym replacement, semantic expansion, etc. to enrich the diversity of problem descriptions, and use semantic similarity technology to ensure the consistency of the expanded data with the original corpus.

[0074] S2 Large model training. S2.1 Model selection and initialization: Select a large model with strong natural language generation capabilities. Load domain-related corpus (such as software test documents, technical blogs, etc.) on the basic model for pre-training to enhance the model's ability to understand test data. S2.2 Model fine-tuning: Use the cleaned and expanded data set to fine-tune the pre-trained model. Focus on optimizing the model's ability to extract and convert the semantics of test data to ensure the accuracy and readability of the generated problem description. S2.3 Performance verification: Use the validation set to evaluate the model performance, indicators include description accuracy, language fluency, semantic completeness, etc.

[0075] S3.1 Data Unit Splitting: Use the test case's characteristic information to identify micro-units and split the data in the test results. S3.2 Problem Data Identification: Data containing Failure items may have software quality issues. Based on this characteristic, filter out the unit set for further processing.

[0076] S4. Problem description generation. Traverse the test data unit set obtained in S3.2 and perform the following operations: S4.1 Call the large model for processing: Submit the unit test data to the large model for processing. S4.2 The large model returns the problem description: After the large model completes the processing, it outputs the software quality problem description.

[0077] S5: Generate a problem list. Collect the problem descriptions output in S4.2 and generate a software quality problem list.

[0078] This embodiment obtains a target language generation model based on a data training set, domain-related corpus, and an initial language generation model; determines unit test data based on use case feature information and target test results; and inputs the unit test data into the target language generation model to obtain target test analysis results. The initial language generation model is trained using a data training set and domain-related corpus to obtain a target language generation model. The target test results are then split and combined with use case feature information to obtain unit test data. The unit test data is then input into the target language generation model to generate target test analysis results. This achieves automatic generation of problem descriptions, reduces manual analysis workload, and significantly improves test efficiency.

[0079] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 3 Before step S20, the test result analysis method further includes steps S21 to S22:

[0080] Step S21, pre-training the initial language generation model based on domain-related corpus to obtain a pre-trained initial language generation model;

[0081] It can be understood that domain-related corpus refers to a collection of representative and authoritative text materials in a specific professional field, and the initial language generation model refers to a large model with strong natural language generation capabilities.

[0082] In the specific implementation, domain-related corpus (such as software testing documents, technical blogs, etc.) is loaded on the basic model to pre-train a large model with strong natural language generation capabilities, enhance the model's ability to understand test data, and then obtain the pre-trained initial language generation model.

[0083] Step S22: Optimize the pre-trained initial language generation model according to the data training set to obtain a target language generation model.

[0084] It can be understood that the target language generation model refers to a large model with strong natural language generation capabilities after training.

[0085] In practice, we fine-tune the pre-trained initial language generation model using the cleaned and expanded dataset. We focus on optimizing the model's ability to extract and transform the semantics of the test data, ensuring the accuracy and readability of the generated problem descriptions. Finally, we obtain the target language generation model.

[0086] In a feasible implementation, step S22 may include steps A221 to A223:

[0087] Step A221, optimizing the pre-trained initial language generation model based on the data training set to determine the model optimization state;

[0088] It can be understood that the model optimization state includes an optimization completion state and an optimization failure state.

[0089] In a specific implementation, the pre-trained initial language generation model is optimized using the cleaned and expanded data set to determine the optimization state of the pre-trained initial language generation model, that is, to determine the model optimization state.

[0090] Step A222: when the model optimization state is the optimization completion state, obtaining an optimized language generation model;

[0091] It can be understood that the optimized language generation model refers to the optimized pre-trained large model. When the model optimization state is the optimization completion state, it indicates that the optimization of the pre-trained initial language generation model has been completed, and the optimized pre-trained large model, that is, the optimized language generation model, is obtained.

[0092] Step A223 , evaluating the optimized language generation model based on the data validation set to obtain a target language generation model.

[0093] In specific implementations, the optimized pre-trained large model is evaluated based on the dataset used for large model validation. That is, the model performance is evaluated using the validation set, with indicators including description accuracy, language fluency, and semantic completeness. Once the evaluation passes, the target language generation model is obtained.

[0094] In a feasible implementation manner, steps B221 to B224 may be further included before step S22:

[0095] Step B221, obtaining historical test data and defect description information corresponding to historical software defect data;

[0096] It should be noted that historical software defect data refers to data with software quality problems extracted from the software BUG management system, historical test data refers to software test data corresponding to software quality problems, and defect description information refers to problem description data corresponding to software quality problems.

[0097] In practice, test data and problem descriptions corresponding to software quality issues are extracted from the software bug management system. Extracted data includes, but is not limited to, exception logs, error codes, test case results, stack traces, and user feedback. Data with complete test data and clear problem descriptions is carefully selected for subsequent processing.

[0098] Step B222: preprocessing the historical test data and the defect description information to obtain preprocessed data;

[0099] It can be understood that preprocessed data refers to software test data and problem description data after data cleaning.

[0100] In the specific implementation, the raw data is cleaned, including removing invalid information, duplicate data, format errors, etc. According to the training requirements of the large model, the data format is unified, the test data is structured, and a standardized problem description is generated to obtain the preprocessed data.

[0101] Step B223, performing data expansion on the pre-processed data according to the data enhancement strategy and the semantic similarity strategy to obtain target expanded data;

[0102] It can be understood that the data enhancement strategy refers to a pre-set strategy for data expansion, the semantic similarity strategy refers to a pre-set strategy for ensuring that the expanded data is consistent with the original corpus, and the target expanded data refers to the pre-processed data after data expansion.

[0103] In the specific implementation, data enhancement techniques such as synonym replacement and semantic expansion are used to enrich the diversity of problem descriptions, and semantic similarity technology is used to ensure the consistency of the expanded data with the original corpus to determine the preprocessed data after data expansion, that is, the target expanded data.

[0104] In a feasible implementation, step B223 may include steps C2231 to C2233:

[0105] Step C2231, performing data labeling on the pre-processed data to obtain target labeled data;

[0106] It is understandable that the target labeled data refers to the pre-processed data after labeling.

[0107] In the specific implementation, the raw data is cleaned, including removing invalid information, duplicate data, format errors, etc. According to the training requirements of the large model, the data format is unified, the test data is structured, and a standardized problem description is generated to obtain the target labeled data.

[0108] Step C2232: determining target enhancement data according to the data enhancement strategy and the target annotation data;

[0109] It can be understood that the target enhanced data refers to the labeled data after data enhancement.

[0110] In a specific implementation, data expansion is performed on the annotated preprocessed data according to a pre-set strategy for data expansion, i.e. synonym replacement, semantic expansion, etc., to obtain data-enhanced annotated data, i.e. target enhanced data.

[0111] Step C2233: Determine target extended data according to the semantic similarity strategy and the target enhanced data.

[0112] It is understandable that the target extended data is obtained by processing the annotated data after data enhancement according to a pre-set strategy for ensuring that the extended data is consistent with the original corpus.

[0113] Step B224: determining the corresponding data validation set and data training set according to the target extended data.

[0114] It can be understood that the data training set refers to the data set used for large model training, and the data validation set refers to the data set used for large model validation.

[0115] In a specific implementation, the pre-processed data after data expansion is divided into data sets, thereby obtaining data sets for model training and model validation, namely, a data validation set and a data training set, respectively.

[0116] It should be noted that the large model training in this embodiment is specifically as follows: Model selection and initialization: Select a large model with strong natural language generation capabilities. Load domain-related corpus (such as software test documents, technical blogs, etc.) on the basic model for pre-training to enhance the model's ability to understand test data; Model fine-tuning: Use the cleaned and expanded data set to fine-tune the pre-trained model. Focus on optimizing the model's ability to extract and convert the semantics of test data to ensure the accuracy and readability of the generated problem description; Performance verification: Use the verification set to evaluate the model performance, and the indicators include description accuracy, language fluency, semantic completeness, etc.

[0117] This embodiment pre-trains an initial language generation model based on domain-related corpus to obtain a pre-trained initial language generation model; then optimizes the pre-trained initial language generation model based on a data training set to obtain a target language generation model. By training and optimizing the initial language generation model using domain-related corpus and a data training set, respectively, the target language generation model is obtained. This leverages the semantic understanding capabilities of a large model to dynamically analyze complex function definitions and test reports, accurately attribute failures, and generate context-sensitive problem descriptions, significantly improving the intelligence of the analysis.

[0118] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the test result analysis method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0119] This application also provides a test result analysis device, please refer to Figure 4 , the test result analysis device comprises:

[0120] Processing module 10, used to determine unit test data based on use case feature information and target test results;

[0121] The analysis module 20 is used to input the unit test data into the target language generation model to obtain target test analysis results.

[0122] Optionally, the analysis module 20 is further configured to:

[0123] Pre-train the initial language generation model based on domain-related corpus to obtain a pre-trained initial language generation model;

[0124] The pre-trained initial language generation model is optimized according to the data training set to obtain the target language generation model.

[0125] Optionally, the analysis module 20 is further configured to:

[0126] Optimize the pre-trained initial language generation model based on the data training set to determine the model optimization status;

[0127] When the model optimization state is an optimization completion state, obtaining an optimized language generation model;

[0128] The optimized language generation model is evaluated based on the data validation set to obtain a target language generation model.

[0129] Optionally, the analysis module 20 is further configured to:

[0130] Obtain historical test data and defect description information corresponding to historical software defect data;

[0131] Preprocessing the historical test data and the defect description information to obtain preprocessed data;

[0132] Performing data expansion on the preprocessed data according to a data enhancement strategy and a semantic similarity strategy to obtain target expanded data;

[0133] A corresponding data validation set and data training set are determined according to the target extended data.

[0134] Optionally, the analysis module 20 is further configured to:

[0135] Performing data labeling on the preprocessed data to obtain target labeled data;

[0136] Determining target enhancement data according to the data enhancement strategy and the target annotation data;

[0137] Target extended data is determined according to a semantic similarity strategy and the target enhanced data.

[0138] Optionally, the processing module 10 is further configured to:

[0139] Split the target test results according to the use case feature information to obtain target split data;

[0140] The target split data is screened according to a preset screening strategy to obtain unit test data.

[0141] Optionally, the analysis module 20 is further configured to:

[0142] Inputting the unit test data into the target language generation model to obtain target description information;

[0143] Determine a corresponding defect data list according to the target description information;

[0144] Generate target test analysis results based on the defect data list.

[0145] The test result analysis device provided in this application, which utilizes the test result analysis method of the aforementioned embodiment, can resolve the technical problem that test analysis tools can only record test steps and results, but are unable to intelligently analyze failed items and generate problem descriptions that engineers can directly use. Compared with the prior art, the beneficial effects of the test result analysis device provided in this application are the same as those of the test result analysis method provided in the aforementioned embodiment, and the other technical features of the test result analysis device are the same as those disclosed in the aforementioned embodiment method, and are not further described here.

[0146] The present application provides a test result analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the test result analysis method in the above-mentioned embodiment 1.

[0147] Reference below Figure 5 , which shows a schematic diagram of the structure of a test result analysis device suitable for implementing the embodiments of the present application. The test result analysis device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The test result analysis device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0148] like Figure 5As shown, the test result analysis device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the test result analysis device. Processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and communication device 1009. The communication device 1009 can allow the test result analysis device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a test result analysis device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems can be implemented or provided instead.

[0149] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0150] The test result analysis device provided in this application, which utilizes the test result analysis method of the aforementioned embodiment, can resolve the technical problem that test analysis tools can only record test steps and results, but are unable to intelligently analyze failed items and generate problem descriptions that engineers can directly use. Compared with the prior art, the beneficial effects of the test result analysis device provided in this application are the same as those of the test result analysis method provided in the aforementioned embodiment, and the other technical features of the test result analysis device are the same as those disclosed in the method of the previous embodiment, and are not further described here.

[0151] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0152] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0153] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the test result analysis method in the above-mentioned embodiment.

[0154] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0155] The computer-readable storage medium may be included in the test result analysis device, or may exist independently without being assembled into the test result analysis device.

[0156] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the test result analysis device, the test result analysis device: determines the unit test data based on the use case feature information and the target test result; inputs the unit test data into the target language generation model to obtain the target test analysis result.

[0157] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0158] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0159] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0160] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., a computer program) for executing the above-mentioned test result analysis method. This can solve the technical problem that test analysis tools can only record test steps and results, but cannot perform intelligent analysis of failed items and generate problem descriptions that engineers can directly use. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the test result analysis method provided in the above-mentioned embodiment, and will not be repeated here.

[0161] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned test result analysis method when executed by a processor.

[0162] The computer program product provided in this application addresses the technical issue that test analysis tools can only record test steps and results, but are unable to intelligently analyze failures and generate problem descriptions that engineers can directly use. Compared to the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the test result analysis methods provided in the aforementioned embodiments, and are not further elaborated here.

[0163] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A test result analysis method, characterized in that: The test result analysis method includes: Determine unit test data based on use case feature information and target test results; Input the unit test data into the target language generation model to obtain target test analysis results; The step of determining unit test data based on use case feature information and target test results includes: The target test result is split according to the use case feature information to obtain target split data, specifically: the target split data is determined by identifying micro-units of the software test result to be analyzed according to the key attributes that can uniquely identify and describe a test case; Filter the target split data according to a preset filtering strategy to obtain unit test data; The step of inputting the unit test data into the target language generation model to obtain the target test analysis result includes: Inputting the unit test data into the target language generation model to obtain target description information; Determine a corresponding defect data list according to the target description information; Generate target test analysis results based on the defect data list.

2. The method according to claim 1, wherein Before the step of inputting the unit test data into the target language generation model to obtain the target test analysis result, the method further includes: Pre-train the initial language generation model based on domain-related corpus to obtain a pre-trained initial language generation model; The pre-trained initial language generation model is optimized according to the data training set to obtain the target language generation model.

3. The method according to claim 2, wherein The step of optimizing the pre-trained initial language generation model according to the data training set to obtain the target language generation model includes: Optimize the pre-trained initial language generation model based on the data training set to determine the model optimization status; When the model optimization state is an optimization completion state, obtaining an optimized language generation model; The optimized language generation model is evaluated based on the data validation set to obtain a target language generation model.

4. The method according to claim 2, wherein Before the step of optimizing the pre-trained initial language generation model according to the data training set to obtain the target language generation model, the method further includes: Obtain historical test data and defect description information corresponding to historical software defect data; Preprocessing the historical test data and the defect description information to obtain preprocessed data; Performing data expansion on the preprocessed data according to a data enhancement strategy and a semantic similarity strategy to obtain target expanded data; A corresponding data validation set and data training set are determined according to the target extended data.

5. The method according to claim 4, wherein The step of performing data expansion on the preprocessed data according to the data enhancement strategy and the semantic similarity strategy to obtain target expanded data includes: Performing data labeling on the preprocessed data to obtain target labeled data; Determining target enhancement data according to the data enhancement strategy and the target annotation data; Target extended data is determined according to a semantic similarity strategy and the target enhanced data.

6. A test result analysis device, characterized in that: The device comprises: A processing module is used to determine unit test data based on use case feature information and target test results; An analysis module, configured to input the unit test data into a target language generation model to obtain target test analysis results; The processing module is further configured to perform data splitting on the target test result according to the use case feature information to obtain target split data, specifically by identifying micro-units of the software test result to be analyzed based on key attributes that can uniquely identify and describe a test case to determine the target split data; Filter the target split data according to a preset filtering strategy to obtain unit test data; The analysis module is further configured to input the unit test data into the target language generation model to obtain target description information; determine a corresponding defect data list based on the target description information; and generate a target test analysis result based on the defect data list.

7. A test result analysis device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the test result analysis method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the test result analysis method according to any one of claims 1 to 5 are implemented.

Images