A method for evaluating test quality based on AI

Abstract

The present invention discloses a method for implementing quality assessment of test submissions based on AI. The present invention analyzes content points and personnel defect portraits through AI algorithms, and specifies test quality assessment use cases for R&D personnel based on the portrait results, so as to achieve the goal of efficient and accurate evaluation and assessment of test submission quality.

Description

Technical Field

[0001] The present invention relates to the field of software development technology, and in particular to a method for implementing a quality assessment method based on AI. Background Art

[0002] Agile development is based on the evolution of user needs and uses an iterative, cyclical and progressive approach to software development. In agile development, software projects are divided into multiple sub-projects at the beginning of construction. The results of each sub-project are tested and have the characteristics of being visible, integrable and operational. The process in which R&D personnel complete the software development and submit the self-test to the tester for formal testing is called software testing. In the agile development process, R&D personnel's testing is an important stage for the formal start of testing. The quality of the testing will directly affect the efficiency of the testing phase and thus affect the project progress. Better testing quality can achieve twice the result with half the effort in improving testing efficiency and optimizing project progress. Traditional software testing quality assessment is generally conducted by testers after all tests are completed, based on the total number of defects found in the test results, the number of defect levels, the number of level distributions calculated by the degree of impact, and other statistical results. Because it requires manual operation, it is inefficient and difficult to meet the R&D team's demand for efficiency.

[0003] Therefore, the prior art has defects and needs to be improved. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method for evaluating the quality of testing based on AI.

[0005] The technical solution of the present invention is as follows: Provide a method for implementing quality assessment of test results based on AI, comprising the following steps:

[0006] Step 1: After the R&D team completes the software development, they upload the corresponding test materials and send a test request;

[0007] Step 2: After the R&D team sends a test request, it enters the data collection phase, which mainly collects various preparation data needed for intelligent AI;

[0008] Step 3: Based on the AI ​​intelligent algorithm and the prepared data collected in step 2, extract the version content portrait, the defect portrait and weight distribution generated by the R&D personnel in the past, and perform algorithm matching based on these two core tags to calculate the list of use cases in this period and the weights of use cases in this period;

[0009] Step 4: After completing the intelligent assessment analysis in step 3, the assessment report is automatically output.

[0010] Furthermore, the information uploaded and provided in step 1 includes: the version to be tested / current version, the content of the test, the time of the test, and a list of R&D personnel.

[0011] Furthermore, the data collected in step 2 includes: historical version content, historical R&D personnel, historical version defects, historical assessment use cases, current version content and current R&D personnel.

[0012] Furthermore, the assessment report output in step 4 includes: overall assessment pass rate, version defects, assessment use cases and R&D defect distribution data.

[0013] Furthermore, the specific steps of step 3 are:

[0014] Step 3.1: The system extracts the version content profile and the defect profile of the R&D personnel based on the prepared data collected in step 2;

[0015] Step 3.2: Based on the content extracted in step 3.1, perform collaborative filtering to obtain the test cases of this version;

[0016] Step 3.3: Based on historical use cases and historical defect distribution, refine the test cases for this version obtained in step 3.2 to obtain the assessment results for this version.

[0017] Furthermore, the specific steps of extracting the version content portrait and the defect portrait of the R&D personnel in step 3.1 are as follows:

[0018] Content defect profile:

[0019] Step 3.1.a.1: After processing the content using a word segmentation tool, a keyword list for the content point is obtained;

[0020] Step 3.1.a.2: Combine the keywords in the initial content disadvantage keyword list, the historical content disadvantage keyword list, and the custom content disadvantage keyword list to generate a keyword list portrait;

[0021] Defect portrait of R&D personnel:

[0022] Step 3.1.b.1: Based on the development and coding habits of each R&D staff, use the word segmentation tool to process the content of the R&D staff’s information to obtain a keyword list for the content point;

[0023] Step 3.1.b.2: Combine the initialization defect content, historical version defect content, and custom defect content of each developer to create a defect list portrait for the developer.

[0024] Furthermore, in step 3.2, the collaborative filtering adopts the nearest neighbor technology to calculate the distance between content defect labels and the distance between personnel defect labels respectively.

[0025] By adopting the above scheme, the present invention analyzes content points and personnel defect portraits through AI algorithms, and specifies R&D personnel's test quality assessment use cases according to the portrait results, so as to achieve the goal of efficient and accurate evaluation and assessment of test quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a business process diagram of the present invention.

[0027] Figure 2 Schematic diagram of the processing flow of AI intelligent algorithm.

[0028] Figure 3 A schematic diagram of content and personnel defect portraits. DETAILED DESCRIPTION

[0029] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0030] See also Figure 1 The present invention provides a method for implementing a quality assessment of a test based on AI, comprising the following steps:

[0031] Step 1: After the R&D team completes the software development, they upload the corresponding test materials and send a test request. The uploaded materials include: test / current version, test content, test time, and R&D personnel list. Each time a version is tested, the current version, test content, test time, and R&D personnel list are required as the data information of the current version.

[0032] Step 2: After the R&D team issues a test request, it enters the data collection phase, which mainly collects various preparation data needed for intelligent AI. The data collected in step 2 includes: historical version content, historical R&D personnel, historical version defects, historical assessment use cases, current version content, and current R&D personnel.

[0033] Step 3: Please refer to Figure 2 Based on the AI ​​intelligent algorithm and the prepared data collected in step 2, the version content portrait, the defect portrait and weight distribution generated by the R&D personnel in the past are extracted, and the algorithm is matched based on these two core tags to calculate the list of use cases in this period and the weight of use cases in this period. The specific steps are:

[0034] Step 3.1: The system extracts the version content profile and the developer's defect profile based on the prepared data collected in step 2. Figure 3,The specific steps to extract the version content portrait and the defect portrait of the R&D personnel are:

[0035] 3.1.a Content defect portrait: Each version of the content point is generally described by a paragraph of text. After the content is processed using a word segmentation tool (such as the jieba tool), a keyword list for the content point is obtained. Combined with the keywords in the initial content defect keyword list, the historical content defect keyword list, and the custom content defect keyword list, a keyword list portrait is summarized, such as: unable to create, failed to delete, state flow error, editing does not take effect, call failure, and data out of sync.

[0036] Step 3.1.a.1: After processing the content using a word segmentation tool, a keyword list for the content point is obtained;

[0037] Step 3.1.a.2: Combine the keywords in the initial content disadvantage keyword list, the historical content disadvantage keyword list, and the custom content disadvantage keyword list to create a keyword list portrait.

[0038] 3.1.b Defect portrait of R&D personnel: The defect portrait of R&D personnel is similar to the extraction method of content defect portrait. Each R&D personnel has different development and coding habits. Combined with the initialization defect content, historical version defect content and custom defect content of each R&D personnel, the word segmentation tool is also used to filter out the key defect list of each R&D personnel as the defect portrait of the R&D personnel, such as: ignoring interface details, multiple discrepancies with the prototype, incomplete functional implementation, bugs still exist after multiple modifications, unstable functions, and smoke test failures.

[0039] Step 3.1.b.1: Based on the development and coding habits of each R&D staff, use the word segmentation tool to process the content of the R&D staff’s information to obtain a keyword list for the content point;

[0040] Step 3.1.b.2: Combine the initial defect content, historical version defect content and custom defect content of each R&D personnel to comprehensively summarize the defect list portrait of the R&D personnel. Step 3.2: Based on the content extracted in step 3.1, perform collaborative filtering to obtain the test cases of this version; the collaborative filtering in step 3.2 uses the nearest neighbor technology to calculate the distance between content defect labels and the distance between personnel defect labels. After establishing the corresponding association relationship, the defects and use cases corresponding to the two labels that have a direct association with the same label are likely to be adapted to the other label.

[0041] Step 3.3: Based on historical use cases and historical defect distribution, refine the test cases for this version obtained in step 3.2 to obtain the assessment results for this version.

[0042] Step 4: After completing the intelligent assessment analysis in step 3, the assessment report is automatically output. The output assessment report includes: overall comprehensive assessment pass rate, version defects, assessment use cases, and R&D defect distribution data.

[0043] When conducting a test, the current data is uploaded to the system, and the current data is refined and filtered through an intelligent algorithm, and then processed and matched with historical data to obtain the assessment results of this version, thereby achieving the goal of efficient and accurate evaluation and assessment of the quality of the test, and automatically outputting the assessment report for the convenience of R&D personnel and R&D teams. Compared with the problem of inaccurate results caused by human errors in the traditional pure manual statistical process, the present invention abandons the traditional manual assessment method and uses an automated method for assessment. At the same time, based on the results of the AI ​​intelligent algorithm, it automatically identifies the use cases of the current version according to different content points and historical defect data of different R&D personnel.

[0044] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the quality of testing based on AI, characterized in that: The steps include: Step 1: After the R&D team completes the software development, they upload the corresponding test materials and send a test request; Step 2: After the R&D team sends a test request, it enters the data collection phase. The data collection phase mainly collects various preparation data needed for intelligent AI; the collected data includes: historical version content, historical R&D personnel, historical version defects, historical assessment use cases, current version content, and current R&D personnel; Step 3: Based on the AI ​​intelligent algorithm and the prepared data collected in step 2, extract the version content portrait, the developer defect portrait and the weight distribution, and perform algorithm matching based on the two core tags of version content portrait and developer defect portrait to calculate the use case list and the weight of the use case in this period. The specific steps are as follows: Step 3.1: The system extracts the version content portrait and the developer defect portrait based on the prepared data collected in step 2. The specific steps for extracting the version content portrait and the developer defect portrait are as follows: Version content portrait: Step 3.1.a.1: After processing the current version content and the historical version content using a word segmentation tool, a keyword list of the current version content and the historical version content is obtained; Step 3.1.a.2: Combine the keywords in the initial content disadvantage keyword list, the historical content disadvantage keyword list, and the custom content disadvantage keyword list to generate a keyword list portrait; Defect portrait of R&D personnel: Step 3.1.b.1: Based on the development and coding habits of each R&D staff, use the word segmentation tool to process the content of the R&D staff's information to obtain a keyword list of the content; Step 3.1.b.2: Combine the initialization defect content, historical version defect content, and custom defect content of each developer to create a defect profile for the developer; Step 3.2: Based on the version content portrait extracted in step 3.1 and the defect portrait of the R&D personnel, collaborative filtering is performed to obtain the test cases for this version; Step 3.3: According to the historical use cases and historical defect distribution, refine the test cases of this version obtained in step 3.2 to obtain the assessment results of this version; Step 4: After completing the intelligent assessment analysis in step 3, the assessment report is automatically output.

2. The AI-based quality assessment method according to claim 1 is characterized in that: The information uploaded in step 1 includes: the current version, test content, test time and list of R&D personnel.

3. The AI-based quality assessment method according to claim 1 is characterized in that: The assessment report output in step 4 includes: the overall comprehensive pass rate of the assessment, version defects, assessment use cases and R&D defect distribution data.

Images