A product performance management method and system based on automation technology

Abstract

The present invention discloses a product performance management method and system based on automation technology. By utilizing computer system integration technology and integrating gitlab, jenkins, and jira tools, the original data of indicators required for product performance are automatically obtained, and the original data of indicators are cleaned by custom cleaning rules, and then the cleaned data is written into a database library cluster. The present invention has formulated a set of performance indicators for measuring products, including product dimension indicators and personal dimension indicators, which are convenient for observing the current performance of each product line. The present invention is based on a data visualization platform for performance indicators, which can display the aggregated data of various indicators in real time. On this basis, any role can timely view the current ranking of their own product performance data, promptly identify problems and solve them in a timely manner.

Description

Technical Field

[0001] The present invention belongs to the field of computer software technology, relates to system integration application technology, and specifically relates to a product performance management method and system based on automation technology. Background Art

[0002] Traditional product performance management primarily relies on various tools to improve product performance at different stages. For example, different systems are used to support development, testing, maintenance, and operations. Current implementations rely on manual data export from each supporting system and subsequent analysis. This approach lacks a comprehensive system that automatically aggregates and perceives data from each supporting system, and it's impossible to visualize and analyze metrics across each stage.

[0003] With the prevalence of agile development, there are now various agile software on the market to control the quality of various links, such as R&D progress, testing progress, and automated operation and maintenance progress. However, none of them can aggregate and analyze the data from each link and display it through a unified visual interface. This results in the inability to promptly control quality issues in each link. For example, in the R&D stage, in project management, developers are only concerned with implementing the functions provided by the product on time. For the developed functions, developers rarely write unit tests for self-testing, resulting in a high number of quality defects in the code submitted for testing. In project management during the testing stage, testers only need to complete the testing of the functions that need to be delivered on time. Often, in order to ensure project progress, a large amount of manpower is endlessly invested to ensure delivery time, but quality cannot be guaranteed. Summary of the Invention

[0004] To solve the above problems, the present invention discloses a product performance management method and system based on automation technology, which enables product managers, developers, testers, and operators to promptly perceive the quality problems of current products and make corresponding adjustments based on actual conditions, thereby achieving the purpose of improving product performance.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A product performance management method based on automation technology includes the following steps:

[0007] Step 1: Define product performance indicators

[0008] Step 2: Design the performance visualization platform table structure

[0009] Using the product and personal indicators provided in step 1, and based on the first normal form theory of relational databases, use database design tools to design a data model for storing performance indicator data;

[0010] Step 3: Integrate GitLab, Jenkins, and Jira through Sprint Boot to obtain data. The specific steps include the following:

[0011] 1. Integrate the / api / json interface of Jenkins to regularly obtain the build information of all modules of the previous day, and count the number of builds and the build success rate based on the module;

[0012] 2. Integrate gitlab's / projects / projectId / repository / commits interface to regularly obtain all commit information submitted to the master branch the previous day, and aggregate it according to the committer's name;

[0013] 3. Write JIRA's built-in search query statements to periodically obtain the number of test cases written and executed by all testers in the previous day;

[0014] 4. Write JIRA's built-in search query statements to periodically obtain all defects found by testers the previous day and the R&D personnel corresponding to the defects;

[0015] Step 4: Clean the raw data information obtained in step 3. The specific cleaning rules are as follows:

[0016] 1. For the full build record information obtained from Jenkins in step 3, classify and count each build by environment type, product type, and module type. Aggregate the build counts for different products, environments, and modules according to the naming convention.

[0017] 2. For the information obtained from GitLab in step 3, only count the code information of the master branch and exclude the merge commit information. Then, classify and summarize the statistics based on the account number carried in the commit;

[0018] 3. Clean out the Chinese names of testers based on email information;

[0019] 4. Define the responsible person and creator fields for defects in JIRA, classify and count the number of defects based on the responsible person and creator, and obtain the number of defects found by testers and the number of defects created by R&D personnel;

[0020] Step 5: Store the cleaned data from step 4 and write the cleaned data from step 4 into the data model defined in step 2 in batches;

[0021] Step 6: Use the performance data visualization platform to display data

[0022] By integrating the open source visualization software platform, write SQL statements to read the data model data in step 2, obtain the various indicator data defined in step 1 and display them.

[0023] Furthermore, the indicator data in step one includes product dimension indicators and personal dimension indicators.

[0024] Furthermore, the product dimension indicators include code development efficiency, defect repair time, test case automation rate, code defect rate, defect leakage rate, and automated build success rate.

[0025] Furthermore, the personal dimension indicators include three sub-dimensional indicators: development engineer, test engineer, and UI engineer; the development engineer indicators include code development efficiency, average defect repair time, and code defect rate; the test engineer indicators include case writing efficiency, case execution efficiency, automated case addition rate, and defect leakage rate; the UI engineer indicators include page development efficiency.

[0026] Furthermore, the data model in step 2 includes a code engineering table, an automated test statistics table, a Jenkins build statistics table, a code quality statistics table, a git group information table, a git code submission quality table, a test case statistics table, a UI statistics table, an employee table, a code submission count statistics table, and a jira issue information table.

[0027] Furthermore, the naming convention in step 4 is environment name-product code-module name.

[0028] The product performance management system based on automation technology is used to implement the product performance management method based on automation technology, including:

[0029] The data layer includes R&D code data, writing case data, execution case data, discovered defect data, manufactured defect data, and data constructed by automated release tools;

[0030] The computing layer includes a data access module and a data cleaning module. The data access module uses the API to obtain data from various tools in the data layer, specifically implementing step three of the product effectiveness management method based on automation technology. The data cleaning module cleans the data obtained by the data access module, specifically implementing step four of the product effectiveness management method based on automation technology.

[0031] The storage layer, including the database cluster module and the data storage service module, is used to store the data obtained by the data cleaning module and specifically implement the content of step 5 of the product performance management method based on automation technology;

[0032] The presentation layer includes a performance data visualization platform, which is used to read data model data and display various indicator data on the visualization page, specifically implementing the content of step six of the product performance management method based on automation technology.

[0033] The beneficial effects of the present invention are:

[0034] 1. The solution of the present invention utilizes computer system integration technology to automatically obtain the original data of indicators required for product performance by integrating gitlab, jenkins, and jira tools, cleans the original data of indicators through customized cleaning rules, and then writes the cleaned data into the database cluster. Through long-term practice, a set of performance indicators for measuring products has been developed, including product dimension indicators and personal dimension indicators, which facilitates the observation of the current performance of each product line. The data visualization platform based on performance indicators can display the aggregated data of various indicators in real time. On this basis, any role can timely check the current ranking of their own product performance data, promptly identify problems and solve them in a timely manner.

[0035] 2. The present invention integrates code quality management tools to promptly detect whether new functions developed by R&D personnel have not written unit tests, and uses thresholds to control whether R&D personnel can submit the code to the code management warehouse.

[0036] 3. The present invention integrates the project management tool (jira) to regularly obtain the number of test cases written by testers, the number of test cases executed, and the number of defects found, and aggregates this data to display the daily work status of each tester through a visual dashboard.

[0037] 4. The present invention provides a SQL-based visualization platform that can display various visualization graphics by simply writing simple SQL statements. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A data model designed for storing performance indicator data in the present invention.

[0039] Figure 2 This is an architectural diagram of the product performance management system based on automation technology provided by the present invention. DETAILED DESCRIPTION

[0040] The technical solutions provided by the present invention will be described in detail below with reference to specific embodiments. It should be understood that the following specific embodiments are only for illustrating the present invention and are not intended to limit the scope of the present invention. In addition, the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0041] The present invention provides a product performance management method based on automation technology, comprising the following steps:

[0042] Step 1: Define the product performance indicators.

[0043] Each indicator data refers to the data information scattered in different tools, which is mainly divided into product dimension indicators and personal dimension indicators. The product dimension indicators are shown in Table 1 (product dimension indicator data) below, including two major categories: efficiency and quality. The efficiency category includes three indicators: code development efficiency, defect repair time, and test case automation rate. The quality category includes three indicators: code defect rate, defect leakage rate, and automated build success rate. Each indicator defines the corresponding formula in Table 1 for calculation.

[0044]

[0045] Table 1

[0046] The personal dimension indicators include three sub-dimension indicators: development engineer, test engineer, and UI engineer.

[0047] The development engineer indicator data is shown in Table 2 below (Personal Dimension - Development Engineer Indicator Data). It includes two categories: efficiency and quality. The efficiency category includes two indicators: code development efficiency and average defect repair time. The quality category includes one indicator: code defect rate. Each indicator is defined in Table 2 for calculation using the corresponding formula.

[0048]

[0049]

[0050] Table 2

[0051] The test engineer indicator data is shown in Table 3 below (Personal Dimension - Test Engineer Indicator Data), which includes two categories: efficiency and quality. The efficiency category includes two indicators: case writing efficiency and case execution efficiency. The quality category includes two indicators: automation case addition rate and defect leakage rate. Each indicator defines a corresponding formula in Table 3 for calculation.

[0052]

[0053] Table 3

[0054] The UI engineer indicator data is shown in Table 4 below (Personal Dimension - UI Engineer Indicator Data), which includes an efficiency category, which includes the indicator of page development efficiency. The calculation formula is defined in Table 4.

[0055] category index Calculation formula efficiency Page development efficiency Pages / month

[0056] Table 4

[0057] Step 2: Design the performance visualization platform table structure.

[0058] This step uses the product indicators and personal indicators provided in step 1, based on the first paradigm theory of relational databases, and uses database design tools to design a data model for storing performance indicator data. The specific data model is as follows: Figure 1 (Product performance indicator data model), including code engineering table, automated test statistics table, Jenkins build statistics table, code quality statistics table, git group information table, git code submission quality table, test case statistics table, UI statistics table, employee table, code submission count statistics table, and Jira issue information table.

[0059] Step 3: Integrate gitlab, jenkins, and jira to obtain data.

[0060] This step integrates Sonar, Jenkins, GitLab, and Jira through Sprint Boot. The specific integration logic is as follows:

[0061] 1. Integrate the / api / json interface of Jenkins to obtain the build information of all modules of the previous day at dawn every day, and count the number of builds and the build success rate based on the modules.

[0062] 2. Integrate gitlab's / projects / projectId / repository / commits interface to obtain all commit information submitted to the master branch the previous day at dawn every day, and aggregate it according to the committer's name.

[0063] 3. Write the search query statement that comes with Jira to obtain all tester information of the previous day, as well as the number of test cases written and executed cases at dawn every day.

[0064] 4. Write the search query statement that comes with Jira to obtain the defects found by all testers the previous day and the R&D personnel corresponding to the defects at dawn every day.

[0065] Step 4: Clean the raw data information obtained in step 3. The specific cleaning rules are as follows:

[0066] 1. Regarding step 1 in step 3, since Jenkins obtains the full amount of build record information, each build needs to be classified and counted according to environment type, product type, and module type. This is because we have established a naming convention for projects in Jenkins: environment name-product code-module name. The number of builds for different products, different environments, and different modules is aggregated according to the naming convention.

[0067] 2. Regarding step 2 in step 3, most companies currently use the feature mode in GitLab for development. However, GitLab's open API requires specifying the branch type. This results in unstandardized branch names during the development process. This is because we only count code information from the master branch and exclude merge commit information. We then categorize and summarize the statistics based on the account number in the commit.

[0068] 3. Regarding step 3, since the tester information obtained in item 3 of the integration step is the email account used to register JIRA, the Chinese name of the tester needs to be cleaned out based on the email information.

[0069] 4. For step 4, the responsible person field and creator field of defects in Jira are defined. Since step 4 obtains detailed information on defects found by testers and defects created by R&D personnel every day, and the responsible person and creator are both email addresses registered for Jira, it is necessary to first convert the email addresses into corresponding Chinese names, and then classify and count the number of daily defects according to the responsible person and creator, so as to obtain the number of defects found by testers and the number of defects created by R&D personnel every day.

[0070] Step 5: Store the cleaned data from step 4 and write the cleaned data from step 4 into the data model defined in step 2 in batches using the open source framework mybaties.

[0071] Step 6: By integrating open source visualization software, we developed an efficiency data visualization platform. We only need to write SQL statements to read the data model data in step 2 to obtain the various indicator data defined in step 1 and display them on the visualization platform.

[0072] The product performance management system based on automation technology proposed in this invention has the following structure: Figure 1 As shown, it contains the following parts:

[0073] The data layer includes code data submitted by R&D personnel, case data written by testers, case data executed by testers, defect data discovered by testers, defect data created by R&D personnel, data constructed by automated release tools, etc.

[0074] The computing layer includes a data access module and a data cleaning module; the data access module is used to use the API to obtain data from various tools in the data layer, specifically implementing the content of step three in the above-mentioned product performance management method based on automation technology, while the data cleaning module cleans the data obtained by the data access module, specifically implementing the content of step four in the above-mentioned product performance management method based on automation technology.

[0075] The storage layer, including the database cluster module and the data storage service module, is used to store the data obtained by the data cleaning module, and specifically implements the content of step 5 of the above-mentioned product performance management method based on automation technology.

[0076] The presentation layer includes a performance data visualization platform, which is used to read data model data and display various indicator data on the visualization page, specifically implementing the content of step 6 of the above-mentioned product performance management method based on automation technology.

[0077] It should be noted that the above content merely illustrates the technical idea of the present invention and cannot be used to limit the scope of protection of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications all fall within the scope of protection of the claims of the present invention.

Claims

1. A product performance management method based on automation technology, characterized in that: The steps include: Step 1: Define various product performance indicator data, including product dimension indicators and personal dimension indicators. The product dimension indicators include code development efficiency, defect repair time, test case automation rate, code defect rate, defect leakage rate, and automated build success rate. The personal dimension indicators include three sub-dimension indicators: development engineer, test engineer, and UI engineer. The development engineer indicators include code development efficiency, average defect repair time, and code defect rate. The indicators for test engineers include case writing efficiency, case execution efficiency, rate of new automated cases, and defect leakage rate; the indicators for UI engineers include page development efficiency; Step 2: Design the performance visualization platform table structure Using the product and personal indicators provided in step 1, and based on the first normal form theory of relational databases, use database design tools to design a data model for storing performance indicator data; Step 3: Integrate GitLab, Jenkins, and Jira through Sprint Boot to obtain data. The specific steps include the following: (1) Integrate the / api / json interface of Jenkins to obtain the build information of all modules of the previous day at regular intervals, and calculate the number of builds and the build success rate based on the modules; (2) Integrate gitlab's / projects / projectId / repository / commits interface to periodically obtain all commit information submitted to the master branch the previous day and aggregate it based on the committer's name; (3) Write a search query statement in Jira to periodically obtain the number of test cases written and executed by all testers in the previous day; (4) Write a search query statement that comes with Jira to regularly obtain the defects found by all testers on the previous day and the R&D personnel corresponding to the defects; Step 4: Clean the raw data information obtained in step 3. The specific cleaning rules are as follows: (1) For the full build record information obtained from Jenkins in step 3, classify and count each build by environment type, product type, and module type, and aggregate the build times of different products, different environments, and different modules according to the naming convention; (2) For the information obtained from GitLab in step 3, only the code information of the master branch is counted, and the merge commit information is excluded. The statistics are classified and summarized according to the account number carried in the commit; (3) Clean out the Chinese names of testers based on email information; (4) Define the responsible person field and creator field of defects in Jira, classify and count the number of defects according to the responsible person and creator, and obtain the number of defects found by testers and the number of defects created by R&D personnel; Step 5: Store the cleaned data from step 4 and write the cleaned data from step 4 into the data model defined in step 2 in batches; Step 6: Use the performance data visualization platform to display data By integrating the open source visualization software platform, write SQL statements to read the data model data in step 2, obtain the various indicator data defined in step 1 and display them.

2. The product performance management method based on automation technology according to claim 1, characterized in that: The data model in step 2 includes a code engineering table, an automated test statistics table, a Jenkins build statistics table, a code quality statistics table, a git group information table, a git code submission quality table, a test case statistics table, a UI statistics table, an employee table, a code submission count statistics table, and a jira issue information table.

3. The product performance management method based on automation technology according to claim 1, characterized in that: The naming convention in step 4 is environment name-product code-module name.

4. A product performance management system based on automation technology, for implementing the product performance management method based on automation technology according to any one of claims 1 to 3, comprising: The data layer includes R&D code data, writing case data, execution case data, discovered defect data, manufactured defect data, and data constructed by automated release tools; The computing layer includes a data access module and a data cleaning module. The data access module uses the API to obtain data from various tools in the data layer, specifically implementing step three of the product effectiveness management method based on automation technology. The data cleaning module cleans the data obtained by the data access module, specifically implementing step four of the product effectiveness management method based on automation technology. The storage layer, including the database cluster module and the data storage service module, is used to store the data obtained by the data cleaning module and specifically implement the content of step 5 of the product performance management method based on automation technology; The presentation layer includes a performance data visualization platform, which is used to read data model data and display various indicator data on the visualization page, specifically implementing the content of step six of the product performance management method based on automation technology.

Images