Quality monitoring operation method, system and equipment of Internet product and storage medium

Abstract

The invention relates to an internet product quality monitoring operation method, system and device and a storage medium. The method comprises the steps of creating a corresponding product monitoring task for each internet product; monitoring a current life cycle stage of each internet product and a product monitoring task execution condition of the current life cycle stage; when an execution condition is satisfied, executing the product monitoring task to obtain quality monitoring index data of corresponding product quality; calculating the quality monitoring index data based on the quality display index to obtain quality display index data; wherein the step of executing the product monitoring task comprises the following steps of: acquiring current stage data in the Internet product data from a data source based on an identifier of an Internet product, a current life cycle stage of the Internet product and a product quality monitoring index; quality monitoring indicator data is calculated based on the product quality monitoring indicator and the collected stage data. According to the embodiment of the invention, the quality of the Internet product can be dynamically, accurately and comprehensively monitored in the whole process.

Description

Technical Field

[0001] The present invention relates to the field of Internet application technology, and in particular to a quality monitoring operation method, system, device and storage medium for Internet products. Background Art

[0002] In the era of rapid development of Internet applications, Internet products have become an indispensable element in people's production and life. For developers of Internet products, the time and cost required for an Internet product from project establishment, development to online operation are different. In order to evaluate whether a product is "good" or "bad", the usual method is to build an evaluation index system and evaluate the product through the calculated index data. For example, a Chinese patent application with publication number CN116126668A and invention name "A software quality evaluation method, device, medium and equipment" discloses a software quality evaluation method, which is mainly based on the test process data to evaluate the software quality evaluation, but the scheme does not take into account the operation after going online. For another example, a Chinese patent with publication number CN110968512B and invention name "Software Quality Evaluation Method, Device, Equipment and Computer-readable Storage Medium" discloses a software quality evaluation method, which is mainly based on the attribute information of the target indicator sent to determine the corresponding target indicator group, and adopts a fuzzy neural network software quality evaluation model to evaluate each indicator in the target indicator group, but the scheme does not disclose which data is based on which the corresponding indicator is obtained. Another Chinese patent application with publication number CN115080413 A and invention name "Software Quality Evaluation Index Extraction Method, Device and Computer Equipment" discloses a software quality evaluation method, which adds data statistics of demand data in the software application process and related form indicator information based on the basic software indicator information, and selects software quality evaluation indicators for evaluating software quality from these indicator information. There are also some similar technical solutions, which will not be described one by one.

[0003] By analyzing the quality evaluation schemes for software products in the prior art, it can be seen that the prior art schemes usually evaluate software products from certain aspects, such as evaluating software quality through testing in the testing phase, or evaluating software quality through the processing of specific demand data by the software in the application phase. However, for an Internet product, these schemes are too one-sided and cannot accurately measure the quality of the product. Summary of the invention

[0004] In view of the technical problems existing in the prior art, the present invention proposes a quality monitoring operation method, system, device and storage medium for Internet products, which can accurately and comprehensively monitor the quality of Internet products.

[0005] In order to solve the above technical problem, according to one aspect of the present invention, the present invention provides a quality monitoring operation method for an Internet product, comprising the following steps:

[0006] Create corresponding product monitoring tasks for each Internet product;

[0007] Monitor the current life cycle stage of each Internet product and the conditions for executing its product monitoring tasks;

[0008] In response to the product monitoring task satisfying the execution condition, executing the product monitoring task to obtain quality monitoring indicator data of the corresponding product quality; and

[0009] Calculating the quality monitoring indicator data based on the quality display indicator to obtain the quality display indicator data;

[0010] The steps of performing product monitoring tasks include:

[0011] Determine a first data source storing product data based on the identification of the Internet product and its current life cycle stage;

[0012] Determine the corresponding product quality monitoring indicators based on the current life cycle stage;

[0013] Collecting current stage data in the Internet product data from the first data source;

[0014] Calculating the quality monitoring index data of the product at the current stage based on the product quality monitoring index and the collected stage data; and

[0015] The quality monitoring index data of the product is stored in a second data source, wherein the second data source is isolated from the first data source.

[0016] Optionally, the life cycle stages include a pre-online stage, a release stage and an online operation stage, wherein the product quality monitoring indicators corresponding to the pre-online stage include R&D process indicators, testing process indicators and product process indicators; the quality monitoring indicators corresponding to the release stage include R&D release indicators; the quality monitoring indicators corresponding to the online operation stage include R&D online indicators, testing online indicators and all-role online indicators.

[0017] Optionally, the method further comprises:

[0018] Acquire employee data, the employee data including employee identification and the corresponding Internet product for which the employee works;

[0019] Create employee performance monitoring tasks;

[0020] Monitoring effectiveness Monitoring the execution conditions of tasks;

[0021] In response to the performance monitoring task satisfying the execution condition, executing the performance monitoring task to obtain quality monitoring indicator data corresponding to the employee's performance;

[0022] The steps of performing the performance monitoring task include:

[0023] Based on the performance monitoring indicator, the employee identification and the Internet product identification of his / her work, collect product data within a corresponding period from a first data source storing product data;

[0024] Calculating the quality monitoring indicator data of the employee's efficiency based on the efficiency monitoring indicator and the product data; and

[0025] The quality monitoring indicator data of the employee effectiveness is stored in a second data source, wherein the second data source is isolated from the first data source.

[0026] Optionally, when a monitoring task fails to be executed, the monitoring task is recorded in a first data table; correspondingly, the first data table is queried when the monitoring task is executed, and the monitoring task in the first data table is added to a list of tasks currently being executed.

[0027] Optionally, when executing a monitoring task, when collecting data from the first data source, based on corresponding monitoring indicators, atomic data that meets the dimensions required for indicator calculation is collected.

[0028] Optionally, the method further comprises:

[0029] Applying for an authentication token for data collection from a first data source;

[0030] Correspondingly, when collecting data from the first data source, permission verification is performed with the first data source through the authentication token, and data is collected after passing the permission verification.

[0031] Optionally, the calculation of the quality monitoring indicator data further includes:

[0032] Determine the corresponding calculation expression and original item data based on the quality monitoring indicators;

[0033] Extracting collected data corresponding to the original item data from the collected data and performing aggregation operations on the collected data, wherein the aggregation operations include one or more of mean operations, maximum operations, median operations, sum operations, minimum operations, count operations, non-repeated value operations, and standard deviation operations;

[0034] The aggregation operation result is calculated accordingly according to the calculation expression to obtain the corresponding quality monitoring index data, and the quality monitoring index includes product quality monitoring index and employee efficiency quality monitoring index data.

[0035] Optionally, the quality monitoring indicator includes a grade attribute graded according to importance.

[0036] Optionally, when quality monitoring indicator data is calculated based on the quality display indicator to obtain quality display indicator data, the obtained quality monitoring indicator data is stored in a cache, and when different quality display indicators are applied to the same quality monitoring indicator data, the corresponding quality monitoring indicator data is obtained from the cache.

[0037] According to one aspect of the present invention, the present invention provides a quality monitoring operation system for an Internet product, comprising:

[0038] A task creation module, configured to create a corresponding product monitoring task for each Internet product;

[0039] A monitoring module, which is connected to the task creation module and is configured to monitor the current life cycle stage of each Internet product and its product monitoring task execution conditions;

[0040] a task module connected to the monitoring module and configured to execute the product monitoring task in response to the product monitoring task satisfying the execution condition to obtain the corresponding product quality monitoring indicator data; and

[0041] A data presentation module, connected to the task module and configured to calculate the quality monitoring indicator data based on the quality presentation indicator to obtain the quality presentation indicator data;

[0042] Wherein, the task module includes:

[0043] an identification unit configured to determine a first data source storing product data and corresponding product quality monitoring indicators based on an identification of the Internet product and its current life cycle stage;

[0044] a collection unit connected to the identification unit and configured to collect current stage data in the Internet product data from the first data source;

[0045] A calculation unit is connected to the identification unit and the collection unit respectively, and is configured to calculate the quality monitoring index data of the product at the current stage based on the product quality monitoring index and the collected stage data; and store the quality monitoring index data of the product in a second data source, wherein the second data source is isolated from the first data source.

[0046] Optionally, the task creation module further includes:

[0047] A first task creation unit, configured to create a corresponding product monitoring task for each Internet product; and

[0048] A second task creation unit, configured to create an employee performance monitoring task based on employee data, wherein the employee data includes an employee identification and a corresponding Internet product for which the employee works;

[0049] Correspondingly, the collection unit also collects product data within the corresponding period from a first data source storing product data based on the performance monitoring indicators, employee identification and the Internet product identification of his / her work; the calculation unit calculates the quality monitoring indicator data of the employee's performance based on the performance monitoring indicators and the product data.

[0050] According to another aspect of the present invention, the present invention also provides an electronic device, which includes a processor and a memory, wherein a computer program instruction set is stored in the memory, and when the processor executes the computer program instruction set in the memory, the aforementioned Internet product quality monitoring operation method is implemented.

[0051] According to another aspect of the present invention, the present invention also provides a computer-readable storage medium, wherein a computer program instruction set is stored on the computer-readable storage medium, and when the computer program instruction set is executed by a processor, the aforementioned Internet product quality monitoring operation method is implemented.

[0052] According to another aspect of the present invention, the present invention also provides a computer program product, which includes a computer program instruction set, and when the computer program instruction set is executed by a processor, it implements the aforementioned Internet product quality monitoring operation method.

[0053] Based on the characteristic that Internet products change rapidly in response to user needs, the embodiments of the present invention reasonably divide the life cycle of Internet products into multiple stages and set corresponding quality monitoring indicators for each stage, so as to dynamically and comprehensively evaluate the quality of Internet products throughout the entire process. In this way, the quality of Internet products can be accurately and comprehensively monitored, and the quality monitoring indicator data can also reflect the quality problems of the product at each stage, which is conducive to improving product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The preferred embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0055] Figure 1 is a flow chart of a quality monitoring operation method for an Internet product according to an embodiment of the present invention;

[0056] Figure 2 is a process flow chart for executing a product monitoring task according to an embodiment of the present invention;

[0057] Figure 3 is a process flow chart for executing multiple monitoring tasks simultaneously according to an embodiment of the present invention;

[0058] Figure 4 is a flow chart of a quality monitoring operation method for an Internet product according to another embodiment of the present invention;

[0059] Figure 5 is a process flow chart of executing a performance monitoring task according to an embodiment of the present invention;

[0060] Figure 6 is a principle block diagram of a quality monitoring and operation system for an Internet product according to an embodiment of the present invention;

[0061] Figure 7 is a principle block diagram of a task module according to an embodiment of the present invention;

[0062] Figure 8 is a principle block diagram of a quality monitoring and operation system for Internet products according to another embodiment of the present invention;

[0063] Fig. 9 is a principle block diagram of a data presentation module according to an embodiment of the present invention; and

[0064] Fig.10 It is a schematic diagram of the hardware structure principle of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] In the following detailed description, reference may be made to the various specification drawings that are part of the present application and are used to illustrate specific embodiments of the present application. In the accompanying drawings, similar reference numerals describe substantially similar components in different figures. The various specific embodiments of the present application are described below in sufficient detail so that a person of ordinary skill in the art with relevant knowledge and skills in the art can implement the technical solutions of the present application. It should be understood that other embodiments may also be used or structural, logical or electrical changes may be made to the embodiments of the present application.

[0067] The present invention provides a quality monitoring operation method and system for Internet products. The Internet products are used to provide certain independent functions or services. For recruitment platforms, Internet products include search engines that provide users with job or resume search services, delivery services for job seekers' resumes to recruiting users, chat tools that establish communication channels for both recruiting and job seeking users, and the like.

[0068] In order to monitor the quality of Internet products, the present invention sets a life cycle for Internet products, that is, from the start of project development to the end of product offline use as a life cycle, and sets multiple life cycle stages and corresponding product quality monitoring indicators based on the characteristics of Internet products. For example, the life cycle stage generally includes the pre-launch stage, the release stage, and the online operation stage. Corresponding to each stage, product quality monitoring indicators related to product quality are set according to the factors that affect product quality at that stage.

[0069] In one embodiment, the product quality monitoring indicators corresponding to the pre-launch stage include R&D process indicators, test process indicators and product process indicators; the quality monitoring indicators corresponding to the release stage include R&D release indicators; the quality monitoring indicators corresponding to the online operation stage include R&D launch indicators, test launch indicators and all-role launch indicators.

[0070] The quality monitoring indicators corresponding to each stage also include one or more specific indicators. For example, for the pre-launch stage, the R&D process indicators include serious defect rate, single test pass rate, test submission delay rate, defect-to-demand ratio, defect rate per thousand lines of code, average defect repair time, etc.; the test process indicators include automation defect rate, interface coverage, defect effectiveness, number of use cases per defect consumption, interface coverage granularity, etc.; the product process indicators include demand rejection rate, number of product defects in the demand process, number of optimized requirements, etc. For the release stage, the R&D release indicators include compilation failure rate, deployment failure rate, release rollback rate, etc. For the online operation stage, the R&D launch indicators include the number of production defects, the number of online accidents, the production alarm Ack rate, the production alarm response time, the monitoring coverage rate, etc.; the test launch indicators include the test missed rate, the online inspection success rate, the number of online recalls, etc.; the full-role launch indicators include the launch delay rate, etc.

[0071] Each of the aforementioned indicators has a corresponding calculation expression and a hierarchical attribute classified according to importance. For example, in descending order of importance, the hierarchical attributes of indicators include core, important, standard and conventional. Core indicators are the most important indicators for measuring quality value. The fluctuation of such indicators is strongly related to the performance of stage quality and needs to be focused on and operated. Important indicators are secondary in importance relative to core indicators, but they are also indicators that affect the quality and operational effects of products or services. Important indicators are usually closely related to core indicators. Standard indicators are quantifiable and are usually defined according to industry best practices and standards, which can reflect the quality and operational level of products or services. Conventional indicators are also quantifiable, but they are usually not the focus of products or services. The present invention can effectively manage numerous indicators by grading all aspects of quality monitoring indicators, and provides a basis for data collection, execution of product monitoring tasks, and calculation of quality display indicator data.

[0072] The quality monitoring indicators of each of the above stages include at least one or more indicators with the highest level attributes. For example, for the pre-launch stage, the level attributes of the severe defect rate, unit test pass rate, and test submission delay rate in the R&D process indicators are the core. This is because the severe defect rate in the R&D process directly reflects the project quality, the unit test pass rate directly reflects the development investment, and the test submission delay rate directly reflects the quality of the demand process; the level attributes of the defect effectiveness, automated test defect rate, and interface coverage rate in the testing process indicators are the core. This is because the defect effectiveness directly reflects the test quality, the automated test defect rate reflects the proportion of defects found in the test, and the interface coverage rate reflects the interfaces that can be covered by automation capabilities. For the release stage, the level attribute of the release rollback rate is the core, because this indicator reflects the smoothness of the release process and the quality of operation and maintenance delivery. For the online operation stage, the level attributes of the number of production defects and the number of online accidents in the R&D online indicator are the core, because this indicator directly reflects the quality of the R&D test delivery process; the level attribute of the test leakage rate in the test online indicator is the core, because this indicator directly reflects the test quality in the testing process; the level attribute of the online delay rate in the all-role online indicator is the core, because this indicator directly reflects the quality of the requirements and the delivery capability.

[0073] Figure 1 1 is a flow chart of a quality monitoring operation method for an Internet product according to an embodiment of the present invention. In this embodiment, the method comprises the following steps:

[0074] Step S1, creating a corresponding product monitoring task for each Internet product.

[0075] Step S2, monitor the current life cycle stage of each Internet product and its product monitoring task execution conditions.

[0076] Step S3, determine whether there is a product monitoring task that meets the execution condition, if yes, then execute the product monitoring task in step S4 to obtain the corresponding product quality monitoring index data, and then execute step S5. If no product monitoring task meets the execution condition, return to step S2.

[0077] Step S5, calculating the quality monitoring indicator data based on the quality display indicator to obtain the quality display indicator data.

[0078] In one embodiment, the present invention configures a life cycle stage table for Internet products in a database, which records the progress of each Internet product in its life cycle stage. For example, a record includes all the life cycle stages of an Internet product. When it goes through a stage, the stage is marked. As the life of the Internet product progresses, the record will mark the stages it has gone through in sequence, where the last stage is usually the current stage. Correspondingly, a life cycle stage status parameter can be used to record the life cycle stage of an Internet product. For example, the life cycle stage status parameter includes multiple binary data bits, each data bit corresponds to a life cycle stage, 0 represents not yet experienced, and 1 represents already experienced or is being experienced. The corresponding data bit of the life cycle stage status parameter is modified according to the mark in the life cycle stage table, and the 1 on the last data bit represents the stage that the Internet product is currently experiencing.

[0079] In order to facilitate the management and monitoring of multiple Internet products, the present invention creates a corresponding product monitoring task for each Internet product in step S1. For example, a product monitoring task table is set in the database, and a trigger condition for task execution is configured for each task. The trigger condition for task execution is, for example, reaching a timing cycle, or receiving a monitoring instruction (for example, a monitoring button from a system interaction interface, usually issued by a staff member), or monitoring that a state parameter of a life cycle stage of an Internet product has changed, or an indicator data has changed, etc. For example, thresholds for some core indicators are set, and the monitoring task is executed when the core indicator data reaches the threshold.

[0080] Figure 2 The flowchart of executing a product monitoring task according to an embodiment of the present invention specifically includes the following steps:

[0081] Step S41, determining a first data source storing product data based on the identification of the Internet product and its current life cycle stage.

[0082] Step S42, determining corresponding product quality monitoring indicators based on the current life cycle stage.

[0083] Step S43: collecting current stage data in the Internet product data from the first data source.

[0084] Step S44, calculating the quality monitoring index data of the product at the current stage based on the product quality monitoring index and the collected stage data.

[0085] Step S45, storing the quality monitoring index data of the product in a second data source, wherein the second data source is isolated from the first data source.

[0086] Among them, for an Internet product, the data generated at different life cycle stages may be stored in different data sources. For example, for some software product development companies, a project management platform is usually used to manage various projects that constitute a software product, and the project management platform stores project data. In order to implement product testing, a test platform is usually used to test Internet products, in which test-related data is stored, and systems and databases related to software development are also included, such as Git for version control, code warehouses storing software codes, CICD (continuous integration, continuous delivery process management system), demand process management platforms that record employee work data, etc. In step S41, the first data source storing product data is determined based on the identification of the Internet product and its current life cycle stage. Then in step S42, the corresponding product quality monitoring indicators are determined based on the current life cycle stage, and the data to be collected is determined according to the specific quality monitoring indicators, and the first data source is interacted with in step S42 to collect the required data. Among them, in order to ensure data security, it is necessary to perform security authentication with the first data, and the required data can only be collected after the authentication is passed. In one embodiment, the two use authentication tokens for security authentication. Therefore, it is first necessary to apply for an authentication token for data collection from the first data source. After obtaining the authentication token, security authentication is performed through the authentication token each time data is collected, thereby ensuring the security of the data.

[0087] In addition, when collecting data, in order to improve data collection efficiency and reduce resource and system consumption when communicating between different systems, batch processing is used to collect quality data scattered in various fields to ensure that all data within this stage is collected without missing any data.

[0088] After the data is collected, the data in different formats are preprocessed in a standardized manner. For example, when the data source types include Mysql, mongo, and the open interface provided by restfulApi, the data of these different data source types are processed into a unified standard Json object format and stored in another database. In one embodiment, the preprocessed data is stored in the form of a data statistics table. In addition, in order to facilitate the calculation of various quality monitoring indicators, atomic data of the dimensions required for indicator calculation are collected when collecting data. The atomic data only includes data of one dimension. In one embodiment, since the Mongo database supports dynamic expansion of fields, the atomic data in the present invention can be stored in the Mongo database in the form of fields and their field values, which is convenient for adding atomic data of new dimensions.

[0089] The quality monitoring indicators in the present invention are based on the data required for calculation, and the collected data include time point data and process data. For time point data, only data at a specific time point is collected, while for process data, data of the corresponding time range is collected according to the time period configured by the specific quality monitoring indicator. Among them, the time point data is real-time data with a high refresh frequency, and the process data can show the process trend. For example, the serious defect rate in the R&D process indicator is configured with four parameters and three time periods in terms of week, month, quarter and related business dimensions. When collecting data, according to the configured time period, data of specific business dimensions from one week, one month and one quarter forward from the current time are collected respectively. For example, the corresponding atomic number collected from 1st to 7th of mm month of xxxx is: [mm month 1st of xxxx year: number of offline defects: xx1; total number of serious offline defects: xy1]; [mm month 2nd of xxxx year: number of offline defects: xx2; total number of serious offline defects: xy2]... [mm month 7th of xxxx year: number of offline defects: xx7; total number of serious offline defects: xy7]. The defect quantity collected above is a kind of time point data. Usually, the time point data will affect the process data.

[0090] Since the collected data of the same type are usually more than one, when calculating specific index data, the collected data is aggregated according to the specific index, and the aggregation operation includes mean operation, maximum operation, median operation, sum operation, minimum operation, count operation, non-repeated value operation, standard deviation operation, etc. For example, the data collected for calculating the serious defect rate for one week includes 7 data, so in this embodiment, in order to obtain specific index data according to its calculation formula Y1=total number of serious offline defects / total number of offline defects, the collected data is summed, that is, the total number of serious offline defects x1=xy1+xy2+…+xy7, the total number of offline defects x2=xx1+xx2+…+xx7, and the two parameter data in the calculation formula are obtained.

[0091] The aggregation operation used in the calculation of each indicator can adopt the system default built-in methods such as $match, $sort, $project, etc., without additional customized operation methods, thereby reducing the overhead of system resources.

[0092] Then, in step S44, specific indicator data Y1 is calculated based on the formula Y1=x1 / x2.

[0093] In step S45, the data used for calculation and the result data obtained after calculation are stored and isolated from the original product data (or metadata), thereby avoiding data pollution.

[0094] In the present invention, a corresponding quantification method is provided for each product quality monitoring indicator, such as a specific calculation formula. For example, the calculation formula for the severe defect rate in the aforementioned R&D process indicator is: severe defect rate = total number of severe offline defects / total number of offline defects; for example, the interface coverage rate in the test process indicator is calculated as: interface coverage rate = covered interfaces / total number of interfaces; the quantification method is also, for example, the statistical data of specific parameters, for example, the number of product defects in the demand process in the product process indicator, the quantification method is: the number of product defects in the demand process = the number of offline bugs attributed to demand; for example, the number of production defects in the online quality indicator, the quantification method is: the number of production defects = the number of defects attributed to online incoming line feedback, and so on.

[0095] Since the present invention monitors the quality of multiple Internet products, in one embodiment, when multiple monitoring tasks meet the execution conditions, multiple monitoring tasks are executed simultaneously in a batch processing manner, and the execution status of each monitoring task is recorded in the monitoring task table. Figure 3 , Figure 3 1 is a flowchart of a process for executing multiple monitoring tasks simultaneously according to an embodiment of the present invention, comprising the following steps:

[0096] Step S51: determine the monitoring tasks that meet the execution conditions as tasks to be executed.

[0097] Step S52, traverse the last execution status of each monitoring task table in the monitoring task table. Wherein, when the execution of the monitoring task is interrupted due to an abnormality, in one embodiment, the interruption will be recorded in the monitoring task table.

[0098] Step S53, judging whether there is an abnormally interrupted monitoring task based on the execution status, if yes, then in step S54, determining the monitoring task that was interrupted last time as the task to be executed. If no, then executing step S55.

[0099] Step S55, execute all the tasks to be executed. During the execution process, if there is an abnormally interrupted task, its execution status is recorded as interrupted in the monitoring task table. After the task is executed, the quality monitoring index data of the corresponding product is stored in step S56.

[0100] This embodiment sets a compensation task for the monitoring task that fails to execute, and effectively prevents the relevant indicator data from being missed through the re-running mechanism of the compensation task, thereby ensuring the accuracy of the final data.

[0101] In another embodiment, when a monitoring task fails to be executed, the failed monitoring task is recorded in another data table, referred to herein as the first data table; correspondingly, as an alternative to step S51, the monitoring task that meets the execution condition is determined as a task to be executed, and a task list for this execution is generated. As an alternative to step S52, the first data table is queried, and the monitoring tasks in the first data table are added to the list of tasks currently being executed. Then all the tasks to be executed are executed. During the execution process, when the tasks added from the first data table are successfully executed, the monitoring tasks are cleared from the first data table, and if there are abnormally interrupted tasks during the task execution process, they are added to the first data table so that they can be re-executed the next time the tasks are executed.

[0102] See again Figure 1In order to facilitate understanding of the quality of Internet products and to view data of various dimensions as needed, the present invention also sets a variety of quality display indicators, which can be the aforementioned quality monitoring indicators, or can be correlation indicators of several quality monitoring indicators. For example, correlation indicators can be used to compare the same quality monitoring indicator data of different products, to compare the quality monitoring indicator data of the same product at different stages, etc., or various statistical indicators of the same quality monitoring indicator in different time ranges. Among them, each quality display indicator includes time attributes and display attributes in addition to the corresponding quantification method. Among them, the time attribute includes a time point and a time period, and the time period is used to represent a time range, which is determined by the start time and the end time. The attribute value of the display attribute corresponds to one or more display methods, and the display method is, for example, a graphical display method such as a curve chart, a line chart, a pie chart, a bar chart, etc., or a static three-dimensional model display method, or a table display method with a certain number of columns and rows.

[0103] For example, when the time attribute of the quality display indicator is a time period, the relevant indicator data can be visually displayed in a graphical manner such as a curve chart or a line chart.

[0104] Curve graphs, line graphs, pie charts, bar charts, etc. are static plane graphics. In another embodiment, a static three-dimensional model can also be used to display relevant indicator data. By using different faces, edges, etc. of the three-dimensional model as a kind of indicator data, multiple related indicator data can be well displayed, and the correlation between different indicators can be displayed.

[0105] In one embodiment, all quality monitoring indicator data of an Internet product can be displayed in a table, with columns corresponding to different quality monitoring indicator indices and rows corresponding to the time of obtaining specific quality monitoring indicator data. For another example, the data of the same one or more quality monitoring indicators of different Internet products can be displayed in a table, with columns corresponding to quality monitoring indicator indices and rows corresponding to the identifiers of Internet products. In addition, the display attributes can include both graphics and tables, so when the staff is viewing a certain quality display indicator, they can set a specific time in the time attribute and a specific display method in the display attribute.

[0106] The aforementioned display method is a static method, but the present invention also provides a dynamic display method for data, such as animation and dynamic three-dimensional models. Since the present invention monitors the quality of Internet products related to time at various stages in their life cycle, when displaying its quality-related data, it can display the changes in the quality of Internet products within a time period through a dynamic display method, such as the quality changes during the entire life cycle or at a certain stage or several stages therein. Among them, in the graphical display method and the dynamic display method, data related to quality, such as certain quality monitoring indicator data, can be added to the graphics or diagrams in the form of text or voice.

[0107] In one embodiment, all quality display indicators include a default time and display mode. In the default display mode, quality display indicator data corresponding to multiple core levels of quality monitoring indicator data are distributed in the interactive interface for staff to view. Therefore, when the staff does not set it, the corresponding quality display indicator is determined based on the default display mode, and the quality display indicator data is calculated and displayed in the interactive interface according to the default display mode.

[0108] When calculating a quality display indicator data, the corresponding quality monitoring indicator is determined according to its quantification method, and the second data source is accessed to obtain the corresponding quality monitoring indicator data, and then the corresponding quality display indicator data is obtained according to the quantification method. In order to reduce the access to the second data source and cause additional system overhead, in one embodiment, the obtained quality monitoring indicator data is stored in a cache, and when calculating other quality display indicators, when the same quality monitoring indicator data is applied, the corresponding quality monitoring indicator data is obtained from the cache.

[0109] See also Figure 4 , Figure 4 This is a flow chart of a quality monitoring operation method for an Internet product according to another embodiment of the present invention. Since the quality of Internet products is usually related to manpower, the present invention also sets indicators related to manpower when monitoring the quality of Internet products, which are called performance monitoring indicators in the present invention, and sets corresponding quantification methods. The performance monitoring indicator data obtained through monitoring shows the relationship between Internet products and manpower, which specifically includes the following steps:

[0110] Step S61, obtaining employee data, the employee data includes employee identification and the corresponding Internet product that the employee works on. The employees mentioned here refer to employees who are involved in work that affects the quality of Internet products, such as developers, testers, etc. The database or management platform records the employee's work data, such as the Internet product that each employee works on, the life cycle stage involved, the employee's tasks and their completion, etc.

[0111] Step S62, create an employee performance monitoring task.

[0112] Step S63: monitor the execution conditions of the performance monitoring task, such as a certain time period or execution instruction.

[0113] Step S64, determine whether the performance monitoring task execution condition is met, if so, then in step S65, execute the performance monitoring task to obtain quality monitoring index data corresponding to the employee's performance. If not, return to step S63.

[0114] When performing a performance monitoring task, in one embodiment, see Figure 5 , Figure 5 The flowchart of executing a performance monitoring task according to an embodiment of the present invention specifically includes the following steps:

[0115] Step S651, based on the performance monitoring index, the employee ID and the Internet product ID of his work, collect product data within the corresponding period from the first data source storing product data. The performance monitoring index is similar to the quality monitoring index, and has a corresponding quantification method. The required product data type can be obtained according to the data applied in the quantification method. Therefore, according to the product data type used by the quantitative performance monitoring index, the corresponding type of product data is read in the database according to the employee ID and the Internet product ID.

[0116] Step S652: Based on the performance monitoring index and the product data, the quality monitoring index data of the employee performance is calculated according to the quantification method of the performance monitoring index.

[0117] Step S653: storing the quality monitoring index data of the employee effectiveness in a second data source, wherein the second data source is isolated from the first data source.

[0118] In one embodiment, the performance monitoring indicator is, for example, demand throughput, and its quantification method is the number of demands for R&D and test delivery within the statistical period. Therefore, in step S651, according to the statistical period and the number of demands for R&D and test delivery in the quantification method, the demand for R&D and test delivery corresponding to a product identifier is read from the database. Since the present invention collects atomic data when collecting data, when reading data, all R&D and test delivery demand records within the time period are read separately, and then, when calculating the performance monitoring indicator data in step S652, the number of records is counted to obtain the corresponding demand throughput.

[0119] The performance monitoring indicator is also an example of the demand capacity determination success rate, and its quantification method is: demand capacity determination success rate = number of demands that have been successfully scheduled and determined by manpower / total number of demands. Therefore, in step S651, according to the requirements in the quantification method, the database is queried and read for demand records, and then in step S652, when calculating the performance monitoring indicator data, the total number of records is counted, and the mark of successful scheduling and determination in the record is referred to, and the number of records with this mark is counted, and then the ratio of the two is calculated to obtain the specific demand capacity determination success rate.

[0120] The present invention is based on the characteristic that Internet products change rapidly in response to user needs. By reasonably dividing the life cycle of Internet products into the pre-launch stage, the release stage, and the online operation stage, and setting corresponding quality monitoring indicators for each stage, the quality of Internet products is evaluated dynamically and comprehensively throughout the whole process. It can not only accurately and comprehensively monitor the quality of Internet products, but also the quality monitoring data can reflect the problems at each stage, which is conducive to improving product quality. In addition, by monitoring the human indicators related to product quality, human resources can be effectively arranged, thereby further improving product quality from a human perspective. Therefore, compared with the quality evaluation method of a single angle and a single product, the present invention can more comprehensively reflect the "good" and "bad" of a product, and provide a reliable and credible basis for improving product quality.

[0121] On the other hand, the present invention also provides a quality monitoring and operation system for Internet products, see Figure 6 , Figure 6 It is a principle block diagram of the quality monitoring operation system of an Internet product according to an embodiment of the present invention. The system includes a task creation module 1, a monitoring module 2, a task module 3 and a data presentation module 4, wherein the task creation module 1 is configured to create a corresponding product monitoring task for each Internet product. In one embodiment, a product monitoring task table is used to record all created product monitoring tasks. The monitoring module 2 is connected to the task creation module 1, and is configured to monitor the current life cycle stage of each Internet product and its product monitoring task execution conditions. When a product monitoring task meets the execution conditions, a notification is sent to the task module 3. After receiving the notification sent by the monitoring module 2, the task module 3 executes the product monitoring task in the notification to obtain the quality monitoring index data of the corresponding product quality. After the calculation is completed, the task module 3 sends a notification to the data presentation module 4, and the data presentation module 4 calculates the quality monitoring index data based on the quality presentation index to obtain the quality presentation index data.

[0122] See also Figure 7 , Figure 7 FIG. 1 is a block diagram of a task module according to an embodiment of the present invention. Figure 6In one embodiment, the task module 3 includes an identification unit 31, a collection unit 32 and a calculation unit 33. The identification unit 31 determines the first data source storing product data and the corresponding product quality monitoring indicators based on the identification of the Internet product and its current life cycle stage. The collection unit 32 is connected to the identification unit 31 to collect the current stage data in the Internet product data from the first data source. The calculation unit 33 is connected to the identification unit 31 and the collection unit 32 respectively, calculates the quality monitoring indicator data of the product at the current stage based on the product quality monitoring indicators and the collected stage data, and stores the quality monitoring indicator data of the product in the second data source, wherein the second data source is isolated from the first data source.

[0123] See also Figure 8 , Figure 8 It is a principle block diagram of the quality monitoring operation system of Internet products according to another embodiment of the present invention. In this embodiment, the task creation module 11 further includes a first task creation unit 11 and a second task creation unit 12, wherein the first task creation unit 11 creates a corresponding product monitoring task for each Internet product; the second task creation unit 12 creates an employee efficiency monitoring task based on employee data, wherein the employee data includes an employee ID and the corresponding Internet product of the employee's work, and stores the created employee efficiency monitoring task in the employee efficiency monitoring task table. In addition to monitoring product monitoring tasks, the monitoring module 2 also monitors employee efficiency monitoring tasks. Task module 3 not only performs product monitoring tasks, but also performs employee efficiency monitoring tasks. When executing various tasks, batch processing can be performed, and after the task is interrupted due to an abnormality, it will be re-executed at the next execution. When executing the employee efficiency monitoring task, the collection unit 32 collects product data within the corresponding period from the first data source storing product data based on the efficiency monitoring index, the employee ID and the Internet product ID of his work; the calculation unit 33 calculates the quality monitoring index data of the employee efficiency based on the efficiency monitoring index and the product data, and stores it in the second data source.

[0124] Fig. 9: is a principle block diagram of a data presentation module according to an embodiment of the present invention. In this embodiment, the data presentation module 4 includes an interactive interface unit 41, a quality presentation index unit 42, a calculation unit 43 and a cache unit 44. Among them, the interactive interface unit 41 provides an interactive interface for providing quality presentation index data in the interactive interface, and provides various tools for interaction, such as buttons, menus, setting windows, etc. The quality presentation index unit 42 is used to provide various quality presentation indicators and creation methods. The quality presentation index includes time attributes, presentation attributes and corresponding quantization methods. The calculation unit 43 is used to read the corresponding data based on the quality presentation index to be calculated when the quality presentation index data needs to be calculated, and calculate according to the corresponding quantization method to obtain the quality presentation index data, and obtain the presentation data according to the corresponding presentation attribute, and display it on the interactive interface. Among them, the cache unit 44 is used to cache the data used in the calculation process to prevent frequent reading of data from causing excessive system overhead.

[0125] See also Fig.10 , Fig.10 It is a schematic diagram of the hardware structure principle of an electronic device according to an embodiment of the present invention. The electronic device can be implemented as a server or other various terminal devices, such as a desktop personal computer, a tablet computer, a laptop computer, a mobile phone, etc., including a processor 601 and a memory 602. The memory 602 stores a program instruction set. When the processor 601 executes the program instruction set on the memory 602, the aforementioned Internet product quality monitoring operation method is implemented.

[0126] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0127] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

[0128] The memory may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the quality monitoring operation method of the Internet product provided by the present invention.

[0129] In one example, the electronic device may further include a communication interface 603 and a bus 604. The processor 601, the memory 602, and the communication interface 603 are connected via the bus 604 and communicate with each other.

[0130] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiment of the present invention.

[0131] Bus 604 includes hardware, software or both, and the components of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 604 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0132] The present invention also provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement any of the Internet product quality monitoring operation methods in the aforementioned embodiments. The computer-readable storage medium can be any tangible medium that contains or stores computer-executable instructions for use by or in combination with instruction execution systems, devices and equipment. The storage medium can be a transient computer-readable storage medium or a non-transient computer-readable storage medium. Non-transient computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical disks based on CD, DVD or Blu-ray technology, and persistent solid-state memories such as flash memory, solid-state drives, and the like.

[0133] The present invention also provides a computer program product, which includes a set of computer program instructions, and when the computer program instruction set is executed by a processor, it implements any one of the Internet product quality monitoring operation methods in the aforementioned embodiments. The computer program product includes but is not limited to application installation packages published on websites and application stores, application plug-ins, and small programs that can be run in certain applications.

[0134] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0135] The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the scope of the present invention. Therefore, all equivalent technical solutions should also fall within the scope of the present invention.

Claims

1. A quality monitoring and operation method for an Internet product, characterized in that: include: Create corresponding product monitoring tasks for each Internet product; Monitor the current life cycle stage of each Internet product and the conditions for executing its product monitoring tasks; In response to the product monitoring task satisfying the execution condition, executing the product monitoring task to obtain quality monitoring indicator data of the corresponding product quality; as well as Calculating the quality monitoring indicator data based on the quality display indicator to obtain the quality display indicator data; The steps of performing product monitoring tasks include: Determine a first data source storing product data based on the identification of the Internet product and its current life cycle stage; Determine the corresponding product quality monitoring indicators based on the current life cycle stage; Collecting current stage data in the Internet product data from the first data source; Calculating the quality monitoring index data of the product at the current stage based on the product quality monitoring index and the collected stage data; and The quality monitoring index data of the product is stored in a second data source, wherein the second data source is isolated from the first data source.

2. The method according to claim 1, characterized in that The life cycle stages include the pre-launch stage, the release stage and the online operation stage, wherein the product quality monitoring indicators corresponding to the pre-launch stage include R&D process indicators, test process indicators and product process indicators; the quality monitoring indicators corresponding to the release stage include R&D release indicators; the quality monitoring indicators corresponding to the online operation stage include R&D launch indicators, test launch indicators and full-role launch indicators.

3. The method according to claim 1, characterized in that Further including: Acquire employee data, the employee data including employee identification and the corresponding Internet product for which the employee works; Create employee performance monitoring tasks; Monitoring effectiveness Monitoring the execution conditions of tasks; In response to the performance monitoring task satisfying the execution condition, executing the performance monitoring task to obtain quality monitoring indicator data corresponding to the employee's performance; The steps of performing the performance monitoring task include: Based on the performance monitoring indicator, the employee identification and the Internet product identification of his / her work, collect product data within a corresponding period from a first data source storing product data; Calculating the quality monitoring indicator data of the employee's efficiency based on the efficiency monitoring indicator and the product data; and The quality monitoring indicator data of the employee effectiveness is stored in a second data source, wherein the second data source is isolated from the first data source.

4. The method according to any one of claims 1 to 3, characterized in that: When a monitoring task fails to be executed, the monitoring task is recorded in the first data table; correspondingly, the first data table is queried when the monitoring task is executed, and the monitoring task in the first data table is added to the currently executed task list.

5. The method according to any one of claims 1 to 3, characterized in that: When executing the monitoring task, when collecting data from the first data source, based on the corresponding quality monitoring indicators, atomic data that meets the dimensions required for indicator calculation is collected.

6. The method according to any one of claims 1 to 3, characterized in that: Further including: Applying for an authentication token for data collection from a first data source; Correspondingly, when collecting data from the first data source, permission verification is performed with the first data source through the authentication token, and data is collected after passing the permission verification.

7. The method according to claim 3, characterized in that The calculation of quality monitoring indicator data further includes: Determine the corresponding calculation expression and original item data based on the quality monitoring indicators; Extracting collected data corresponding to the original item data from the collected data and performing aggregation operations on the collected data, wherein the aggregation operations include one or more of mean operations, maximum operations, median operations, sum operations, minimum operations, count operations, non-repeated value operations, and standard deviation operations; The aggregation operation result is calculated accordingly according to the calculation expression to obtain the corresponding quality monitoring index data, and the quality monitoring index includes product quality monitoring index and employee efficiency quality monitoring index data.

8. The method according to any one of claims 1 to 3, characterized in that: The quality monitoring indicators include grade attributes graded according to importance.

9. The method according to claim 1, characterized in that: When the quality monitoring indicator data is calculated based on the quality display indicator to obtain the quality display indicator data, the obtained quality monitoring indicator data is stored in the cache. When the same quality monitoring indicator data is applied to the calculation of different quality display indicators, the corresponding quality monitoring indicator data is obtained from the cache.

10. A quality monitoring and operation system for Internet products, characterized in that: include: A task creation module, configured to create a corresponding product monitoring task for each Internet product; A monitoring module, which is connected to the task creation module and is configured to monitor the current life cycle stage of each Internet product and its product monitoring task execution conditions; a task module connected to the monitoring module and configured to execute the product monitoring task in response to the product monitoring task satisfying the execution condition to obtain the corresponding product quality monitoring indicator data; and A data presentation module, connected to the task module and configured to calculate the quality monitoring indicator data based on the quality presentation indicator to obtain the quality presentation indicator data; Wherein, the task module includes: an identification unit configured to determine a first data source storing product data and corresponding product quality monitoring indicators based on an identification of the Internet product and its current life cycle stage; a collection unit connected to the identification unit and configured to collect current stage data in the Internet product data from the first data source; A calculation unit is connected to the identification unit and the collection unit respectively, and is configured to calculate the quality monitoring index data of the product at the current stage based on the product quality monitoring index and the collected stage data; and store the quality monitoring index data of the product in a second data source, wherein the second data source is isolated from the first data source.

11. The system according to claim 10, characterized in that The task creation module further includes: A first task creation unit, configured to create a corresponding product monitoring task for each Internet product; and A second task creation unit, configured to create an employee performance monitoring task based on employee data, wherein the employee data includes an employee identification and a corresponding Internet product for which the employee works; Correspondingly, the collection unit also collects product data within the corresponding period from a first data source storing product data based on the performance monitoring indicators, employee identification and the Internet product identification of his / her work; the calculation unit calculates the quality monitoring indicator data of the employee's performance based on the performance monitoring indicators and the product data.

12. An electronic device, comprising a processor and a memory, wherein a computer program instruction set is stored in the memory, characterized in that: When the processor executes the computer program instruction set on the memory, the quality monitoring operation method of the Internet product described in any one of claims 1-9 is implemented.

13. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program instruction set, wherein the computer program instruction set, when executed by a processor, implements the quality monitoring operation method for an Internet product as described in any one of claims 1-9.

14. A computer program product comprising a set of computer program instructions, characterized in that: When the computer program instruction set is executed by the processor, the quality monitoring operation method of the Internet product described in any one of claims 1-9 is implemented.

Images