Product quality evaluation method, system and equipment of service platform and medium

Abstract

The invention relates to a product quality evaluation method and system for a service platform, and the method comprises the steps: determining a consistency index, an operability index and a performance index of the service platform through obtaining the product data, user use data and platform operation data of the service platform; finally, weighted scoring is carried out on each index according to the importance degree of each index, the product quality of the service platform is determined, the method can help the product to carry out continuous and effective quality evaluation from multiple dimensions of product itself, user use and product use, an optimization scheme is fed back to a product decision-making party and an execution party in a quantitative form, and the product quality evaluation efficiency is improved. And the product quality is determined based on the data of the user and the product, so that the user experience of the product is scientifically and efficiently improved, and the method has great significance for product development and upgrading.

Description

Technical Field

[0001] The present invention belongs to the field of computer service technology, and in particular, relates to a product quality assessment method, system, electronic device and storage medium for a service platform. Background Art

[0002] With the development of Internet information technology and the rapid growth of the civil aviation travel market, the black box (command line interface) flight control system can no longer meet the needs of the existing market. In order to ensure the operation scenarios of airlines conducting a large number of business interactions between flights, and to support the needs of airlines and business departments to control and manage business and sell tickets and additional services through a graphical interface, the new generation of passenger service system reasonably integrates functional business processes and combines it with a new graphical display method to display the various functions of the new generation of passenger service system platform to airline users in a simple and intuitive way.

[0003] During the development and production of the new generation of passenger service system platform, we focused on the construction of product functions. During the production process, airlines continuously put forward demands for improved experience. The personalized needs of each airline vary greatly, resulting in instability in the passenger service platform version development plan and adding a lot of additional communication costs. We are unable to scientifically and efficiently understand the specific usage of the passenger service platform and reasonably plan product function iteration plans based on local feedback from airlines.

[0004] Therefore, how to scientifically and effectively evaluate the product quality of passenger service platforms has become a problem in the civil aviation industry. Summary of the invention

[0005] To solve the above problems, the present invention provides a product quality assessment method, system, electronic device and storage medium for a service platform, which conducts a multi-dimensional assessment of the service platform based on the service platform's page data, user usage data and platform operation data, and then determines the service platform product quality based on the multi-dimensional assessment indicators. This can more accurately and efficiently determine the service platform's product quality, provide support for the update of the service platform, and enhance the user experience.

[0006] In order to solve the above technical problems, the first aspect of the present invention proposes a product quality evaluation method for a service platform, the method comprising:

[0007] Obtain page data, user usage data, and platform operation data of the service platform;

[0008] Performing a consistency check on the service platform according to the page data to determine a consistency index of the service platform;

[0009] Generating operability indicators of the service platform based on the user usage data;

[0010] Determine the execution efficiency of tasks completed by users in the service platform based on the platform operation data, and generate performance indicators of the service platform according to the execution efficiency;

[0011] According to the importance of the consistency index, the operability index and the performance index, the consistency index, the operability index and the performance index are weighted and scored to evaluate the product quality of the service platform.

[0012] According to a preferred embodiment of the present invention, the step of performing a consistency check on the service platform according to the page data to determine a consistency index of the service platform includes:

[0013] Obtaining visual design information, interactive feedback information, and special page information from the page data;

[0014] According to the preset inspection criteria, the inspection points in the visual design information, the interactive feedback information and the special page information are respectively inspected to determine the inspection results of each of the inspection points;

[0015] A consistency index of the service platform is generated according to the inspection result.

[0016] According to a preferred embodiment of the present invention, generating the operability index of the service platform based on the user usage data includes:

[0017] Obtaining users' acceptance ratings of the service platform after using the service platform;

[0018] According to the preset acceptance scoring standard, each of the acceptance scores is standardized and summarized to obtain the user's acceptance index of the service platform as the operational index.

[0019] According to a preferred embodiment of the present invention, generating the operability index of the service platform based on the user usage data includes:

[0020] Obtaining at least two user ratings of the ease of use of the service platform;

[0021] Based on the objectivity of each of the user types, a weighted sum is taken for each of the operability scores to obtain the user's operability index for the service platform as the operability index.

[0022] According to a preferred embodiment of the present invention, determining the execution efficiency of users completing tasks in the service platform based on the platform operation data, and generating performance indicators of the service platform according to the execution efficiency, includes:

[0023] Obtaining platform operation data of tasks performed by each user in the service platform;

[0024] Extracting historical platform operation data of historical users performing tasks before the service platform is updated from the historical data;

[0025] Determine the time variation of each user completing a task based on the platform operation data and the historical platform operation data;

[0026] The execution efficiency of the service platform is calculated based on the time change, the actual number of events in which each user performs task operations in the platform operation data, and the historical number of events in which historical users perform task operations in the historical platform operation data, as a performance indicator of the service platform.

[0027] According to a preferred embodiment of the present invention, determining the execution efficiency of users completing tasks in the service platform based on the platform operation data, and generating performance indicators of the service platform according to the execution efficiency, includes:

[0028] Acquire platform operation data of each user performing different preset tasks in the service platform;

[0029] Extracting the task execution time of the user performing different preset tasks from the platform operation data, and scoring each task execution time;

[0030] The scoring results are weighted according to the types of preset tasks to obtain the execution efficiency as the performance indicator of the service platform.

[0031] According to a preferred embodiment of the present invention, the method further comprises:

[0032] Obtaining importance scores of different users on the consistency index, operability index, and performance index;

[0033] According to each of the importance scores, the importance of the consistency index, the operability index and the performance index is statistically obtained.

[0034] In order to solve the above technical problems, the second aspect of the present invention proposes a product quality evaluation system for a service platform, the system comprising:

[0035] Data acquisition module, used to obtain page data, user usage data and platform operation data of the service platform;

[0036] A first evaluation module, configured to perform a consistency check on the service platform according to the page data to determine a consistency index of the service platform;

[0037] A second evaluation module, used to generate an operability index of the service platform based on the user usage data;

[0038] A third evaluation module is used to determine the execution efficiency of users completing tasks in the service platform based on the platform operation data, and generate a performance indicator of the service platform according to the execution efficiency;

[0039] The comprehensive evaluation module is used to weight the consistency indicators, operability indicators and performance indicators according to their importance, so as to evaluate the product quality of the service platform.

[0040] In order to solve the above technical problem, the third aspect of the present invention provides an electronic device, comprising:

[0041] Processor; and

[0042] A memory storing computer executable instructions, wherein the computer executable instructions, when executed, cause the processor to perform any one of the methods described in the above embodiments.

[0043] In order to solve the above technical problems, the fourth aspect of the present invention proposes a computer storage medium, wherein the computer storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method described in any one of the above embodiments is implemented.

[0044] Compared with the prior art, the present invention has the following advantages: the present invention determines the consistency index, operability index and performance index of the service platform by obtaining the product data, user usage data and platform operation data of the service platform, and finally weights and scores each index according to its importance to determine the product quality of the service platform. This method can help the product to conduct continuous and effective quality assessment from multiple dimensions such as the product itself, user usage and product usage, and feed back the optimization plan to the product decision makers and executors in a quantitative form, determine the product quality based on the user and product data, and then scientifically and efficiently improve the product user experience, which is of great significance to product development and upgrading.

[0045] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or be understood by implementing the present disclosure. The purpose and other advantages of the present disclosure can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A schematic diagram of a product quality assessment method for a service platform according to an embodiment of the present disclosure is shown;

[0048] Figure 2 A first flow chart of a method for evaluating product quality on a service platform according to an embodiment of the present disclosure is shown;

[0049] Figure 3 A second flow chart of a method for evaluating product quality on a service platform according to an embodiment of the present disclosure is shown;

[0050] Figure 4 A third flowchart of a method for evaluating product quality on a service platform according to an embodiment of the present disclosure is shown;

[0051] Figure 5 A fourth flowchart of a method for evaluating product quality on a service platform according to an embodiment of the present disclosure is shown;

[0052] Figure 6 A fifth flow chart of a method for evaluating product quality of a service platform according to an embodiment of the present disclosure is shown;

[0053] Figure 7 A sixth flowchart of a method for evaluating product quality on a service platform according to an embodiment of the present disclosure is shown;

[0054] Figure 8 A block diagram of a product quality evaluation system of a service platform according to an embodiment of the present disclosure is shown;

[0055] Fig. 9 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0057] The same reference numerals in the drawings represent the same or similar elements, components or parts, and thus the repeated description of the same or similar elements, components or parts may be omitted below. It should also be understood that although the first, second, third and other attributives representing the numbers may be used herein to describe various devices, elements, components or parts, these devices, elements, components or parts should not be limited by these attributives. In other words, these attributives are only used to distinguish one from another. For example, the first device may also be called the second device, but it does not deviate from the essential technical solution of the present invention. In addition, the terms "and / or" and "and / or" refer to all combinations including any one or more of the listed items.

[0058] See also Figure 1 , Figure 1 is a flow chart of a product quality assessment method of a service platform provided by the present invention, such as Figure 1 As shown, the method includes:

[0059] S11. Obtaining page data, user usage data, and platform operation data of the service platform.

[0060] In this embodiment, the service platform is a relatively complete form of e-commerce based on and serving a certain service industry. It mainly connects consumers and a certain service industry, enables better communication between them, and enables the service industry to serve consumers in a timely manner. Currently, the more mature ones include life service platforms, passenger service systems, shopping service platforms, public service platforms, cloud service platforms, etc.

[0061] In this embodiment, the product quality of the service platform is evaluated through multiple dimensions such as the page status of the service platform, user usage, and the status of the platform operation. That is, the product quality of the service platform is evaluated from the quality of the service platform itself, the convenience of user use, and the status of the platform operation. It includes various situations that users may encounter in the process of using the service platform. It is both an evaluation of the quality of the service platform and an evaluation of the user experience. It provides evaluation indicators for the development and update of the service platform, and promotes the service platform to provide users with a better service experience.

[0062] In this embodiment, the page data includes: design standards, information structure, text, icons, pictures, controls, feedback mechanism, operating specifications, user guidance, framework layout, process, network anomalies, server anomalies, empty state and other standard page data.

[0063] In this embodiment, the user usage data includes: acceptance evaluation filled in by the user after using the service platform, operability evaluation of each space filled in by the user after using the service platform, and other user usage data.

[0064] In this embodiment, the platform operation data includes: the time it takes for users to complete various tasks in the service platform at different version stages, the loading time of various pages of the service platform at different version stages, the response time of users' requests in the service platform at different version stages, etc.

[0065] S12. Perform a consistency check on the service platform based on the page data to determine the consistency index of the service platform.

[0066] In this embodiment, a consistency check of the service platform is completed based on the page data to determine the consistency index of the service platform. Specifically, after obtaining the value or display result of each page data, a check standard list of the service platform is obtained, and the value or display result of each page data is compared based on each standard in the check standard list. The consistency index is calculated based on the difference between each page data and the corresponding standard and the importance of each page data.

[0067] S13. Generate operational indicators of the service platform based on user usage data.

[0068] In this embodiment, the operability of the service platform is determined based on user usage, for example, the service platform does not need to be so complicated, the service platform is easy to operate, the different functions in the service platform are well combined, the service platform is too inconsistent, and I think most people can quickly learn to use the service platform.

[0069] In this embodiment, the user's evaluation of the service platform after use can be evaluated using a multi-level quantitative standard, such as strongly oppose, generally oppose, neutral, generally agree, strongly agree, or, with level 1 representing strong oppose and level 5 representing strong agree, and then setting standards 2, 3, and 4 as evaluation criteria.

[0070] S14. Determine the execution efficiency of users completing tasks in the service platform based on the platform operation data, and generate performance indicators of the service platform according to the execution efficiency.

[0071] In this embodiment, the execution efficiency of each user in completing each task in the service platform is determined based on the data of platform operation to determine the performance index of the service platform. For example, if the time consumed to complete the same task in the service platform is longer before and after the service platform is updated, it means that the performance of the updated service platform has declined. Conversely, it means that the performance of the service platform has improved. Specifically, for example, in the passenger service platform, if the time consumed by the user to order a ticket before the update is longer than the time consumed by the user to order a ticket after the update, it means that the performance of the service platform has declined. Of course, it may also be because the time has changed due to unfamiliarity with the operation after the service platform is updated. Therefore, after the service platform has been updated for a period of time, the time spent by users on tasks can be collected and compared with the time spent on the same tasks before the service platform was updated, so as to determine the performance of the service platform. Performance indicators can also be determined for new users. For example, before the service platform is updated, the time spent by any new user on a task can be obtained, and then after the service platform is updated, the time spent by the same user on the task can be obtained, or the time spent by another new user on the task can be obtained, so as to determine the performance of the service platform. If the amount of data is sufficient, the performance indicators of the service platform can be effectively verified through the above method.

[0072] S15. According to the importance of consistency indicators, operability indicators and performance indicators, weighted scores are given to consistency indicators, operability indicators and performance indicators to evaluate the product quality of the service platform.

[0073] In this embodiment, according to the importance of each indicator obtained in the above steps, a weighted score is given to each indicator to achieve a quantitative score for the product quality of the service platform.

[0074] In this embodiment, before weighted scoring is performed on each indicator, each indicator may be standardized or normalized so that the dimensions of each indicator are the same, the calculation process is simplified, and the consistency and comparability of data processing are ensured.

[0075] In this embodiment, the consistency index, operability index and performance index of the service platform are determined by obtaining the product data, user usage data and platform operation data of the service platform, and finally each index is weighted and scored according to its importance to determine the product quality of the service platform. This method can help products to conduct continuous and effective quality assessment, and feed back the optimization plan to the product decision-makers and executors in a quantitative form, determine product quality based on user and product data, and then scientifically and efficiently improve the product user experience, which is of great significance to product development and upgrading.

[0076] like Figure 2As shown, in this embodiment, the method further includes:

[0077] S21. Obtain the importance scores of consistency indicators, operability indicators, and performance indicators from different users.

[0078] In this embodiment, a questionnaire survey can be conducted on the service platform based on various indicators, and the importance score can be obtained by summarizing the user's scores on the questionnaire survey on each indicator. The weights of different indicators can be allocated from the user's perspective, thereby promoting the service platform to meet the user's usage experience.

[0079] In this embodiment, tests can be conducted based on changes in the service platform to determine the importance preferences of different users for each indicator. For example, after different versions of the service platform are updated, different users are invited to use different versions of the service platform and score each version of the service platform respectively. Based on the scores at this time and the differences in the indicators updated by each version of the service platform, the importance preferences of each user for each indicator are comprehensively calculated, and then the corresponding importance scores are obtained. This solution can determine the user's preference for each indicator of the service platform by means of a neural network model. For example, after determining the change amount of the indicator in the above manner, the change amount of each indicator is used as the input of the training sample, and the score of each user for the currently used service platform is collected synchronously, and the score is used as the output of the training sample. Based on the training sample, a neural network model for identifying the score of the service platform is obtained. Based on the neural network model, the influence of the change amount of different indicators on the final score is determined, and the influence amount corresponding to the influence is the user's importance score for each indicator in the service platform.

[0080] S22. According to each importance score, the importance of the consistency index, the operability index and the performance index are statistically obtained.

[0081] In this embodiment, the importance scores of each user for each indicator are summarized, and the importance of each indicator is determined according to the total importance score of each indicator. Specifically, the total importance score of the indicator can be directly used as the importance, or the total importance score of each indicator can be normalized and the normalized value can be used as the importance.

[0082] In a specific embodiment, the operational indicators in this solution include: acceptance indicators and ease of operation indicators. The performance indicators include: task efficiency indicators for common tasks and product performance indicators for preset tasks. Taking the use of airline software tool products as an example, we obtain the importance of each dimension, that is, the weight relationship, through testing and questionnaire scale weight scoring, and finally obtain the product quality of the service platform.

[0083]

[0084] H is the number of questionnaires collected; i is the i-th questionnaire; E is the score system used in the questionnaire;

[0085] is the weight score of the consistency indicator, is the weight score of the acceptance indicator, is the weight score of the operability indicator, is the weight score of the task efficiency index, Provide weighted scores for product performance indicators;

[0086] f1 is the weight of the consistency index, f2 is the weight of the acceptance index, f3 is the weight of the ease of operation index, f4 is the weight of the task efficiency index, and f5 is the weight of the product performance index.

[0087] like Figure 3 As shown, a flow chart of a product quality assessment method of a service platform provided by the present invention is similar to Figure 1 Compared with the embodiment shown, the difference is that the method includes:

[0088] S31. Obtain visual design information, interactive feedback information, and special page information from the page data.

[0089] In this embodiment, visual design information includes design standards, information structure, text, icons, images, controls and other information in the page data; interactive feedback information includes: feedback mechanism, operating specifications, user guidance, framework layout, process and other information; special page information includes: network abnormalities, server abnormalities, empty state and other information.

[0090] S32. According to the preset inspection standard, the inspection points in the visual design information, the interactive feedback information and the special page information are respectively inspected to determine the inspection results of each inspection point.

[0091] In this embodiment, the inspection standard may be various indicators in the inspection standard list of the service platform. The parameters in various information are compared and inspected according to the inspection standard to determine the inspection points with problems.

[0092] In this embodiment, the relative deviations of the parameters in each information and the inspection standard can be calculated respectively, and the checkpoints with relative deviations greater than the preset deviations can be determined as checkpoints with problems. The above information and inspection standards can also be converted into data vectors respectively, and the differences between the data vectors can be calculated, such as by using Euclidean distance or cosine value to calculate the differences, and the differences can be compared with corresponding thresholds to determine the checkpoints with differences that do not meet the thresholds as checkpoints with problems.

[0093] S33. Generate consistency indicators of the service platform based on the inspection results.

[0094] In this embodiment, the consistency index of the service platform is generated according to the inspection results of each inspection point, specifically including:

[0095] S1=c1S 11 +c2S 12 +c3S 13

[0096]

[0097] The specific interpretation of the formula of product consistency index is: S1 is the consistency index, S 11 is the consistency score corresponding to the interactive feedback information, S 12 is the consistency score corresponding to the visual design information, S 13 Scoring the consistency of information corresponding to a particular page;

[0098] n1 is the total number of problem check items in the interactive feedback information, n2 is the total number of problem check items in the visual design information, and n3 is the total number of problem check items in the special page information;

[0099] N1 is the number of inspection dimension item indicators checked in the interactive feedback information, N2 is the number of inspection dimension item indicators checked in the visual design information, and N3 is the number of inspection dimension item indicators checked in the special page information;

[0100] m is the weight of the product non-compliance rate (m∈(0,1));

[0101] k1 is the k1th check dimension item checked (k1∈[1,N1] integer); k2 is the k2th check dimension item checked (k2∈[1,N2] integer); k3 is the k3th check dimension item checked (k3∈[1,N3] integer); Weight for the k1th check dimension item Weight for the k2th check dimension item Check the k3th dimension item weight Score the severity of the problem for the k1th inspection dimension item Score the severity of the problem for the k2th inspection dimension item Score the severity of the problem for the k3th inspection dimension item

[0102] c1 is the weight of the interactive feedback information, c2 is the weight of the visual design information, c3 is the weight of the special page information, and c1+c2+c3=1.

[0103] like Figure 4As shown, a flow chart of a product quality assessment method of a service platform provided by the present invention is similar to Figure 1 Compared with the embodiment shown, the difference is that the method includes:

[0104] S41. Obtain the user's acceptance rating of the service platform after using the service platform.

[0105] In this embodiment, the acceptance score can be obtained by obtaining the user's score on the service platform through a questionnaire survey. For example, when the user closes the service platform, a questionnaire pop-up window is popped up to display the questionnaire questions to the user and obtain the score filled in by the user. The user's acceptance of the service platform can also be obtained from data such as the user's frequency of use, the user's consumption on the service platform, whether the user has opened a membership on the service platform, the same type of APP clients downloaded by the user, or the same type of service platforms used by the user. For example, the higher the frequency of user use of the service platform, the higher the user's acceptance of the service platform. The user's opening of a membership on the service platform can also indicate that the user has a high degree of acceptance of the service platform. In this solution, a preset acceptance score can be determined for the opened membership.

[0106] S42. Based on the pre-set acceptance scoring standard, standardize and summarize the acceptance scores to obtain the user's acceptance index of the service platform as an operational index.

[0107] In this embodiment, the acceptance rating standard may be a scoring system used in a questionnaire survey. Specifically, for example, the multi-level quantitative standard used in the above embodiment corresponds to a 5-level scoring system.

[0108] Specifically, user acceptance of a product is determined by administering acceptance scale questionnaires and collecting scores from a large number of real users of different types, ensuring that the acceptance score data feedback is close to the actual situation of real users. The larger the user acceptance sample size, the more accurate the product acceptance index will be.

[0109] Specifically, the corresponding acceptance index of the product can be obtained from the questionnaire based on the following formula.

[0110]

[0111] The specific interpretation of the product acceptance index formula is:

[0112] S2 is the user acceptance index of the service platform; Xi is the total score of the i-th user acceptance scale questionnaire; z represents the number of indicators in the acceptance scale questionnaire; Y represents the total number of users who accept the questionnaire; i represents the i-th user (i∈[1,Y] integer); E represents the score system used in the scale.

[0113] like Figure 5As shown, a flow chart of a product quality assessment method of a service platform provided by the present invention is similar to Figure 4 Compared with the embodiment shown, the difference is that the method further comprises:

[0114] S51. Obtain at least two user ratings of the ease of use of the service platform.

[0115] In this embodiment, Figure 4 On the basis of the illustrated embodiment, in addition to the user's acceptance of the service platform which can be used as an operability indicator for evaluating the service platform, the operability indicator also includes an ease of operation score. This solution also provides a solution for evaluating the ease of operation of the service platform for users.

[0116] In this embodiment, the types of users may include: ordinary users, member users, invited test users, administrator users, and expert users. Different types of users score the operability of the service platform separately, thereby increasing the objectivity of the evaluation of the operability of the service platform and intuitively expressing the evaluation of different users on the service platform.

[0117] S52. Based on the objectivity of each user type, weighted sum is performed on each operability score to obtain the user's operability index for the service platform as the operability index.

[0118] In this embodiment, corresponding objectivity and corresponding weight values ​​can be configured for different types of users. In this step, the ease of operation scores made by different users are weighted according to their corresponding weights. For example, member users have high stickiness, often use the service platform, and can skillfully use various functions in the service platform. The objectivity of the ease of operation evaluation of the service platform made by them is lower; and administrator users, as developers or maintainers of the service platform, are more familiar with the service platform, so the objectivity of the ease of operation evaluation they give will be lower. Ordinary users, invited test users, and expert users will also give different evaluations due to their different identities. In general, the objectivity of expert users will be higher and higher than that of invited test users, and the objectivity of invited test users will also be higher than that of ordinary users. The objectivity of ordinary users will be higher. The objectivity of a completely objective user is higher than that of a member user, and the objectivity of a member user is higher than that of an administrator user. In this scheme, the weight of a completely objective user can be set to 1. According to the order of objectivity, the weight values ​​of different types of users are configured in a manner of decreasing the preset weights. Specifically, for example, if the decreasing preset weight is 0.1, the weight of an expert user is 0.9, the weight of an invited test user is 0.8, the weight of an ordinary user is 0.7, the weight of a member user is 0.6, and the weight of an administrator user is 0.5. Of course, this scheme can also use other methods to configure weights according to the objectivity of the user type, and this scheme does not specifically limit this.

[0119] Specifically, the ease of operation of the service platform is tested by some real users and expert users on system usage, questionnaires on the ease of operation scale are distributed, and scores are collected. The quantitative link of expert user scores is introduced here to increase the objectivity of the ease of operation of the service platform. The ease of operation index of the service platform is obtained based on the following formula.

[0120]

[0121] The specific interpretation of the product operability index formula is:

[0122] S3 is the operability index of the service platform, Oi is the total score of the i-th real user operability scale questionnaire; M represents the sample size of real users under test; Pj is the score of the j-th expert user operability scale questionnaire; W represents the sample size of expert users under test; z is the number of indicators in the operability scale questionnaire. i represents the i-th real user under test (i∈[1,M] integer); j represents the j-th expert user under test (j∈[1,W] integer); d is the corresponding weight when the user is a real user, and E represents the score used in the scale.

[0123] like Figure 6 As shown, a flow chart of a product quality assessment method of a service platform provided by the present invention is similar to Figure 1 Compared with the embodiment shown, the difference is that the method includes:

[0124] S61. Obtain platform operation data of each user performing tasks in the service platform.

[0125] In this embodiment, the tasks executed in the service platform may be the workflows performed by users in the service platform. For example, the tasks may be user login, product ordering, payment process jump, activity participation process, etc.

[0126] S62: extracting historical platform operation data of historical users performing tasks before the service platform is updated from the historical data.

[0127] In this embodiment, historical platform operation data of historical users performing tasks before the service platform is updated is extracted from past historical data. The historical platform operation data is the platform operation data stored when the tasks are performed before the service platform is updated.

[0128] In this embodiment, the service platform update includes updates to the various functions of the service platform, the parameters of each control, the overall layout of the page, etc. The operation data before the service platform update can be used to evaluate the task efficiency of the operation data of the current service platform after the update, and then evaluate the service efficiency before and after the service platform update.

[0129] S63. Determine the time variation for each user to complete the task based on the platform operation data and the historical platform operation data.

[0130] In this embodiment, the time variation of the time taken to complete each task is determined through historical platform operation data and current platform operation data. This time variation can effectively illustrate the service efficiency of the service platform before and after the update. If the time variation shown is a negative number, it means that the updated service platform can provide relevant services to users more effectively. Otherwise, it means that the updated service platform is not as good as the service platform before the update. At this time, the service platform needs to be corrected in a timely manner to ensure the user experience.

[0131] In this embodiment, since the service platform may be unfamiliar before and after the update, or different users may have different levels of operation on the service platform, this solution can obtain platform operation data or historical platform operation data according to the solution mentioned in step S14, and this solution will not be repeated here.

[0132] S64. Calculate the execution efficiency of the service platform as a performance indicator of the service platform based on the time variation, the actual number of events of each user performing task operations in the platform operation data, and the historical number of events of historical users performing task operations in the historical platform operation data.

[0133] In this embodiment, the execution efficiency of the service platform is calculated based on the change in time and the number of events operated by the previous and subsequent executions of the task. In this solution, the more events are operated by the execution of the task, the longer the task process becomes. Combined with the change in time, the task execution efficiency of the current service platform can be determined.

[0134] The task efficiency of the service platform refers to the task completion rate and task completion efficiency. By connecting to the data tracking system, for clear operation paths or user preferred operation paths, after establishing tasks, the efficiency of users completing tasks can be obtained through the collected data indicators. By comparing the difference between the user path and the ideal path of product design, the smaller the difference, the higher the efficiency. Under the same conditions, make corresponding improvements to the operation path, and obtain the execution efficiency of the improved service platform based on the following formula.

[0135]

[0136] The interpretation of the execution efficiency formula of the service platform is as follows:

[0137] S4 is the task efficiency index of common tasks, M represents the actual number of users collected; H iis the actual number of events operated by the i-th user in the service platform before improvement; i represents the i-th user; G i represents the actual number of events operated by the i-th user in the improved service platform; F H represents the number of task indicators in the service platform before improvement, F G Represents the number of task indicators in the improved service platform; T H represents the task completion time in the service platform before improvement, T G represents the task completion time in the improved service platform; V represents the ideal improved efficiency of the service platform after the task is improved.

[0138] like Figure 7 As shown, a flow chart of a product quality assessment method of a service platform provided by the present invention is similar to Figure 6 Compared with the embodiment shown, the difference is that the method further comprises:

[0139] S71. Obtain platform operation data of each user performing different preset tasks in the service platform.

[0140] In this embodiment, Figure 6 On the basis of the illustrated embodiment, in addition to the execution efficiency index of the user for the non-fixed task being used as the performance index for evaluating the service platform, the performance index also includes the product performance index of the service platform for the preset task.

[0141] In this embodiment, the preset tasks may be the first screen rendering time (FMP, the time from when the user sends a request to when the user sees the main content of the console, the page request response time (the time from when the user sends a request to when the server responds), the request response time (API, when the console requests a business data interface, the time from when the request is sent to when the server responds), etc. The loading time of these situations can give users different usage experiences, and can also be used to evaluate product quality. Specifically, the longer the loading time, the worse the user experience, and the worse the quality of the product for the user.

[0142] S72: extracting the task execution time of the user performing different preset tasks from the platform operation data, and scoring each task execution time.

[0143] In this embodiment, the task execution time of each preset task of the platform is directly obtained, and each task execution time is scored accordingly. For example, the execution time of each preset task can be scored by popping up or sending a scoring sheet to the user when the user completes the preset task.

[0144] Specifically, users can set time standards for each preset task based on experience, and score the task based on the deviation between the actual execution time and the time standard. Users can also determine the time standard based on the average execution time of each preset task in historical data, and then score the task. Users can also obtain the average execution time of each preset task before the service platform is updated, and use the relative optimization amount of the current task execution time and the average execution time as the score.

[0145] S73. Perform weighted calculation on the scoring results according to the types of preset tasks to obtain execution efficiency as a performance indicator of the service platform.

[0146] In this embodiment, the scoring results are weighted according to the types of preset tasks to obtain the execution efficiency. In this solution, the weight of the preset tasks can be assigned according to the importance of the preset tasks. For example, the homepage rendering is generally available when the APP is opened, so its weight will be relatively low, while the page request response time and the data request response time will have higher weights. Among them, when the user uses it, the speed of page jump is more important than the speed of data response, so the weight of the page request response time is higher than the weight of the data request impact time.

[0147] Product performance indicators, obtained through tracking tools, product usage data, and user performance testing and scale scoring, including first screen rendering time (FMP, the time from when the user sends a request to when the user sees the main content of the console), page request response time (the time from when the user sends a request to when the server responds), and request response time (API, when the console requests the business data interface, the time from when the request is sent to when the server responds). Specifically include: Product page rendering time score S 51 , Page request response time score S 52 and API request response time score S 53 Three modules, product performance indicators are obtained based on the following formula.

[0148] S5=d1*S 51 +d2*S 52 +d3*S 54

[0149]

[0150] S5 is the performance indicator of the product, M is the number of page rendering and loading, L is the number of page requests, and N is the number of API requests; The performance score for the i-th page rendering loading time; The performance score for the i-th page request time; The performance score for the i-th API request time; i mRepresents the i-th page rendering and loading; i l Represents the i-th page request; i n represents the i-th API request; E represents the score used in the scale;

[0151] d is the weight of the inspection module, d1 is the weight of the page loading and rendering time, d2 is the weight of the page request response, d3 is the weight of the API response time, d1+d2+d3=1.

[0152] In the above embodiment, multiple different formulas are mentioned. Since users can score corresponding dimensions when scoring different indicators, the same parameter symbols exist in different formulas to represent parameters when scoring different dimensions. There will be no incomprehensible situation between the above parameters. In subsequent modifications, if different formulas need to be merged into one solution, the same parameter symbols can be added with lower right numbers 1, 2, 3... in order. The specific interpretation of the parameter symbols can refer to the interpretation below the formula.

[0153] See also Figure 8 , Figure 8 is a block diagram of a product quality evaluation system of a service platform provided by the present invention, such as Figure 8 As shown, the system includes: a data acquisition module 11, a first evaluation module 12, a second evaluation module 13, a third evaluation module 14 and a comprehensive evaluation module 15.

[0154] In this embodiment, the data acquisition module 11 is used to acquire page data, user usage data and platform operation data of the service platform.

[0155] In this embodiment, the first evaluation module 12 is used to perform a consistency check on the service platform according to the page data to determine the consistency index of the service platform.

[0156] In this embodiment, the second evaluation module 13 is used to generate an operability index of the service platform based on user usage data.

[0157] In this embodiment, the third evaluation module 14 is used to determine the execution efficiency of users completing tasks in the service platform based on the platform operation data, and generate performance indicators of the service platform according to the execution efficiency.

[0158] In this embodiment, the comprehensive evaluation module 15 is used to weight the consistency indicators, operability indicators and performance indicators according to their importance, so as to evaluate the product quality of the service platform.

[0159] In this embodiment, the first evaluation module 12 is specifically used to obtain visual design information, interactive feedback information and special page information in the page data; according to the pre-set inspection standards, the check points in the visual design information, interactive feedback information and special page information are respectively inspected to determine the inspection results of each check point; and the consistency index of the service platform is generated according to the inspection results.

[0160] In this embodiment, the second evaluation module 13 is specifically used to obtain the user's acceptance score of the service platform after using the service platform; according to the pre-set acceptance score standard, each acceptance score is standardized and summarized to obtain the user's acceptance index of the service platform as an operational index.

[0161] In this embodiment, the second evaluation module 13 is specifically used to obtain at least two types of user operability scores for the service platform; based on the objectivity of each user type, each operability score is weighted and summed to obtain the user's operability index for the service platform as an operability index.

[0162] In this embodiment, the third evaluation module 14 is specifically used to obtain the platform operation data of each user performing tasks in the service platform; extract the historical platform operation data of historical users performing tasks before the service platform is updated from the historical data; determine the time change for each user to complete the task based on the platform operation data and the historical platform operation data; calculate the execution efficiency of the service platform as a performance indicator of the service platform based on the time change, the actual number of events of each user performing task operations in the platform operation data, and the historical number of events of historical users performing task operations in the historical platform operation data.

[0163] In this embodiment, the third evaluation module 14 is specifically used to obtain the platform operation data of each user performing different preset tasks in the service platform; extract the task execution time of the user performing different preset tasks from the platform operation data, and score each task execution time respectively; perform weighted calculation on the scoring results according to the type of preset tasks to obtain the execution efficiency as a performance indicator of the service platform.

[0164] In this embodiment, the comprehensive evaluation module 15 is also used to obtain the importance scores of different users for the consistency index, operability index and performance index; according to each importance score, the importance of the consistency index, operability index and performance index is statistically obtained.

[0165] like Fig. 9As shown, an embodiment of the present invention provides an electronic device, including a processor 1110, a communication interface 1120, a memory 1130 and a communication bus 1140, wherein the processor 1110, the communication interface 1120, and the memory 1130 communicate with each other through the communication bus 1140;

[0166] Memory 1130, used for storing computer programs;

[0167] The processor 1110 is configured to implement any of the above-mentioned evaluation methods when executing the program stored in the memory 1130 .

[0168] In the electronic device provided by the embodiment of the present invention, the processor 1110 obtains page data, user usage data and platform operation data of the service platform by executing the program stored in the memory 1130; performs a consistency check on the service platform according to the page data to determine the consistency index of the service platform; generates operational indicators of the service platform based on the user usage data; determines the execution efficiency of users completing tasks in the service platform based on the platform operation data, and generates performance indicators of the service platform according to the execution efficiency; and performs weighted scoring on the consistency index, operational index and performance index according to their importance to evaluate the product quality of the service platform.

[0169] The communication bus 1140 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0170] The communication interface 1120 is used for communication between the above electronic device and other devices.

[0171] The memory 1130 may include a random access memory 1130 (RAM), or may include a non-volatile memory 1130 (non-volatile memory), such as at least one disk memory 1130. Optionally, the memory 1130 may also be at least one storage device located away from the processor 1110.

[0172] The above-mentioned processor 1110 can be a general-purpose processor 1110, including a central processing unit 1110 (CPU), a network processor 1110 (NP), etc.; it can also be a digital signal processor 1110 (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0173] An embodiment of the present invention provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors 1110 to implement the evaluation method of any of the above embodiments.

[0174] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidStateDisk (SSD)), etc.

[0175] Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A product quality evaluation method for a service platform, characterized in that: The method comprises: Obtain page data, user usage data, and platform operation data of the service platform; Performing a consistency check on the service platform according to the page data to determine a consistency index of the service platform; Generating operability indicators of the service platform based on the user usage data; Determine the execution efficiency of tasks completed by users in the service platform based on the platform operation data, and generate performance indicators of the service platform according to the execution efficiency; According to the importance of the consistency index, the operability index and the performance index, the consistency index, the operability index and the performance index are weighted and scored to evaluate the product quality of the service platform.

2. The method according to claim 1, characterized in that The performing a consistency check on the service platform according to the page data to determine a consistency index of the service platform includes: Obtaining visual design information, interactive feedback information, and special page information from the page data; According to the preset inspection criteria, the inspection points in the visual design information, the interactive feedback information and the special page information are respectively inspected to determine the inspection results of each of the inspection points; A consistency index of the service platform is generated according to the inspection result.

3. The method according to claim 2, characterized in that Generating the operability index of the service platform based on the user usage data includes: Obtaining users' acceptance ratings of the service platform after using the service platform; According to the preset acceptance scoring standard, each of the acceptance scores is standardized and summarized to obtain the user's acceptance index of the service platform as the operational index.

4. The method according to claim 3, characterized in that Generating the operability index of the service platform based on the user usage data includes: Obtaining at least two user ratings of the ease of use of the service platform; Based on the objectivity of each of the user types, a weighted sum is taken for each of the operability scores to obtain the user's operability index for the service platform as the operability index.

5. The method according to claim 1, characterized in that Determining the execution efficiency of tasks completed by users in the service platform based on the platform operation data, and generating performance indicators of the service platform according to the execution efficiency, includes: Obtaining platform operation data of tasks performed by each user in the service platform; Extracting historical platform operation data of historical users performing tasks before the service platform is updated from the historical data; Determine the time variation of each user completing a task based on the platform operation data and the historical platform operation data; The execution efficiency of the service platform is calculated based on the time change, the actual number of events in which each user performs task operations in the platform operation data, and the historical number of events in which historical users perform task operations in the historical platform operation data, as a performance indicator of the service platform.

6. The method according to claim 5, characterized in that Determining the execution efficiency of tasks completed by users in the service platform based on the platform operation data, and generating performance indicators of the service platform according to the execution efficiency, includes: Acquire platform operation data of each user performing different preset tasks in the service platform; Extracting the task execution time of the user performing different preset tasks from the platform operation data, and scoring each task execution time; The scoring results are weighted according to the types of preset tasks to obtain the execution efficiency as the performance indicator of the service platform.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Obtaining importance scores of different users on the consistency index, operability index, and performance index; According to each of the importance scores, the importance of the consistency index, the operability index and the performance index is statistically obtained.

8. A product quality evaluation system for a service platform, characterized in that: The system comprises: Data acquisition module, used to obtain page data, user usage data and platform operation data of the service platform; A first evaluation module, configured to perform a consistency check on the service platform according to the page data to determine a consistency index of the service platform; A second evaluation module, used to generate an operability index of the service platform based on the user usage data; A third evaluation module is used to determine the execution efficiency of users completing tasks in the service platform based on the platform operation data, and generate a performance indicator of the service platform according to the execution efficiency; The comprehensive evaluation module is used to weight the consistency indicators, operability indicators and performance indicators according to their importance, so as to evaluate the product quality of the service platform.

9. An electronic device, characterized in that: include: processor; as well as A memory storing computer executable instructions which, when executed, cause the processor to perform the method according to any one of claims 1-7.

10. A computer storage medium, characterized in that: in, The computer storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method of any one of claims 1 to 7 is implemented.

Images