A method, device and medium for service evaluation

Abstract

The present application relates to the technical field of service evaluation, and discloses a method, device and medium for service evaluation, including: obtaining simulated test traffic and real traffic of each service to be evaluated to obtain target traffic data, and extracting multi-dimensional index data from the target traffic data, and then assigning corresponding weights to the indicators of different dimensions according to preset rules, and calculating the evaluation score of each service to be evaluated based on the index data of different dimensions and the corresponding weights in order to select the best service. Therefore, the introduction of simulated test traffic avoids the inaccuracy of service evaluation when the real traffic data generated by the service is small, and when extracting index data, extracts index data of multiple dimensions from the target traffic data, so as to further improve the accuracy of service evaluation in multiple dimensions. At the same time, when calculating the service evaluation score, the corresponding weights are assigned to the index data of different dimensions according to the preset rules to improve the accuracy of service evaluation, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the technical field of service evaluation, and in particular to a method, device, and medium for service evaluation. Background Art

[0002] As technology continues to evolve, evaluating the performance of services in various scenarios, such as different software and AI models, is crucial to technological innovation and development. Accuracy is a key metric in evaluating services, while also balancing efficiency and ease of use.

[0003] Currently, the quality of model services is mainly evaluated by fitting the test set results and the validation set results. This method can reflect the quality of model services to a certain extent. However, when the data sets are different, there will be large deviations in the evaluation results, which will lead to low accuracy of the evaluation results.

[0004] Alternatively, Prometheus monitoring, which supports various types of monitoring metrics, can be used to evaluate services. This collects metrics, analyzes them, and stores the results in a real-time database, resolving the dataset source issue. However, Prometheus monitoring can only store time series data and lacks service evaluation capabilities. If service evaluation is required, additional packaging is required, increasing costs. Alternatively, the use of pre-built metrics to evaluate stored time series data can reduce evaluation accuracy.

[0005] It can be seen that how to improve the accuracy of service evaluation and thus enhance user experience is an urgent problem to be solved by technical personnel in this field. Summary of the Invention

[0006] The purpose of this application is to provide a method, device and medium for service evaluation to improve the accuracy of service evaluation and thereby enhance user experience.

[0007] To solve the above technical problems, this application provides a service evaluation method, including:

[0008] Obtain the simulated test traffic and actual traffic of each service to be evaluated to obtain the target traffic data;

[0009] Extracting multi-dimensional indicator data from the target traffic data;

[0010] Assign corresponding weights to indicators of different dimensions according to preset rules;

[0011] The evaluation score of each of the services to be evaluated is calculated based on the indicator data of different dimensions and the corresponding weights, so as to recommend the best service.

[0012] Preferably, the extracting multi-dimensional indicator data from the target traffic data includes:

[0013] Trigger multiple child threads;

[0014] Each of the sub-threads extracts the indicator data of different dimensions in parallel.

[0015] Preferably, before allocating corresponding weights to the indicator data of different dimensions according to preset rules, the method further includes:

[0016] The indicator data corresponding to the pre-built-in indicators and the indicator data corresponding to the custom indicators are respectively extracted from the target traffic data.

[0017] Preferably, the selecting the best service includes:

[0018] Sorting the evaluation scores of the target focus indicators in each of the services to be evaluated according to the preset indicator evaluation direction to obtain a ranking result; wherein, there is one target focus indicator;

[0019] The optimal service is determined from the ranking results according to the preset indicator evaluation direction.

[0020] Preferably, when there are multiple target focus indicators, the selecting of the optimal service includes:

[0021] Sorting the evaluation scores of the target focus indicators in each of the services to be evaluated according to the preset indicator evaluation direction to obtain a ranking result;

[0022] Setting corresponding scores for indicators of different rankings in the ranking results based on the indicator evaluation direction, and assigning corresponding weights to different target focus indicators according to the focus levels;

[0023] Calculate the final score of each service to be evaluated based on the scores corresponding to different rankings in the ranking results and the weights corresponding to different target focus indicators;

[0024] The service with the highest final score is taken as the optimal service.

[0025] Preferably, after extracting multi-dimensional indicator data from the target traffic data, the method further includes:

[0026] Eliminate data that does not fall within the preset range corresponding to the indicator data of each dimension.

[0027] Preferably, the extracting multi-dimensional indicator data from the target traffic data includes:

[0028] Determining whether the actual flow rate reaches a preset value;

[0029] If the preset value is reached, only the index data corresponding to the actual flow rate is extracted from the target flow rate data;

[0030] If the preset value is not reached, the index data corresponding to the simulated test flow and the real flow are extracted from the target flow data according to a preset ratio.

[0031] In order to solve the above technical problems, the present application also provides a service evaluation device, including:

[0032] The acquisition module is used to obtain the simulated test traffic and real traffic of each service to be evaluated to obtain the target traffic data;

[0033] An extraction module, configured to extract multi-dimensional indicator data from the target traffic data;

[0034] The allocation module is used to assign corresponding weights to indicators of different dimensions according to preset rules;

[0035] The calculation module is used to calculate the evaluation score of each of the services to be evaluated based on the indicator data of different dimensions and the corresponding weights, so as to recommend the best service.

[0036] In order to solve the above technical problems, the present application also provides a service evaluation device, including a memory for storing a computer program;

[0037] A processor is configured to implement the steps of the service evaluation method when executing the computer program.

[0038] In order to solve the above technical problems, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the service evaluation method are implemented.

[0039] The present invention provides a method for service evaluation, including: obtaining the simulated test traffic and real traffic of each service to be evaluated to obtain target traffic data, and extracting multi-dimensional index data from the target traffic data, and then assigning corresponding weights to the indicators of different dimensions according to preset rules, and calculating the evaluation score of each service to be evaluated based on the index data of different dimensions and the corresponding weights, and then selecting the best service based on the evaluation score. It can be seen that the technical solution provided by the present application introduces simulated test traffic to avoid reducing the inaccuracy of service evaluation when the real traffic data generated by the service is small. In addition, when extracting index data, index data of multiple dimensions are extracted from the target traffic data to further improve the accuracy of service evaluation in multiple dimensions. At the same time, when calculating the service evaluation score, corresponding weights are assigned to the index data of different dimensions according to preset rules, thereby obtaining an evaluation score based on a weight algorithm to improve the accuracy of service evaluation, thereby improving the user experience.

[0040] In addition, the present application also provides a device and medium for service evaluation, which correspond to the above-mentioned service evaluation method and have the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 A flowchart of a service evaluation method provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of another service evaluation method provided in an embodiment of the present application;

[0044] Figure 3 A structural diagram of a service evaluation device provided in an embodiment of the present application;

[0045] Figure 4 A structural diagram of a service evaluation device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0047] The core of this application is to provide a method, device and medium for service evaluation, introduce simulated test traffic, combine module test traffic and real traffic to perform service evaluation, and avoid affecting the evaluation accuracy when the real traffic generated by the service is small. At the same time, it obtains indicator data based on multiple dimensions and assigns corresponding weights to indicator data of different dimensions according to preset rules, so as to further improve the accuracy of service evaluation and enhance user experience.

[0048] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0049] As technology continues to evolve, evaluating the performance of services in various scenarios, such as different software and AI models, is crucial to technological innovation and development. Accuracy is a key metric in evaluating services, while also balancing efficiency and ease of use.

[0050] Currently, the quality of model services is mainly evaluated by fitting the test set results and the validation set results. This method can reflect the quality of model services to a certain extent. However, when the data sets are different, there will be large deviations in the evaluation results, which will lead to low accuracy of the evaluation results.

[0051] Alternatively, Prometheus monitoring, which supports various types of monitoring metrics, can be used to evaluate services. This collects metrics, analyzes them, and stores the results in a real-time database, resolving the dataset source issue. However, Prometheus monitoring can only store time series data and lacks service evaluation capabilities. If service evaluation is required, additional packaging is required, increasing costs. Alternatively, the use of pre-built metrics to evaluate stored time series data can reduce evaluation accuracy.

[0052] In order to improve the accuracy of service evaluation, an embodiment of the present application provides a service evaluation method, which combines the simulated test traffic and real traffic of the service to be evaluated to avoid affecting the accuracy of evaluation when the real traffic generated by the service is small. At the same time, indicator data in the traffic data is obtained from multiple dimensions, and corresponding weights are assigned to the indicator data of different dimensions to further improve the accuracy of evaluation and thereby improve the user experience.

[0053] Figure 1 A flowchart of a service evaluation method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0054] S10: Obtain the simulated test traffic and actual traffic of each service to be evaluated to obtain target traffic data;

[0055] In a specific embodiment, the service to be evaluated may generate less actual traffic. For example, when using HTTP service, the evaluation accuracy will be lower because the traffic data generated by users actually clicking to access the Internet is less. Therefore, the embodiment of the present application introduces simulated test traffic data, and the test traffic is obtained by simulating the actual business. During the test, the simulated test traffic and the actual traffic corresponding to each service to be evaluated are obtained to obtain the target traffic data, thereby ensuring the diversity of the test traffic entrance.

[0056] S11: extract multi-dimensional indicator data from target traffic data;

[0057] The core of this step is to extract multi-dimensional indicator data based on the target traffic data obtained in step S10, wherein the multi-dimensional indicators may include basic indicators, performance indicators, and SLO indicators, etc. For example, when the service to be evaluated is an HTTP service, the extracted indicator data may include response time, error rate, number of requests, delay percentage, etc. When the service to be evaluated is a model evaluation of an artificial intelligence scenario, the extracted indicators may include accuracy, error rate, precision, recall rate, ROC curve, AUC curve, PR curve, determination coefficient, and variance, etc. It should be noted that the specific types and quantities of dimensions to be extracted can be set according to actual business needs and are not limited in this application.

[0058] S12: assign corresponding weights to indicators of different dimensions according to preset rules;

[0059] After extracting indicators of different dimensions, assign corresponding weights to indicators of different dimensions according to preset rules, where the preset rules can be that users pre-set weights of different proportions based on the degree of attention paid to different indicators by actual business. For example, when the service to be evaluated is HTTP service, if the user's attention to response time, error rate, and number of requests decreases in sequence, then a weight of 0.6 is set for the response time, a weight of 0.3 is set for the error rate, and a weight of 0.1 is set for the number of requests. Of course, the impact of service quality can also be set according to different indicators, and this application does not limit the preset rules.

[0060] S13: Calculate the evaluation score of each service to be evaluated based on the indicator data of different dimensions and the corresponding weights, so as to recommend the best service.

[0061] Based on the indicator data and corresponding weights for different dimensions obtained in steps S11 and S12, the evaluation scores for each service to be evaluated are calculated, and the final service is selected based on the evaluation scores. In practice, after extracting the indicator data, each indicator data is first sorted, and corresponding scores are assigned to the indicator data with different rankings in the sorted results. The evaluation scores are then calculated based on the assigned scores and weights. To facilitate understanding, the following example will be used.

[0062] For example, in HTTP services, if the metric users care most about is response time, and the weight corresponding to response time is 0.6, when the response time of service A is t1, the response time of service B is t2, and the response time of service C is t3, it can be understood that in HTTP services, the shorter the response time, the better. Therefore, if t1>t2>t3, service A is assigned 6 points, service B is assigned 8 points, and service C is assigned 10 points. The evaluation score of service A is 0.6*6=3.6 points, the evaluation score of service B is 0.6*8=4.8 points, and the evaluation score of service C is 0.6*10=6 points. The evaluation scores are then sorted from high to low, and the service with the highest evaluation score is selected as the optimal service.

[0063] The method for service evaluation provided in the embodiment of the present application includes: obtaining the simulated test traffic and the real traffic of each service to be evaluated to obtain the target traffic data, and extracting multi-dimensional index data from the target traffic data, and then assigning corresponding weights to the indicators of different dimensions according to the preset rules, and calculating the evaluation score of each service to be evaluated based on the index data of different dimensions and the corresponding weights, and then selecting the best service based on the evaluation score. It can be seen that the technical solution provided by the present application introduces simulated test traffic to avoid reducing the inaccuracy of service evaluation when the real traffic data generated by the service is small. In addition, when extracting the index data, the index data of multiple dimensions are extracted from the target traffic data to further improve the accuracy of the service evaluation in multiple dimensions. At the same time, when calculating the service evaluation score, the index data of different dimensions are assigned corresponding weights according to the preset rules, thereby obtaining the evaluation score based on the weight algorithm to improve the accuracy of the service evaluation, thereby improving the user experience.

[0064] In a specific embodiment, when extracting multi-dimensional indicator data from target traffic data, if the target traffic data volume is large, it may take a long time to extract the multi-dimensional indicator data, thereby resulting in low overall service efficiency and affecting user experience. Therefore, in the technical solution provided in the embodiment of the present application, extracting multi-dimensional indicator data from target traffic data includes:

[0065] Trigger multiple child threads;

[0066] Each sub-thread extracts indicator data of different dimensions in parallel.

[0067] During implementation, the entire evaluation service serves as the main thread. After the main thread obtains the target traffic data from the simulated test traffic and real traffic of each service to be evaluated, it triggers multiple sub-threads, each of which extracts indicator data of different dimensions in parallel. It is understood that the number of sub-threads triggered is equal to the number of dimensions of the indicator data. After obtaining the target traffic data, each sub-thread runs in parallel to extract data for indicators of different dimensions.

[0068] The service evaluation method provided in the embodiment of the present application calls multiple sub-threads to extract in parallel when extracting multi-dimensional indicator data, thereby saving time in extracting indicator data, improving extraction efficiency, and further improving the overall evaluation efficiency of service evaluation and improving user experience.

[0069] As a preferred embodiment, before assigning corresponding weights to indicator data of different dimensions according to preset rules, the method further includes:

[0070] Indicator data corresponding to pre-built-in indicators and indicator data corresponding to custom indicators are extracted from the target traffic data.

[0071] To ensure ease of use in service evaluation, a number of pre-built-in indicators are pre-installed. When extracting indicator data from target traffic data, in addition to extracting data corresponding to the system default indicators, the indicator data corresponding to the preset built-in indicators are also extracted. Of course, it should be noted that this application does not limit the type and number of pre-built-in indicators.

[0072] In addition, in order to ensure the flexibility of service evaluation, users can set the indicators they need to pay attention to by themselves. That is, the technical solution provided in the embodiment of the present application can customize indicators. Therefore, when extracting indicator data from the target traffic data, it is also necessary to extract the indicator data corresponding to the user-defined indicators at the same time.

[0073] The service evaluation method provided in the embodiments of the present application, before assigning corresponding weights to indicator data of different dimensions according to preset rules, simultaneously extracts indicator data corresponding to a batch of preset built-in indicators to improve evaluation usability. Furthermore, to enhance evaluation flexibility, the indicator data corresponding to user-defined indicators is also extracted. This improves the accuracy of service evaluation while enhancing the usability and flexibility of evaluation, thereby enhancing the user experience.

[0074] During implementation, when recommending the optimal service, if the user only cares about one indicator, that is, if there is only one target focus indicator, the evaluation scores of the target focus indicators in each service to be evaluated are first sorted according to the preset indicator evaluation direction to obtain the sorting result, wherein the preset evaluation direction is that the indicator moves in a better direction. For example, in the HTTP service scenario, the evaluation direction of the response time indicator is that the shorter the time, the better.

[0075] After obtaining the sorting results, the optimal service is determined from the sorting results according to the preset indicator direction. For example, for the response time indicator, the service with the shortest response time is selected as the optimal service.

[0076] The service evaluation method provided in the embodiments of this application, when there is only one target indicator, sorts the evaluation scores of the target indicator in each service to be evaluated according to a preset indicator evaluation direction to obtain a ranking result, and then determines the optimal service from the ranking result based on the preset indicator evaluation direction. This method quickly determines the optimal service based on the indicator evaluation direction, improves service evaluation efficiency, improves evaluation accuracy, and thus enhances user experience.

[0077] Figure 2 This is a flowchart of another service evaluation method provided in an embodiment of the present application. In implementation, if there are multiple target focus indicators, as a preferred embodiment, Figure 2 As shown, the recommended optimal services include:

[0078] S20: sorting the evaluation scores of the target focus indicators in each service to be evaluated according to the preset indicator evaluation direction to obtain a sorting result;

[0079] S21: Set corresponding scores for indicators of different rankings in the ranking results based on the indicator evaluation direction. And assign corresponding weights to different target focus indicators according to the focus level;

[0080] After calculating each evaluation score, the evaluation scores corresponding to the target focus indicators are sorted according to the preset indicator evaluation direction to obtain the sorting results. It can be understood that different target focus indicators have different expected evaluation trends. Therefore, the evaluation scores are sorted according to the preset evaluation directions of different indicators.

[0081] After sorting, the corresponding scores are set for the indicators of different rankings in the sorting results based on the indicator evaluation direction. For example, the first place is assigned 10 points, the second place is assigned 9 points, and so on. Different scores are assigned to different rankings. Of course, it should be noted that the embodiment of the present application does not limit the size of the scores set for different rankings. In addition, the corresponding weights are assigned to different target attention indicators according to the attention level, among which the attention level is set by the user. The specific attention level and weight distribution are not limited in this application, but it should be noted that the sum of the weights corresponding to all target attention indicators is 1.

[0082] S22: Calculate the final score of each service to be evaluated based on the scores corresponding to different rankings in the ranking results and the weights corresponding to different target focus indicators;

[0083] S23: The service with the highest final score is regarded as the best service.

[0084] Based on the scores corresponding to the different rankings and the weights corresponding to the different target focus indicators in the ranking results obtained in step S20 and step S21, the final score of each service to be evaluated is calculated, and the service with the highest final score is taken as the optimal service. For ease of understanding, the following example will be used for illustration.

[0085] For example, in the HPPT service scenario, the target focus indicators include response time, error rate, and number of requests. The user's attention to the target focus indicators decreases in turn, and the response time weight is set to 0.6, the error rate weight is 0.4, and the first place in the ranking result is set to 10 points, the second place is 8 points, and the third place is 6 points.

[0086] If the response times of services A, B, and C are ranked in the order of shorter response times, then service A receives a score of 10, service C receives a score of 8, and service B receives a score of 6 under the response time metric. Furthermore, if the error rates of services A, B, and C are ranked in the order of fewer response times, then service B receives a score of 10, service C receives a score of 8, and service A receives a score of 6 under the error rate metric.

[0087] Thus, the final score of service A can be calculated as: 10*0.6+6*0.4=8.4 points, the final score of service B is: 6*0.6+10*0.4=7.6 points, and the final score of service C is: 8*0.6+8*0.4=8 points. Therefore, service A with the highest score is selected as the optimal service.

[0088] The service evaluation method provided in the embodiment of the present application, when there are multiple target attention indicators, first sorts the evaluation scores of the target attention indicators in each service to be evaluated according to the preset indicator evaluation direction to obtain the sorting result, then sets corresponding scores for indicators of different rankings in the sorting result based on the indicator evaluation direction, and assigns corresponding weights to different target attention indicators according to the attention level. Finally, the final score of each service to be evaluated is calculated based on the scores corresponding to different rankings in the sorting result and the weights corresponding to different target attention indicators, and the one with the highest final score is taken as the optimal service. Thus, based on the weight algorithm, attention level and indicator evaluation direction, the optimal service can be selected when there are multiple target attention indicators, ensuring the ease of use of service evaluation while improving the accuracy of service evaluation.

[0089] As a preferred embodiment, in order to further improve the evaluation efficiency and accuracy, after extracting multi-dimensional indicator data from the target traffic data, the data that does not fall within the preset range corresponding to the indicator data of each dimension is eliminated. For example, when the contemporary evaluation service is an HTTP service, the indicator data with a response time of zero is eliminated. This application does not set restrictions on the indicator data of each dimension, and can be set according to actual business needs.

[0090] The service evaluation method provided in the embodiment of the present application, after extracting multi-dimensional indicator data from the target traffic data, eliminates the data that does not fall within the preset range corresponding to the indicator data of each dimension, thereby improving the accuracy and efficiency of the service evaluation, and further improving the user experience.

[0091] In a specific embodiment, if the traffic generated by the service to be evaluated is large, in order to avoid reducing the evaluation efficiency, in the embodiment of the present application, multi-dimensional indicator data is extracted from the target traffic data, including:

[0092] Determine whether the actual flow rate reaches the preset value;

[0093] If the preset value is reached, only the indicator data corresponding to the actual flow rate is extracted from the target flow data;

[0094] If the preset value is not reached, the indicator data corresponding to the simulated test flow and the actual flow will be extracted from the target flow data according to the preset ratio.

[0095] During implementation, when extracting multi-dimensional indicator data, we first determine whether the actual traffic has reached the preset value. If it has reached the preset value, it is determined that the actual traffic generated by the current service to be evaluated is relatively large. In order to ensure the accuracy of the evaluation results and improve the evaluation efficiency, the corresponding indicator data is only extracted from the actual traffic generated.

[0096] If the real traffic generated is small, simulated test traffic is introduced, and the indicator data corresponding to the simulated test traffic and real traffic are extracted from the target traffic data according to a preset ratio. The preset ratio is set according to the actual amount of real traffic generated by the service to be evaluated, thereby avoiding reducing the evaluation accuracy when the real traffic is small.

[0097] The service evaluation method provided in the embodiment of the present application extracts multi-dimensional indicator data from the target traffic data. If the actual traffic reaches a preset value, only the indicator data corresponding to the actual traffic is extracted from the target traffic data. If it does not reach the preset value, the indicator data corresponding to the simulated test traffic and the actual traffic are extracted from the target traffic data according to a preset ratio. This avoids the situation where the simulated test traffic is still introduced when there is a lot of actual traffic data, resulting in low evaluation efficiency. At the same time, when the actual traffic is small, the simulated test traffic is introduced to improve the evaluation accuracy, thereby enhancing the user experience.

[0098] In the above embodiments, the service evaluation method is described in detail. This application also provides corresponding embodiments of the service evaluation device. It should be noted that this application describes the embodiments of the device from two perspectives: one is based on the functional module perspective, and the other is based on the hardware structure perspective.

[0099] Figure 3A structural diagram of a service evaluation device provided in an embodiment of the present application, such as Figure 3 As shown, the device includes:

[0100] An acquisition module 10 is used to obtain the simulated test traffic and the actual traffic of each service to be evaluated to obtain target traffic data;

[0101] Extraction module 11, used to extract multi-dimensional indicator data from target traffic data;

[0102] An allocation module 12 is used to allocate corresponding weights to indicators of different dimensions according to preset rules;

[0103] The calculation module 13 is used to calculate the evaluation score of each service to be evaluated based on the indicator data of different dimensions and the corresponding weights, so as to recommend the best service.

[0104] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, and they will not be repeated here.

[0105] The device for service evaluation provided by the present invention includes: obtaining the simulated test traffic and real traffic of each service to be evaluated to obtain target traffic data, and extracting multi-dimensional index data from the target traffic data, and then assigning corresponding weights to the indicators of different dimensions according to preset rules, and calculating the evaluation score of each service to be evaluated based on the index data of different dimensions and the corresponding weights, and then selecting the best service based on the evaluation score. It can be seen that the technical solution provided by the present application introduces simulated test traffic to avoid reducing the inaccuracy of service evaluation when the real traffic data generated by the service is small. In addition, when extracting index data, index data of multiple dimensions are extracted from the target traffic data to further improve the accuracy of service evaluation in multiple dimensions. At the same time, when calculating the service evaluation score, corresponding weights are assigned to the index data of different dimensions according to preset rules, thereby obtaining an evaluation score based on the weight algorithm to improve the accuracy of service evaluation, thereby improving the user experience.

[0106] Figure 4 A structural diagram of a service evaluation device provided in another embodiment of the present application is shown as follows: Figure 4 As shown, the service evaluation apparatus includes: a memory 20 for storing a computer program;

[0107] The processor 21 is configured to implement the steps of the service evaluation method mentioned in the above embodiment when executing the computer program.

[0108] The service evaluation device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0109] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0110] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the service evaluation method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include but is not limited to the service evaluation method, etc.

[0111] In some embodiments, the service evaluation device may further include a display screen 22 , an input and output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .

[0112] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the apparatus for service evaluation, and may include more or fewer components than shown in the figure.

[0113] The service evaluation device provided in an embodiment of the present application includes a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method: a service evaluation method.

[0114] The service evaluation device provided by the present invention introduces simulated test traffic to avoid inaccurate service evaluations when the service generates less actual traffic data. Furthermore, when extracting indicator data, multiple dimensions of indicator data are extracted from the target traffic data, further improving the accuracy of service evaluations in multiple dimensions. Furthermore, when calculating the service evaluation score, corresponding weights are assigned to the indicator data of different dimensions according to preset rules. This results in an evaluation score based on a weighted algorithm, improving the accuracy of service evaluations and, in turn, enhancing the user experience.

[0115] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiment.

[0116] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc. Various media that can store program code.

[0117] The above is a detailed introduction to the method, device and medium for service evaluation provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0118] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A method for service evaluation, characterized in that: include: Obtain the simulated test traffic and actual traffic of each service to be evaluated to obtain the target traffic data; The simulated test traffic is the test traffic obtained by simulating the actual business; Extracting multi-dimensional indicator data from the target traffic data; Assign corresponding weights to indicators of different dimensions according to preset rules; Calculate the evaluation score of each service to be evaluated based on the indicator data of different dimensions and the corresponding weights, so as to select the best service; The optimal service recommendation includes: Sorting the evaluation scores of the target focus indicators in each of the services to be evaluated according to a preset indicator evaluation direction to obtain a ranking result; wherein there are multiple target focus indicators; Setting corresponding scores for indicators of different rankings in the ranking results based on the indicator evaluation direction, and assigning corresponding weights to different target focus indicators according to the focus levels; Calculate the final score of each service to be evaluated based on the scores corresponding to different rankings in the ranking results and the weights corresponding to different target focus indicators; The service with the highest final score is used as the optimal service; The extracting multi-dimensional indicator data from the target traffic data includes: Determining whether the actual flow rate reaches a preset value; If the preset value is reached, only the index data corresponding to the actual flow rate is extracted from the target flow rate data; If the preset value is not reached, the index data corresponding to the simulated test flow and the real flow are extracted from the target flow data according to a preset ratio.

2. The service evaluation method according to claim 1, characterized in that: The extracting multi-dimensional indicator data from the target traffic data includes: Trigger multiple child threads; Each of the sub-threads extracts the indicator data of different dimensions in parallel.

3. The service evaluation method according to claim 2, characterized in that: Before allocating corresponding weights to the indicator data of different dimensions according to preset rules, the method further includes: The indicator data corresponding to the pre-built-in indicators and the indicator data corresponding to the custom indicators are respectively extracted from the target traffic data.

4. The service evaluation method according to claim 1, characterized in that: The optimal service recommendation includes: Sorting the evaluation scores of the target focus indicators in each of the services to be evaluated according to the preset indicator evaluation direction to obtain a ranking result; wherein, there is one target focus indicator; The optimal service is determined from the ranking results according to the preset indicator evaluation direction.

5. The service evaluation method according to claim 1, characterized in that: After extracting multi-dimensional indicator data from the target traffic data, the method further includes: Eliminate data that does not fall within the preset range corresponding to the indicator data of each dimension.

6. A device for service evaluation, characterized in that: include: The acquisition module is used to obtain the simulated test traffic and real traffic of each service to be evaluated to obtain the target traffic data; The simulated test traffic is the test traffic obtained by simulating the actual business; An extraction module, configured to extract multi-dimensional indicator data from the target traffic data; The allocation module is used to assign corresponding weights to indicators of different dimensions according to preset rules; A calculation module, configured to calculate the evaluation score of each of the services to be evaluated based on the indicator data of different dimensions and the corresponding weights, so as to select the best service; The calculation module is used to: Sorting the evaluation scores of the target focus indicators in each of the services to be evaluated according to a preset indicator evaluation direction to obtain a ranking result; wherein there are multiple target focus indicators; Setting corresponding scores for indicators of different rankings in the ranking results based on the indicator evaluation direction, and assigning corresponding weights to different target focus indicators according to the focus levels; Calculate the final score of each service to be evaluated based on the scores corresponding to different rankings in the ranking results and the weights corresponding to different target focus indicators; The service with the highest final score is used as the optimal service; The extraction module is used to: Determining whether the actual flow rate reaches a preset value; If the preset value is reached, only the index data corresponding to the actual flow rate is extracted from the target flow rate data; If the preset value is not reached, the index data corresponding to the simulated test flow and the real flow are extracted from the target flow data according to a preset ratio.

7. A device for service evaluation, characterized in that: including a memory for storing a computer program; A processor is configured to implement the steps of the service evaluation method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the service evaluation method according to any one of claims 1 to 5.

Images