System service performance determination method and device, electronic device and readable storage medium

Abstract

The present disclosure provides a method for determining system service performance, which can be applied to the field of data analysis and mining technology. The system service performance determination method includes: obtaining historical operation data of a preset type of service within a preset historical time period; determining the service failure rate of the preset type of service based on the historical operation data of the preset type of service and the preset service impact coefficient of the preset type of service, wherein the preset service impact coefficient is used to characterize the degree of impact of the preset type of service interruption on the target service system, wherein the target service system is a system including at least one preset type of service; determining the service failure rate of the target service system based on the service failure rate of at least one preset type of service; determining the index value of the service availability index of the target service system based on the service failure rate of the target service system. The present disclosure also provides a system service performance determination device, equipment, storage medium and program product.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data analysis and mining, and more specifically, to a method and device for determining system service performance, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] As the types of corporate business grow and the complexity of business increases, the functions of the application service systems used to handle business are becoming more and more diverse.

[0003] Currently, most of the application system service level assessment methods are qualitative evaluations, which can generally only reflect whether the system has achieved its goals, but cannot reflect the gap with the goals, and cannot provide a basis for service level planning, design, improvement and enhancement. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a method and apparatus for determining system service performance, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to a first aspect of the present disclosure, a method for determining system service performance is provided, comprising:

[0006] Obtaining historical operation data of a preset type of service within a preset historical time period, wherein the historical operation data includes the interruption duration, the number of faulty subsystems, and the total number of operating subsystems of the preset type of service;

[0007] Determine the service failure rate of the preset type of service according to the historical operation data of the preset type of service and the preset service impact coefficient of the preset type of service, wherein the preset service impact coefficient is used to characterize the impact degree of the preset type of service interruption on the target service system, wherein the target service system is a system including at least one preset type of service;

[0008] Determining a service failure rate of the target service system according to a service failure rate of at least one of the preset types of services;

[0009] According to the service failure rate of the target service system, an indicator value of the service availability indicator of the target service system is determined.

[0010] According to an embodiment of the present disclosure, determining the service failure rate of the preset type of service according to the historical operation data of the preset type of service and the preset service impact coefficient of the preset type of service includes:

[0011] Calculate the ratio of the interruption duration to the statistical time period to obtain the value of the first impact item;

[0012] Calculate the ratio of the number of faulty subsystems to the total number of running subsystems to obtain a second impact item value;

[0013] The product of the first impact item value, the second impact item value, and the preset service impact coefficient is calculated to obtain the service failure rate of the preset type of service.

[0014] According to an embodiment of the present disclosure, determining the service failure rate of the target service system according to the service failure rate of at least one of the preset types of services includes:

[0015] Calculating the sum of the service failure rates of at least one of the preset types of services to obtain an initial service failure rate of the target service system;

[0016] The service failure rate of the target service system is determined based on the initial service failure rate and the preset service contribution coefficient, wherein the preset service contribution coefficient is used to characterize the contribution degree of the service of the target service system to the business.

[0017] According to an embodiment of the present disclosure, the above-mentioned preset type services include software service type services, platform service type services, and infrastructure service type services.

[0018] According to an embodiment of the present disclosure, the system service performance determination method further includes:

[0019] Determine the index value of the optional evaluation index of the target service system, wherein the optional evaluation index includes at least one of the following: average service recovery time, average service response time, number of accidents, and user satisfaction;

[0020] Determine the weight of the above service availability indicator and the weight of the above optional evaluation indicator;

[0021] The comprehensive service level index of the target service system is determined according to the index values ​​of the service availability index and the optional evaluation index, and the weights of the service availability index and the optional evaluation index.

[0022] According to an embodiment of the present disclosure, the above-mentioned determination of the weight of the above-mentioned service availability index and the weight of the above-mentioned optional evaluation index includes:

[0023] The weights of the above service availability indicators and the above optional evaluation indicators are determined using a preset weight algorithm.

[0024] According to an embodiment of the present disclosure, the system service performance determination method further includes:

[0025] The indicator values ​​of the above service availability indicator and the above optional evaluation indicator are normalized.

[0026] A second aspect of the present disclosure provides a system service performance determination device, including:

[0027] An acquisition module, used to acquire historical operation data of a preset type of service within a preset historical time period, wherein the historical operation data includes the interruption duration, the number of faulty subsystems, and the total number of operating subsystems of the preset type of service;

[0028] A first determination module is used to determine the service failure rate of the preset type of service according to the historical operation data of the preset type of service and the preset service impact coefficient of the preset type of service, wherein the preset service impact coefficient is used to characterize the impact degree of the preset type of service interruption on the target service system, wherein the target service system is a system including at least one of the preset type of service;

[0029] A second determining module, configured to determine a service failure rate of the target service system according to a service failure rate of at least one of the preset types of services;

[0030] The third determination module is used to determine the index value of the service availability index of the above-mentioned target service system according to the service failure rate of the above-mentioned target service system.

[0031] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned system service performance determination method.

[0032] The fourth aspect of the present disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned system service performance determination method.

[0033] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the above-mentioned system service performance determination method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0035] Figure 1 The application scenario diagram of the system service performance determination method according to an embodiment of the present disclosure is schematically shown;

[0036] Figure 2 A flowchart of a method for determining system service performance according to an embodiment of the present disclosure is schematically shown;

[0037] Figure 3A flowchart of determining a service failure rate of a preset type of service according to an embodiment of the present disclosure is schematically shown;

[0038] Figure 4 A flowchart of a method for determining system service performance according to another embodiment of the present disclosure is schematically shown;

[0039] Figure 5 The structure block diagram of the system service performance determination device according to the embodiment of the present disclosure is schematically shown;

[0040] Figure 6 A structural block diagram schematically shows a device for determining system service performance according to another embodiment of the present disclosure; and

[0041] Figure 7 A block diagram of an electronic device suitable for implementing a method for determining system service performance according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0042] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0043] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0044] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0045] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0046] When providing system services to users, qualitative evaluation methods are usually used to evaluate service levels. For example, a target service level is set. If the system reaches the target service level, the qualitative evaluation method determines that it meets the target, otherwise it does not meet the target.

[0047] It can be seen that the above qualitative evaluation method can only reflect whether the system meets the standards, but cannot reflect the gap between the service level and the target, and thus cannot provide a reference for the planning, design and improvement of the service level.

[0048] In the process of implementing the present disclosure, it is found that the quantitative index values ​​of various evaluation indicators of the service system can be calculated based on the historical operation data of the service system that provides preset types of services to users, so that the service level of the application system can be evaluated more accurately and intuitively.

[0049] Specifically, an embodiment of the present disclosure provides a method and apparatus for determining system service performance, an electronic device, a computer-readable storage medium, and a computer program product, the method comprising obtaining historical operation data of a preset type of service within a preset historical time period, wherein the historical operation data comprises the interruption duration of the preset type of service, the number of faulty subsystems, and the total number of operating subsystems; determining a service failure rate of the preset type of service based on the historical operation data of the preset type of service and a preset service impact coefficient of the preset type of service, wherein the preset service impact coefficient is used to characterize the degree of impact of the preset type of service interruption on a target service system, wherein the target service system is a system including at least one preset type of service; determining a service failure rate of a target service system based on the service failure rate of at least one preset type of service; and determining an indicator value of a service availability indicator of the target service system based on the service failure rate of the target service system.

[0050] It should be noted that the system service performance determination method and device provided in the present disclosure can be used in the field of data analysis and mining technology, and can also be used in the financial field, such as banks and other financial institutions, and can also be used in any field other than the field of data analysis and mining technology and the financial field. The embodiments of the present disclosure do not limit the application field of the system service performance determination method and device.

[0051] Figure 1 The application scenario diagram of the system service performance determination method according to an embodiment of the present disclosure is schematically shown.

[0052] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0053] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive services provided by the server 105 or send requests for services, etc. Various client applications can be installed on the terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0054] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0055] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0056] According to the embodiment of the present disclosure, the user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104, for example, to initiate a request for obtaining the service performance result data of the system to be evaluated to the server 105. In response to the user request, the server 105 can obtain the historical operation data of the service system to be evaluated within a preset historical time period from the database, and perform the method of the embodiment of the present disclosure to process the historical operation data to obtain the system performance evaluation result of the service system to be evaluated, and then return it to the user through the terminal devices 101, 102, 103.

[0057] It should be noted that the system service performance determination method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the system service performance determination device provided in the embodiment of the present disclosure can generally be set in the server 105. The system service performance determination method provided in the embodiment of the present disclosure can also be performed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the system service performance determination device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0058] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0059] The following will be based on Figure 1 The scene described by Figure 2 to Figure 4 The method for determining the system service performance of the disclosed embodiment is described in detail.

[0060] Figure 2 The flowchart of the method for determining system service performance according to an embodiment of the present disclosure is schematically shown.

[0061] like Figure 2 As shown, the system service performance determination method of this embodiment may include operations S210 to S240.

[0062] In operation S210, historical operation data of a preset type of service within a preset historical time period is obtained, wherein the historical operation data includes an interruption duration of the preset type of service, a number of faulty subsystems, and a total number of operating subsystems.

[0063] In operation S220, the service failure rate of the preset type service is determined based on the historical operation data of the preset type service and the preset service impact coefficient of the preset type service, wherein the preset service impact coefficient is used to characterize the impact of the preset type service interruption on the target service system, wherein the target service system is a system including at least one preset type service.

[0064] In operation S230, a service failure rate of a target service system is determined according to a service failure rate of at least one preset type of service.

[0065] In operation S240, an indicator value of a service availability indicator of the target service system is determined according to the service failure rate of the target service system.

[0066] According to an embodiment of the present disclosure, the preset type service may refer to a service model of cloud computing, and the service model may include software-as-a-Service (SaaS), platform-as-a-Service (PaaS), and infrastructure-as-a-Service (IaaS).

[0067] Among them, software service-type services can refer to the administrator deploying the application software uniformly on the server of the system platform. Users can order the required application software services from the administrator through the Internet according to actual work needs, pay the owner of the application software according to the ordered services and duration, and obtain the services provided by the system platform through the Internet.

[0068] Platform service services may refer to services provided to users that deploy applications developed or acquired by users using provided development languages ​​and / or tools (such as Java, Python, etc.) onto cloud computing infrastructure.

[0069] Infrastructure service-oriented services can refer to a service model in which IT infrastructure is provided as a service through the network and charges users based on their actual usage or occupancy of resources.

[0070] According to an embodiment of the present disclosure, the service failure rate may refer to a rate at which a system is unavailable within a prescribed service time.

[0071] According to an embodiment of the present disclosure, the service availability index may refer to the ratio of the system being available within the specified service time. The service availability may be obtained by reverse calculation based on the service failure rate, that is, service availability = 1-service failure rate. For example, in an embodiment of the present disclosure, the service failure rate of the preset type service is first determined based on the historical operation data of the preset type service and the preset service impact coefficient, and then the service failure rate of the target service system is determined based on the service failure rates of multiple preset type services.

[0072] According to an embodiment of the present disclosure, the interruption duration of a preset type of service may refer to the total duration of the service system of the service type that cannot provide the corresponding service to the user. The number of faulty subsystems may refer to the number of subsystems that have experienced faults. The total number of operating subsystems may refer to the total number of subsystems that are currently in operation and can provide services to the user, wherein the total number of operating subsystems = the number of faulty subsystems + the number of normal subsystems.

[0073] According to an embodiment of the present disclosure, the service failure rate of the target service system is determined according to the service failure rate of at least one preset type of service. For example, the service failure rate of at least one preset type of service may be calculated to obtain the service failure rate of the target service system. The service failure rate of the target service system is determined according to the service failure rate of at least one preset type of service. For example, the service failure rate of the target service system may be determined by calculating the sum of the service failure rates of at least one preset type of service and then combining the contribution of the target service system's services to the business.

[0074] According to an embodiment of the present disclosure, after determining a preset service type and a preset historical time period, the corresponding historical data is obtained, so that the service failure rate of different systems can be determined according to the historical data, thereby obtaining the total service failure rate of the preset type service under the target service system, and then obtaining the index value of the service availability index of the target service system according to the total service failure rate. The historical operation data can be obtained from the target service system using a system monitoring method.

[0075] Service availability can also be calculated based on the service available time and the total service time. For example, when a system provides functional services to users, the prescribed service time is one day, that is, 24 hours (the specific time can be set according to specific needs). During the service time, the system can provide services to users for 18 hours. The indicator value of the service availability index of the system is 18 / 24*100%=75%.

[0076] According to the embodiments of the present disclosure, by processing the historical operation data of the preset type service, the service availability index value can be finally determined. The index value is a quantitative parameter, which can more accurately and quantitatively reflect the level of system service performance, overcoming the technical problem that the related technology can only reflect whether the system has achieved the goal, but cannot reflect the gap with the goal, and achieves the gap between the service availability index value and the goal, thereby providing a basis for service level planning, improvement and enhancement. According to the embodiments of the present disclosure, in the process of determining the service failure rate of the preset type service, the preset service impact coefficient of the preset type service is introduced, and the impact of the preset type service interruption on the target service system is fully considered, so that the calculated availability index value is more referenceable; and in the process of determining the service failure rate of the target service system based on the service failure rate of at least one preset type service, it is distinguished according to the service type, and the impact of each service type on the entire target service system is considered, and the obtained availability index value has strong universality.

[0077] Figure 3 The flowchart of determining the service failure rate of a preset type of service according to an embodiment of the present disclosure is schematically shown.

[0078] like Figure 3 As shown, determining the service failure rate of the preset type service according to the historical operation data of the preset type service and the preset service impact coefficient of the preset type service may include operations S310 to S330.

[0079] In operation S310, a ratio of the interruption duration to the statistical time period duration is calculated to obtain a first impact item value.

[0080] In operation S320, a ratio of the number of faulty subsystems to the total number of operating subsystems is calculated to obtain a second impact item value.

[0081] In operation S330, the product of the first impact item value, the second impact item value, and the preset service impact coefficient is calculated to obtain a service failure rate of a preset type of service.

[0082] According to an embodiment of the present disclosure, the calculation of the first impact item value T1 is as shown in formula (1).

[0083] T1=H i / H (1)

[0084] Among them, i represents the i-th fault, H i Characterizes the interruption duration, that is, the duration of the service interruption caused by the ith fault, and H characterizes the duration of the statistical time period. According to an embodiment of the present disclosure, the calculation of the second impact item value T2 is shown in formula (2).

[0085] T2=S ij / S j (2)

[0086] Among them, S ij Characterizes the number of faulty subsystems, that is, the number of subsystems affected within the j-type preset service range in the i-th accident, S j The total number of subsystems representing the j-type preset services of the application system. The preset services include but are not limited to three types, such as software service type services, platform service type services, and infrastructure service type services.

[0087] According to an embodiment of the present disclosure, the service availability index A of the target service system x The calculation of is shown in formula (3) to formula (5).

[0088] A x =[1-B x ]×100% (3)

[0089]

[0090]

[0091] Among them, W x Indicates the importance of the service provided by the application system to the business, and the value range is (0, 1]; W j is the importance of the system interruption of the j-type preset service to the service, and the value range is (0, 1]; B x Characterizes the service failure rate; L j It represents the impact of all faults in the statistical time period H on the j-type preset type service of the application system; N represents the number of preset type services.

[0092] According to the embodiments of the present disclosure, W can be x Divided into multiple impact levels, such as low, medium, and high impact levels, the corresponding W x The possible values ​​are 0.3, 0.7, 1.

[0093] According to the embodiment of the present disclosure, based on the importance of different preset types of service interruptions to the service, W jIt is divided into multiple important levels, for example, including but not limited to the three important levels of 1, 0.7, and 0.3.

[0094] According to the embodiments of the present disclosure, in the process of determining the service failure rate of a preset type of service, since multiple indicators such as the proportion of interruption duration of the preset type of service and the proportion of faulty subsystems are fully considered, the resulting service failure rate can provide a basis for the calculation of the service availability index more accurately, making the calculated availability index value more referenceable.

[0095] According to an embodiment of the present disclosure, determining the service failure rate of the target service system according to the service failure rate of at least one preset type of service may include the following operations:

[0096] The sum of the service failure rates of at least one preset type of service is calculated to obtain an initial service failure rate of the target service system. The service failure rate of the target service system is determined based on the initial service failure rate and a preset business contribution coefficient, wherein the preset business contribution coefficient is used to characterize the contribution degree of the service of the target service system to the business.

[0097] According to an embodiment of the present disclosure, when there are multiple preset type services providing services to users, the service failure rate of each preset type service can be added together to obtain the service failure rate under multiple preset type services. For example, when there are three preset type services, N in formula (5) is 3.

[0098] According to the embodiments of the present disclosure, by adding up the service failure rates of multiple preset types of services provided to users, the quality of services provided to users can be more accurately reflected, thereby providing an important reference basis for service level planning, design or improvement.

[0099] According to an embodiment of the present disclosure, the preset type services include software service type services, platform service type services, and infrastructure service type services.

[0100] According to the embodiments of the present disclosure, the subsystem in the above embodiments represents at least one of the functions, components, equipment units, etc. involved in the preset service type. For example, when the preset type service is a software service type service, the subsystem represents the functional points of the application system, when the preset type service is a platform service type service, the subsystem represents the service cluster scale or the number of middleware, etc., and when the preset type service is an infrastructure service type service, the subsystem represents the number of infrastructure equipment units, etc.

[0101] Figure 4 The flowchart of a method for determining system service performance according to another embodiment of the present disclosure is schematically shown.

[0102] like Figure 4 As shown, the system service performance determination method may further include operations S410 to S430.

[0103] In operation S410, an indicator value of an optional evaluation indicator of the target service system is determined, wherein the optional evaluation indicator includes at least one of the following: average service recovery time, average service response time, number of incidents, and user satisfaction;

[0104] In operation S420, a weight of a service availability indicator and a weight of an optional evaluation indicator are determined;

[0105] In operation S430, a comprehensive service level index of the target service system is determined according to the index values ​​of the service availability index and the optional evaluation index, and the weights of the service availability index and the optional evaluation index.

[0106] According to the embodiments of the present disclosure, the mean time to repair (MTTR) represents the average time it takes for a system or service to recover after multiple interruptions during a statistical period after a system failure. The mean time to repair is an inverse indicator, and the calculation of the mean time to repair MTTR is shown in formula (6).

[0107]

[0108] Among them, t i is the time of the i-th repair, and K is the number of repairs.

[0109] According to an embodiment of the present disclosure, the service response time represents the average time for the system to respond to a request within a statistical time period, and the service average response time may refer to the average response time of all services of the system. The service average response time is an inverse indicator.

[0110] According to an embodiment of the present disclosure, the number of accident occurrences represents the number of failures (obtained according to the accident definition within the organization) that occurred during the system service period. The number of accident occurrences is an inverse indicator. It should be noted that the definition of a failure can be defined according to actual needs. For example, the situation where the service cannot be provided within a preset time period is positioned as a failure. The preset time period can be defined according to needs, such as 1 second.

[0111] According to the embodiment of the present disclosure, user satisfaction represents the user's satisfaction with the system within the specified service time. User satisfaction can be expressed in percentage, and its value range is (0, 100%], which is a positive indicator. The calculation of user satisfaction is shown in formula (7).

[0112] User satisfaction = number of satisfied users surveyed / total number of users surveyed * 100% (7)

[0113] It should be noted that the parameter value of the average service response time can be obtained through the application system monitoring method, the average service recovery time and the number of accidents can be obtained through the management system that records relevant information, and user satisfaction can be obtained through user surveys.

[0114] According to the embodiments of the present disclosure, in the multi-index evaluation, some index values ​​are better the larger they are, and such indexes are called positive indexes, while some index values ​​are better the smaller they are, and such indexes are called inverse indexes. Therefore, before performing the multi-index comprehensive evaluation calculation, it is necessary to perform consistency processing on the indexes, and convert the inverse indexes into positive indexes. The conversion of the inverse index includes but is not limited to the method of converting it into a positive index in the form of an inverse.

[0115] According to the embodiments of the present disclosure, when multiple optional evaluation indicators are converted into positive indicators, the weight of the service availability indicator and the weight of the optional evaluation indicator can be determined by using, but not limited to, the CRITIC weight determination method, so as to determine the comprehensive service level index of the target service system according to the indicator values ​​of the service availability indicator and the optional evaluation indicator and the corresponding weights. Then, the service level of each system can be ranked, analyzed, etc. according to the index size.

[0116] According to the embodiments of the present disclosure, the CRITIC weight determination method is an objective weighting method that comprehensively considers data fluctuations and the correlation between indicators, conforms to the scenario characteristics of the application system operation service, and avoids the subjectivity of artificially determining weights.

[0117] According to the embodiments of the present disclosure, in the process of providing services to users, the impact of various influencing factors on the system service performance is comprehensively considered to facilitate determining the weights of different influencing factors, thereby making the calculated comprehensive service level index more accurate and more valuable for reference. At the same time, since the CRITIC weight determination method is used to determine the weights of different influencing factors, the comprehensive service level index can more objectively and accurately reflect the gap between the services provided by the system and the targets.

[0118] According to an embodiment of the present disclosure, determining the weight of the service availability indicator and the weight of the optional evaluation indicator may include the following operations:

[0119] The weights of the service availability indicators and the optional evaluation indicators are determined using a preset weight algorithm.

[0120] According to an embodiment of the present disclosure, the preset weight algorithm may include but is not limited to the CRITIC weight determination method involved above, for example, it may also include factor analysis method, principal component analysis method, AHP hierarchy method, priority diagram method, entropy value method, consciousness weight value method and data volume weight value method, etc.

[0121] According to an embodiment of the present disclosure, the system service performance determination method may further include the following operations:

[0122] The indicator values ​​of service availability indicators and optional evaluation indicators are normalized.

[0123] According to the embodiments of the present disclosure, since the service level evaluation indicators are of different dimensions and the numerical values ​​of the indicators vary greatly, directly weighted averaging multiple indicators is not conducive to the evaluation of the indicators. Therefore, it is necessary to normalize the multiple different service availability indicators and optional evaluation indicators.

[0124] According to an embodiment of the present disclosure, the method used for normalization processing may include but is not limited to the extreme value method. The method of the extreme value method is shown in formula (8).

[0125]

[0126] Among them, x represents the current index value among multiple evaluation indicators, m represents the minimum index value among multiple evaluation indicators, M represents the maximum index value among multiple evaluation indicators, and x * Represents the target indicator value after the current indicator value is processed.

[0127] According to the embodiments of the present disclosure, the present disclosure quantitatively calculates the service level index by integrating the calculation method of the service availability of different preset types of services, the service availability index and optional evaluation index and other multi-indicator comprehensive evaluation methods, the CRITIC weight determination method, and can clarify the service level of a group of different systems within the statistical time period.

[0128] At the same time, the service level can be intuitively quantified based on the service level indicator value, which can provide important reference for service level planning, design, improvement and enhancement, allowing system administrators to understand the service level of the system they manage. It is also of great significance to promptly identify weaknesses in the application system's service operation process based on service level evaluation indicators.

[0129] Based on the above system service performance determination method, the present disclosure also provides a system service performance determination device. Figure 5 and Figure 6 The device is described in detail.

[0130] Figure 5 The structural block diagram of the system service performance determination device according to an embodiment of the present disclosure is schematically shown.

[0131] like Figure 5 As shown, the system service performance determination device 500 of this embodiment may include an acquisition module 510 , a first determination module 520 , a second determination module 530 and a third determination module 540 .

[0132] The acquisition module 510 is used to acquire historical operation data of a preset type of service within a preset historical time period, wherein the historical operation data includes the interruption duration of the preset type of service, the number of faulty subsystems, and the total number of operating subsystems.

[0133] The first determination module 520 is used to determine the service failure rate of the preset type service based on the historical operation data of the preset type service and the preset service impact coefficient of the preset type service, wherein the preset service impact coefficient is used to characterize the impact of the preset type service interruption on the target service system, wherein the target service system is a system including at least one preset type service.

[0134] The second determining module 530 is used to determine the service failure rate of the target service system according to the service failure rate of at least one preset type of service.

[0135] The third determination module 540 is used to determine the indicator value of the service availability indicator of the target service system according to the service failure rate of the target service system.

[0136] According to the embodiments of the present disclosure, by processing the historical operation data of the preset type service, the service availability index value can be finally determined. The index value is a quantitative parameter, which can more accurately and quantitatively reflect the level of system service performance, overcoming the technical problem that the related technology can only reflect whether the system has achieved the goal, but cannot reflect the gap with the goal, and achieves the gap between the service availability index value and the goal, thereby providing a basis for service level planning, design, improvement and improvement. According to the embodiments of the present disclosure, in the process of determining the service failure rate of the preset type service, the preset service impact coefficient of the preset type service is introduced, and the impact of the preset type service interruption on the target service system is fully considered, so that the calculated availability index value is more referenceable; and in the process of determining the service failure rate of the target service system according to the service failure rate of at least one preset type service, it is distinguished according to the service type, and the impact of each service type on the entire target service system is considered, and the obtained availability index value has strong universality.

[0137] According to an embodiment of the present disclosure, the first determining module 520 may include a first calculating unit, a second calculating unit, and a third calculating unit.

[0138] The first calculation unit is used to calculate the ratio of the interruption duration to the statistical time period duration to obtain a first impact item value.

[0139] The second calculation unit is used to calculate the ratio of the number of faulty subsystems to the total number of running subsystems to obtain a second impact item value.

[0140] The third calculation unit is used to calculate the product of the first impact item value, the second impact item value, and the preset service impact coefficient to obtain the service failure rate of the preset type of service.

[0141] According to an embodiment of the present disclosure, the second determining module 530 may include a fourth calculating unit and a first determining unit.

[0142] The fourth calculation unit is used to calculate the sum of the service failure rates of at least one preset type of service to obtain an initial service failure rate of the target service system.

[0143] The first determining unit is used to determine the service failure rate of the target service system according to the initial service failure rate and the preset service contribution coefficient, wherein the preset service contribution coefficient is used to characterize the contribution degree of the service of the target service system to the business.

[0144] According to an embodiment of the present disclosure, the preset type services include software service type services, platform service type services, and infrastructure service type services.

[0145] Figure 6 The structural block diagram of a system service performance determination device according to another embodiment of the present disclosure is schematically shown.

[0146] like Figure 6 As shown, the system service performance determination device 500 may include a fourth determination module 550 , a fifth determination module 560 and a sixth determination module 570 in addition to the acquisition module 510 , the first determination module 520 , the second determination module 530 and the third determination module 540 .

[0147] The fourth determination module 550 is used to determine the index value of the optional evaluation index of the target service system, wherein the optional evaluation index includes at least one of the following: average service recovery time, average service response time, number of accidents, and user satisfaction.

[0148] The fifth determination module 560 is used to determine the weight of the service availability index and the weight of the optional evaluation index.

[0149] The sixth determination module 570 is used to determine the comprehensive service level index of the target service system according to the indicator values ​​of the service availability indicator and the optional evaluation indicator, and the weights of the service availability indicator and the optional evaluation indicator.

[0150] According to an embodiment of the present disclosure, the fifth determining module 560 may include a second determining unit.

[0151] The second determining unit is used to determine the weight of the service availability indicator and the weight of the optional evaluation indicator by using a preset weight algorithm.

[0152] According to an embodiment of the present disclosure, the system service performance determination device 500 may further include a normalization module.

[0153] The normalization module is used to normalize the indicator values ​​of the service availability indicator and the optional evaluation indicator.

[0154] According to an embodiment of the present disclosure, any multiple modules of the acquisition module 510, the first determination module 520, the second determination module 530, and the third determination module 540 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 510, the first determination module 520, the second determination module 530, and the third determination module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 510 , the first determination module 520 , the second determination module 530 , and the third determination module 540 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0155] Figure 7 A block diagram of an electronic device suitable for implementing a method for determining system service performance according to an embodiment of the present disclosure is schematically shown.

[0156] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0157] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0158] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0159] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0160] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0161] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the system service performance determination method provided by the embodiment of the present disclosure.

[0162] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 701. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0163] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0164] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0165] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0166] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0167] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0168] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for determining system service performance, include: Obtaining historical operation data of a preset type of service within a preset historical time period, wherein the historical operation data includes the interruption duration of the preset type of service, the number of faulty subsystems, and the total number of operating subsystems; Determining a service failure rate of the preset type of service according to historical operation data of the preset type of service and a preset service impact coefficient of the preset type of service, wherein the preset service impact coefficient is used to characterize the degree of influence of the preset type of service interruption on a target service system, wherein the target service system is a system including at least one of the preset type of services; Determining a service failure rate of the target service system according to a service failure rate of at least one of the preset types of services; Determining an indicator value of a service availability indicator of the target service system according to a service failure rate of the target service system; Wherein, determining the service failure rate of the preset type service according to the historical operation data of the preset type service and the preset service impact coefficient of the preset type service includes: Calculate the ratio of the interruption duration to the duration of the statistical time period to obtain a first impact item value; calculate the ratio of the number of faulty subsystems to the total number of running subsystems to obtain a second impact item value; calculate the product of the first impact item value, the second impact item value, and the preset service impact coefficient to obtain a service failure rate of the preset type of service; Wherein, determining the service failure rate of the target service system according to the service failure rate of at least one of the preset types of services comprises: Calculate the sum of the service failure rates of at least one of the preset types of services to obtain the initial service failure rate of the target service system; determine the service failure rate of the target service system based on the initial service failure rate and a preset business contribution coefficient, wherein the preset business contribution coefficient is used to characterize the degree of contribution of the service of the target service system to the business.

2. The method according to claim 1, in: The preset type services include software service type services, platform service type services, and infrastructure service type services.

3. The method according to claim 1, further comprising: include: Determine the index value of the optional evaluation index of the target service system, wherein the optional evaluation index includes at least one of the following: average service recovery time, average service response time, number of accidents, and user satisfaction; Determining the weight of the service availability indicator and the weight of the optional evaluation indicator; The comprehensive service level index of the target service system is determined according to the indicator values ​​of the service availability indicator and the optional evaluation indicator, and the weights of the service availability indicator and the optional evaluation indicator.

4. The method according to claim 3, wherein the weight of the service availability indicator and the weight of the optional evaluation indicator are determined. include: The weight of the service availability indicator and the weight of the optional evaluation indicator are determined using a preset weight algorithm.

5. The method according to claim 3, further comprising: include: The indicator values ​​of the service availability indicator and the optional evaluation indicator are normalized.

6. A system service performance determination device, include: An acquisition module, used to acquire historical operation data of a preset type of service within a preset historical time period, wherein the historical operation data includes the interruption duration of the preset type of service, the number of faulty subsystems, and the total number of operating subsystems; A first determination module is configured to determine a service failure rate of the preset type of service according to historical operation data of the preset type of service and a preset service impact coefficient of the preset type of service, wherein the preset service impact coefficient is used to characterize the degree of influence of the preset type of service interruption on a target service system, wherein the target service system is a system including at least one of the preset type of services; A second determining module, configured to determine a service failure rate of the target service system according to a service failure rate of at least one of the preset types of services; A third determination module, configured to determine an index value of a service availability index of the target service system according to a service failure rate of the target service system; The first determination module includes a first calculation unit, a second calculation unit and a third calculation unit; The first calculation unit is used to calculate the ratio of the interruption duration to the statistical time period to obtain a first impact item value; The second calculation unit is used to calculate the ratio of the number of faulty subsystems to the total number of running subsystems to obtain a second impact item value; The third calculation unit is used to calculate the product of the first impact item value, the second impact item value, and the preset service impact coefficient to obtain the service failure rate of the preset type of service; The second determination module includes a fourth calculation unit and a first determination unit; The fourth calculation unit is used to calculate the sum of the service failure rates of at least one preset type of service to obtain an initial service failure rate of the target service system; The first determining unit is used to determine the service failure rate of the target service system according to the initial service failure rate and the preset service contribution coefficient, wherein the preset service contribution coefficient is used to characterize the contribution degree of the service of the target service system to the business.

7. An electronic device, include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images