Assessment method and device for application in cloud native environment, equipment, medium and product

Abstract

The embodiment of the invention provides an evaluation method and device for an application in a cloud native environment, equipment, a medium and a product, and belongs to the technical field of cloud native. The method comprises the steps of monitoring first information of a target application; wherein the target application is any application in an application system; the first information comprises at least one of service availability information, resource usability information, log information and business index information; determining second information according to the first information; generating an observability evaluation report according to the second information; wherein the observability evaluation report is used for maintaining the application system. According to the embodiment of the invention, the evaluation capability of the application in the cloud native environment can be improved. The application can be an insurance system, a bank system, a transaction system, an order system and the like.

Description

Technical Field

[0001] The present application relates to the field of cloud native technology, and in particular to an evaluation method, apparatus, equipment, medium, and product for applications in a cloud native environment. Background Art

[0002] As the concept of cloud computing matures, cloud computing has become the de facto standard for IT infrastructure. Incorporating practical experience, cloud computing has evolved from simple cloud computing to a more mature cloud-native architecture, forming a comprehensive infrastructure ecosystem. Cloud-native environments are primarily built on container hosting platforms (Kubernetes, K8S). K8S leverages this to implement a container-hosted, cloud-native IT infrastructure, providing infrastructure services including computing power, networking, and storage.

[0003] In cloud-native environments, monitoring, a key focus of operations and maintenance (O&M) tasks, is evolving into observability systems engineering. Observability has become a crucial step in evaluating application systems before they go live. This requires O&M teams to adapt monitoring and assessments to the unique characteristics of cloud-native environments during pre-launch evaluations of systems (such as insurance, banking, trading, and order systems). Therefore, there is an urgent need to develop an application evaluation method for cloud-native environments to better meet O&M needs. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose an evaluation method, device, equipment, medium and product for applications in a cloud-native environment, aiming to improve the evaluation capabilities of applications in a cloud-native environment.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application provides an evaluation method for applications in a cloud-native environment, the method comprising:

[0006] Monitoring first information of a target application; wherein the target application is any application in the application system; the first information includes at least one of the following: service availability information, resource usage information, log information, and business indicator information;

[0007] Determine second information based on the first information; wherein the second information includes at least one of the following: an availability evaluation result of the target application determined based on the service availability information; a resource usage result of the target application determined based on the resource usage information; a log evaluation result of the target application determined based on the log information; or an indicator evaluation result of the target application determined based on the business indicator information;

[0008] Generate an observability assessment report based on the second information.

[0009] In some embodiments, the service availability information includes: response information of the target application to the service call request;

[0010] The determining the second information according to the first information includes:

[0011] In the case where the response information is monitored, determining that the availability assessment result is available;

[0012] In the case that the response information is not monitored, it is determined that the available evaluation result is unavailable.

[0013] In some embodiments, the first information of the monitoring target application includes:

[0014] In the case that the target application does not have a corresponding public service interface and private service interface, sending the service call request by using an automation platform;

[0015] Monitor the response information of the target application to the service call request.

[0016] In some embodiments, the resource usage information includes: resource usage of the target application, and a response rate of the target application corresponding to a service call request;

[0017] The determining the second information according to the first information includes:

[0018] Comparing the resource usage with a benchmark performance indicator to determine a first resource usage result; wherein the benchmark performance indicator is determined based on a performance test or an average value of resource usage over a preset time period;

[0019] determining a second resource usage result based on the response rate;

[0020] The resource usage result is determined based on the first resource usage result and / or the second resource usage result.

[0021] In some embodiments, the log information includes a log storage location, log content, and log output format of the log;

[0022] The determining the second information according to the first information includes:

[0023] Determining a first log evaluation result based on a degree of relevance between the log content of the log and a preset log content; wherein the preset log content is a content template designed by a developer;

[0024] Determining a second log evaluation result based on the log storage location and the log output format of the log; wherein the log storage location is used to determine whether the log of the target application is stored by a third party; and the log output format is used to determine whether the log of the target application is output according to importance;

[0025] The log evaluation result is determined based on the first log evaluation result and the second log evaluation result.

[0026] In some embodiments, the target application includes at least one business; and when the first information includes the business indicator information, the first information of the monitored target application includes:

[0027] Obtaining the occurrence volume of a target business corresponding to the target application; wherein the target business is determined by a developer based on the requirements of the target application;

[0028] Obtaining interface performance corresponding to a target service corresponding to the target application;

[0029] By analyzing historical business data, a future usage trend of the target business corresponding to the target application is obtained; wherein the business indicator information includes the occurrence volume, the interface performance and the future usage trend.

[0030] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides an evaluation device for applications in a cloud-native environment, the device comprising:

[0031] A monitoring module configured to monitor first information of a target application; wherein the target application is any application in an application system; and the first information includes at least one of the following: service availability information, resource usage information, log information, and business indicator information;

[0032] a determination module, configured to determine second information based on the first information; wherein the second information includes at least one of the following: an availability evaluation result of the target application determined based on the service availability information; a resource usage result of the target application determined based on the resource usage information; a log evaluation result of the target application determined based on the log information; or an indicator evaluation result of the target application determined based on the business indicator information;

[0033] A generation module is used to generate an observability assessment report based on the second information.

[0034] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0035] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect.

[0036] This application proposes a method, apparatus, device, medium, and product for evaluating applications in a cloud-native environment. The method monitors first information of a target application, wherein the target application is any application in an application system. The first information includes at least one of the following: service availability information, resource usage information, log information, and business indicator information. Based on the first information, second information is determined. The second information includes at least one of the following: an availability evaluation result of the target application determined based on the service availability information; a resource usage result of the target application determined based on the resource usage information; a log evaluation result of the target application determined based on the log information; and an indicator evaluation result of the target application determined based on the business indicator information. Based on the second information, an observability evaluation report is generated. This application obtains service availability information, resource usage information, log information, and business indicator information corresponding to the target application and utilizes this information to obtain an evaluation result for the corresponding application. Furthermore, based on multiple or one of the evaluation results for the corresponding application, an observability evaluation report is generated. This improves the ability to evaluate applications in a cloud-native environment. The observability assessment report can provide key clues for operation and maintenance personnel. By analyzing the observability assessment report, they can quickly locate the specific applications or links involved in the problem, and then perform targeted repairs and optimizations to ensure the normal operation of the application system, improve the maintenance capabilities of the application system in the cloud-native environment, and further save maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flowchart of a method for evaluating applications in a cloud-native environment provided by an embodiment of the present application;

[0038] Figure 2 yes Figure 1 Flowchart of step S102 in FIG.

[0039] Figure 3 yes Figure 1 Flowchart of step S101 in FIG.

[0040] Figure 4 yes Figure 1 Flowchart of step S102 in FIG.

[0041] Figure 5 yes Figure 1 Flowchart of step S102 in FIG.

[0042] Figure 6 yes Figure 1 Flowchart of step S101 in FIG.

[0043] Figure 7 This is a schematic diagram of the structure of an evaluation device applied in a cloud native environment provided by an embodiment of the present application;

[0044] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0046] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0048] First, let’s analyze some of the terms used in this application:

[0049] In a cloud-native environment, business is the core goal, applications are the specific implementation of business logic, application systems are the infrastructure for application operation, and services are the modular implementation of application functions.

[0050] Application systems can be insurance systems, banking systems, transaction systems, order systems, etc.

[0051] Third-party monitoring is the process of monitoring and evaluating services, products, and systems, typically conducted by a third-party organization or platform independent of the service provider and user. In assessing the observability of application systems, third-party monitoring can provide objective and impartial results, helping businesses and organizations better understand system performance, availability, and user experience.

[0052] Rights management can be a management process that ensures the legal and safe use of resources by controlling the access rights of users or systems to resources.

[0053] Based on this, the embodiments of the present application provide a method, apparatus, equipment, medium and product for evaluating applications in a cloud-native environment, aiming to improve the maintenance capabilities of application systems in a cloud-native environment.

[0054] The evaluation method, apparatus, equipment, medium, and product for applications in a cloud-native environment provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the evaluation method for applications in a cloud-native environment in the embodiments of the present application is described.

[0055] The evaluation method for applications in a cloud-native environment provided in the embodiment of the present application relates to the field of cloud-native technology. The evaluation method for applications in a cloud-native environment provided in the embodiment of the present application can be applied to electronic devices, and further, can be applied to the software of electronic devices. The electronic device can be a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the evaluation method for applications in a cloud-native environment, etc., but is not limited to the above forms.

[0056] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0057] Although the embodiment of the present application is an evaluation method applied in a cloud-native environment, the method of the embodiment of the present application can be applied in a non-native environment as long as the first information can be monitored.

[0058] Figure 1 This is an optional flowchart of the evaluation method for applications in a cloud-native environment provided by an embodiment of the present application. Figure 1The method may include but is not limited to steps S101 to S103.

[0059] Step S101: monitoring first information of a target application; wherein the target application is any application in the application system; the first information includes at least one of the following: service availability information, resource usage information, log information, and business indicator information;

[0060] Step S102: Determine second information based on the first information; wherein the second information includes at least one of the following: an availability evaluation result of the target application determined based on the service availability information; a resource usage result of the target application determined based on the resource usage information; a log evaluation result of the target application determined based on the log information; or an indicator evaluation result of the target application determined based on the business indicator information.

[0061] Step S103: Generate an observability assessment report based on the second information.

[0062] Steps S101 to S103 shown in the embodiment of the present application monitor the first information of the target application; wherein the target application is any application in the application system; the first information includes at least one of the following: service availability information, resource usage information, log information and business indicator information; according to the first information, the second information is determined; wherein the second information includes at least one of the following: the available evaluation result of the target application determined based on the service availability information; the resource usage result of the target application determined based on the resource usage information; the log evaluation result of the target application determined based on the log information; the indicator evaluation result of the target application determined based on the business indicator information; according to the second information, an observability evaluation report is generated. The present application obtains the service availability information, and / or resource usage information, and / or log information, and / or business indicator information corresponding to the target application, and uses this information to obtain the evaluation result of the corresponding application. Furthermore, based on multiple or one of the evaluation results of the corresponding application, an observability evaluation report is generated. In this way, the evaluation capability of applications in a cloud native environment is improved. The observability assessment report can provide key clues for operation and maintenance personnel. By analyzing the observability assessment report, they can quickly locate the specific applications or links involved in the problem, and then perform targeted repairs and optimizations to ensure the normal operation of the application system, improve the maintenance capabilities of the application system in the cloud-native environment, and further save maintenance costs.

[0063] In step S101 of some embodiments, first information of any application in the application system is monitored, where the first information includes at least one of service availability information, resource usage information, log information, and business indicator information.

[0064] In step S102 of some embodiments, an availability evaluation result of the target application is determined based on the service availability information. A resource usage result of the target application is determined based on the resource usage information. A log evaluation result of the target application is determined based on the log information. A metric evaluation result of the target application is determined based on the business metric information. The second information includes at least one of the aforementioned evaluation results.

[0065] In step S103 of some embodiments, an observability assessment report is automatically generated using at least one of the above assessment results.

[0066] It should be noted that the target application can be any application in the application system and is the object of evaluation. For example, in an insurance system, the target application can be a claims management application.

[0067] The service availability information may reflect whether the service interface of the target application can respond to requests normally.

[0068] Resource usage information may reflect the computing, storage, network, and other resources used by the target application.

[0069] The log information may be log data generated during the operation of the target application.

[0070] The business indicator information may be an indicator related to the business of the target application, such as the amount of business occurring, interface performance, future usage trends, etc.

[0071] The second information may be result information determined based on the first information, and the second information may include at least one of the following: an available evaluation result, a resource usage result, a log evaluation result, and an indicator evaluation result.

[0072] The observability assessment report may be a comprehensive report that summarizes at least one assessment result in the second information, and is used to help maintain and optimize the application system.

[0073] For example, an insurance company has an insurance management system built on multiple applications, including but not limited to policy management, claims management, customer information management, and premium calculation. To maintain stable operation and efficient management of the entire application system, the company decided to conduct an observability assessment of these applications. The following assessment will focus on the claims management application.

[0074] Step 1: Monitor the first information of the target application.

[0075] 1. Service availability information monitoring:

[0076] Monitor whether the service interfaces of the claims management application (such as the claim application submission interface and the claim progress query interface) can respond to requests normally. For example, within a period of time, 100 service call requests were sent, 95 of which received successful responses and 5 received server errors.

[0077] It should be noted that if the service interface is not found, a service call request is sent through the automation platform and the response is monitored.

[0078] 2. Resource usage information monitoring:

[0079] Monitoring resource usage of the claims management application showed an average CPU usage of 75% and memory usage of 80%.

[0080] In addition, we monitored the response rate of the application to service call requests and found that the average response time was 200ms.

[0081] 3. Log information monitoring:

[0082] Check that the claims management application logs exist and are stored in a specific directory on the local server.

[0083] The log content records detailed information such as operation time, operation type, operation results, etc., and the log is output according to the importance level (such as INFO, WARNING, ERROR).

[0084] 4. Business indicator information monitoring:

[0085] Obtain the number of claim applications generated by the claims management application. Statistics show that 1,000 claim applications occurred in the past month.

[0086] Monitoring the performance of the claims application interface, the average response time is 200ms and the throughput is 100 requests per second.

[0087] Analyzing historical claims data, it is predicted that the number of claims applications will increase by 20% in the next month. The interface performance needs to support 120 requests per second.

[0088] The second step is to determine the second information based on the first information.

[0089] 1. Available assessment results can be used to determine:

[0090] Since 95% of the service call requests were successfully responded to, the availability assessment result of the claims management application was determined to be "available".

[0091] 2. Determination of resource usage results:

[0092] Comparing resource usage (such as CPU usage 75%, memory usage 80%, etc.) with benchmark performance indicators (such as CPU benchmark 60%, memory benchmark 70%, etc.), it is found that resource usage pressure is relatively high.

[0093] Combined with the monitoring results of the response rate (average 200ms), the resource usage result is comprehensively determined to be "high pressure, needs optimization".

[0094] 3. Log evaluation results are determined:

[0095] Based on the completeness and accuracy of the log content, as well as the rationality of the log storage location and output format, the comprehensive assessment result of the log is "good".

[0096] 4. Determination of indicator evaluation results:

[0097] A weighted average calculation of the volume of claims application business, interface performance, and future usage trends yielded an index evaluation result of 0.79 (out of a full score of 1.0), indicating good overall business performance.

[0098] Step 3: Generate an observability assessment report based on the second information.

[0099] Report Contents:

[0100] Usability evaluation: The service interface of the claims management application is available with a success rate of 95%.

[0101] Resource usage assessment: Resource usage pressure is high, and it is recommended to optimize or expand resources.

[0102] Log evaluation: The log evaluation results are good, the log content is complete and the output format is standard.

[0103] Business indicator evaluation: The overall performance of the claims application business is good, with an indicator evaluation result of 0.79.

[0104] Detailed information on each first message can also be attached as an appendix, allowing operations personnel to better maintain the application system. Furthermore, developers can optimize the performance of the claims application interface based on business metric evaluation results to meet future growth needs. Business personnel can use these results to plan business strategies in advance to address future growth in claims applications.

[0105] In a cloud-native environment, there are no fixed computing resources; services are generated on demand, with only fixed external service interfaces. Therefore, traditional solutions are no longer feasible. Instead, we only need to evaluate the service availability interface. Currently, availability monitoring can be performed by assessing whether the system to be launched supports external services. This is because services based on computing resources are stable and conform to the fluid nature of the computing power pool.

[0106] See also Figure 2 In some embodiments, the service availability information includes: the response information of the target application to the service call request; step S102 may include but is not limited to steps S201 to S202:

[0107] Step S201: When the response information is monitored, determining that the availability assessment result is available;

[0108] Step S202: If the response information is not detected, determining that the availability evaluation result is unavailable.

[0109] In step S201 of some embodiments, if response information of the target application to the service call request is monitored, the availability evaluation result is available.

[0110] In step S202 of some embodiments, if no response information of the target application to the service call request is monitored, the availability evaluation result is unavailable.

[0111] In the embodiment of the present application, it should be noted that the service availability information may be information reflecting whether the service interface of the target application can normally respond to the request, including the response status code, response time, etc.

[0112] The response information may be a response of the target application to the service call request, including a success response (such as 200 OK) or a failure response (such as 500 Internal Server Error).

[0113] The availability evaluation result may be an availability status of the target application determined according to the response information, and may be “available” or “unavailable”.

[0114] The acquisition of a service call request depends on whether the target application has a corresponding service interface. If the target application does not have a corresponding service interface, the service call request is sent through the automation platform. If the target application does have a corresponding service interface, the service call request is sent by the application system to test the service availability.

[0115] In some implementations (providing a specific interface), if a corresponding service interface exists for the target application, a service call request corresponding to the service interface of the target application is received; the response information of the service interface to the service call request is monitored; wherein the service interface includes a public service interface or a private service interface; the public service interface is the interface corresponding to each application; the specific service interface is the interface corresponding to the target application; the service availability information includes the response information of the service interface to the service interface request corresponding to the target application; if the response information is monitored, the availability evaluation result is determined to be available; if the response information is not monitored, the availability evaluation result is determined to be unavailable. In this implementation, it can also be divided into the following situations:

[0116] 1. Monitor the response information of the service call request corresponding to the private service interface, i.e., the service interface corresponding to the target application; and determine the available evaluation results based on the response information corresponding to the specific service interface.

[0117] Second, first monitor the response information for service call requests corresponding to the private service interface (i.e., the target application's corresponding service interface). If no response information exists for the private service interface, monitor the response information for service call requests corresponding to the public service interface (i.e., the common interface provided by each application). Determine the available evaluation results based on the response information corresponding to the public service interface.

[0118] In other implementations (providing specific services), when the target application does not have a corresponding service interface, the automation platform is used to send a service call request corresponding to the target application; the response information of the target application to the service call request is monitored; when the response information is monitored, the availability evaluation result is determined to be available; when the response information is not monitored, the availability evaluation result is determined to be unavailable.

[0119] In the case that the application system supports current platform detection, the service availability can be evaluated by the specific services and specific interfaces provided. Through these specific interfaces and specific services, the accuracy of application system availability detection can be evaluated.

[0120] For example, let's assume a claims management application in an insurance system, which is responsible for processing customer claims. The claims management application is evaluated as the target application.

[0121] Step 1: Monitor the first information of the target application.

[0122] Service call request:

[0123] First service call request: Send a request through the public service interface or private service interface of the claims management application, such as "submit a claim application" or "query claim progress".

[0124] Monitoring response information:

[0125] Sample data: 100 service call requests were sent, 95 of which returned a 200 OK response, and 5 returned a 500 Internal Server Error response.

[0126] Step 2: Determine the second information based on the first information.

[0127] Available assessment results:

[0128] Service call request: 95 successful responses, with a success rate of 95%. The evaluation result is "available".

[0129] No response information is detected: If the request times out or there is no response, the evaluation result is "Unavailable".

[0130] In the embodiments of the present application, by monitoring the response information of the service call request, anomalies of the service interface can be discovered in a timely manner, thereby improving the reliability of the system. Furthermore, unavailable service interfaces can be identified and repaired in a timely manner to ensure the high availability of the application system. Clear availability assessment results can help the operation and maintenance team quickly locate problems and simplify the troubleshooting process. Available service interfaces can ensure that user requests are responded to in a timely manner, thereby improving the user experience. For applications that do not have service interfaces, by simulating the sending of requests, deficiencies in the application system design can be discovered and the interface design can be optimized.

[0131] See also Figure 3 In some embodiments, when the first information includes service availability information, step S101 may include but is not limited to steps S301 to S302:

[0132] Step S301: When the target application does not have a corresponding public service interface and private service interface, the service call request is sent using the automation platform;

[0133] Step S302: monitoring the response information of the target application to the service call request.

[0134] In step S301 of some embodiments, when the target application has neither a corresponding private service interface nor a corresponding public service interface, a service call request is simulated and sent through the automation platform.

[0135] In step S302 of some embodiments, the response of the target application to the request sent by the automation platform is recorded.

[0136] Consider an insurance system's claims management application, which handles customer claims. Select the claims management application as the target application for evaluation.

[0137] Step 1: Monitor the first information of the target application.

[0138] If the target application does not have a service interface, that is, the claims management application does not have a predefined service interface, service calls cannot be made directly through the interface. The automation platform simulates sending a claim application request or a claim progress query request.

[0139] Monitoring response information:

[0140] Sample data: 50 service call requests were sent, 45 of which returned a 200OK response, and 5 times out without a response.

[0141] Step 2: Identify available assessment results.

[0142] Available assessment results:

[0143] Available: 45 successful responses, with a success rate of 90%, and the evaluation result is "Available".

[0144] The embodiments of the present application ensure that even applications without predefined service interfaces can be evaluated, thereby improving the comprehensiveness of the evaluation. Secondly, for applications that do not have service interfaces, sending requests through the automation platform can identify deficiencies in the system design, optimize the interface design, and ensure that all key functions are accessible through the interface. In addition, the use of the automation platform improves monitoring efficiency, reduces manual intervention, and improves operation and maintenance efficiency. It can also ensure that all applications have corresponding service requests, providing a consistent user experience.

[0145] In a cloud-native environment, fixed resources don't exist. Traditional resource detection and response models can be replaced by service availability: if services are available, resources must exist. Resource redundancy can be assessed through service performance. If service performance is within the specified range, resources are sufficient. This avoids the problem of observing a dynamically changing resource pool.

[0146] See also Figure 4 In some embodiments, the resource usage information includes: resource usage of the target application, and the response rate of the target application to the service call request; step S102 may include but is not limited to steps S401 to S403:

[0147] Step S401: comparing the resource usage with a benchmark performance indicator to determine a first resource usage result; wherein the benchmark performance indicator is determined based on a performance test or an average value of resource usage over a preset time period;

[0148] Step S402: determining a second resource usage result based on the response rate;

[0149] Step S403: Determine the resource usage result based on the first resource usage result and / or the second resource usage result.

[0150] In step S401 of some embodiments, when the resource usage information includes the resource usage of the target application and the response rate of the target application corresponding to the service call request, a benchmark performance indicator is obtained, and a comparison result is obtained by comparing the benchmark performance indicator and the resource usage. Based on the comparison result, it is determined whether the resource usage is within an appropriate range.

[0151] In step S402 of some embodiments, if the target application has a target service interface, it is determined whether the response capability of the target application meets the requirements based on the response rate.

[0152] In step S403 of some embodiments, the resource usage of the target application is comprehensively evaluated by combining the first resource usage result and the second resource usage result.

[0153] In the embodiment of the present application, it should be noted that resource usage may be computing resources (such as CPU, memory, disk I / O) and network bandwidth used by the target application.

[0154] The response rate can be the response time of the target application to the service call request, reflecting the processing efficiency of the application.

[0155] Baseline performance indicators can be obtained through stress testing of the target application's performance. This stress testing determines the peak resource usage, and 60%-70% of this resource usage can be used as the baseline performance indicator. The percentage of resource usage used as the baseline performance indicator can be determined based on actual circumstances. Alternatively, if stress testing is not performed beforehand, the baseline performance indicator can be obtained by recording the target application's resource usage over a preset time period and taking the average value of resource usage over that time period. This means first verifying whether the development and testing phase provides baseline performance indicators. If not, equivalent baseline performance indicators can be obtained indirectly through monitoring cluster resources and application resource allocation.

[0156] The first resource usage result may be an evaluation of whether resource usage is within an appropriate range based on a comparison of resource usage with a benchmark performance indicator. For example, by comparing the current resource usage with the benchmark performance indicator, if the current resource usage is less than the benchmark performance indicator, the current resource usage is insufficient; if it is equal to the benchmark performance indicator, the current resource usage is sufficient; and if it is greater than the benchmark performance indicator, the current resource usage is excessive.

[0157] The second resource usage result can be to evaluate whether the target application's responsiveness meets the requirements based on the response rate. This can be understood as checking whether the core process of the application system has relevant services or interfaces, and assessing whether resources are scarce based on the response of the interfaces.

[0158] The resource usage result may be a final resource usage assessment result obtained by combining the first resource usage result and the second resource usage result. The resource usage result may be obtained by directly combining the first resource usage result and the second resource usage result, or by weighting the first resource usage result and the second resource usage result to obtain a weighted average.

[0159] For example, let's consider a claims management application in an insurance system, which is responsible for processing customer claims. The claims management application is evaluated.

[0160] Step 1: Monitor the first information of the target application.

[0161] Monitor resource usage:

[0162] Sample data: CPU usage is 75% on average and memory usage is 85%.

[0163] Monitoring response rate:

[0164] Service call request: When the claims management application does not have a service interface, the average response time for the "query claim progress" request sent by the automation platform is 300ms.

[0165] Step 2: Determine the second information based on the first information.

[0166] Baseline performance indicators: The baseline values ​​determined based on performance testing are 60% CPU usage and 70% memory usage.

[0167] First, resource usage results: CPU and memory usage are higher than the baseline, indicating high resource usage pressure.

[0168] Response rate benchmark value: The set response time benchmark value is 150ms.

[0169] Second resource usage result: The response time of the second service call request is 300ms, which exceeds the benchmark value, indicating that the response rate is slow.

[0170] Resource usage results: Combining the first and second resource usage results, the comprehensive assessment of resource usage of the claims management application is "high pressure and slow response rate."

[0171] By monitoring resource usage and response rates, the present application embodiment can identify resource bottlenecks, optimize resource allocation, and improve application system performance. It provides detailed resource usage assessment results, helping the operation and maintenance team quickly locate and resolve problems (promptly discover and resolve issues such as excessive or insufficient resource usage), simplifying operation and maintenance management.

[0172] In a cloud-native environment, computing resources are only temporarily allocated and may be created and destroyed at any time. Therefore, logs are the most important and perhaps the only solution for post-event tracing and analysis, and they need to be communicated and clarified.

[0173] See also Figure 5 In some embodiments, the log information includes the log storage location, log content, and log output format of the log; step S102 may include but is not limited to steps S501 to S503:

[0174] Step S501: determining a first log evaluation result based on the relevance between the log content of the log and preset log content; wherein the preset log content is a content template designed by a developer;

[0175] Step S502: Determine a second log evaluation result based on the log storage location and the log output format of the log; wherein the log storage location is used to determine whether the log of the target application is stored by a third party; and the log output format is used to determine whether the log of the target application is output according to importance.

[0176] Step S503: Determine the log evaluation result based on the first log evaluation result and the second log evaluation result.

[0177] In step S501 of some embodiments, when a target application generates a log, the content of the log is monitored to check whether it conforms to a preset content template, thereby determining whether the record is comprehensive.

[0178] In step S502 of some embodiments, the log storage location is monitored to determine whether it is third-party storage. Furthermore, the log output format is monitored to determine whether the log output format is output according to importance. Specifically, whether the log design takes into account the environment of immediate distribution and destruction, and whether the log is stored by a third party, such as an ELK mechanism, is considered. For critical business processes, whether a multi-level log output mechanism (according to importance) is designed.

[0179] In step S503 of some embodiments, the first log evaluation result and the second log evaluation result are combined to comprehensively evaluate the log quality of the target application. Specifically, the observability of the logs can be examined to assess the amount of logs available for observation and whether maintenance requirements for important services are met. For example, the comprehensive evaluation can be performed by directly combining the first and second log evaluation results, or by combining the first and second log evaluation results in a weighted average or other form.

[0180] In the embodiment of the present application, it should be noted that the first log evaluation result may be a result of evaluation based on the log content and a preset template.

[0181] The second log evaluation result may be a result of evaluation based on the log storage location and output format.

[0182] The log evaluation result may be a final evaluation result obtained by combining the first log evaluation result and the second log evaluation result. The log evaluation result may be obtained by directly combining the first log evaluation result and the second log evaluation result, or by weightedly averaging the first log evaluation result and the second log evaluation result according to their weights.

[0183] For example, let's consider a claims management application in an insurance system, which is responsible for processing customer claims. The claims management application is evaluated as the target application.

[0184] Step 1: Monitor the first information of the target application.

[0185] Check whether the claims management application generates log files.

[0186] Sample data: A log file exists.

[0187] Then, monitor the log information:

[0188] Log storage location: Log files are stored in the / var / log / claims directory on the local server.

[0189] Log content: The log records information such as operation time, operation type, and operation results.

[0190] Log output format: Logs are output according to their importance and are divided into levels such as INFO, WARNING, and ERROR.

[0191] Step 2: Determine the second information based on the first information.

[0192] Determine the first log evaluation result based on the log content and the preset template:

[0193] Preset log content template: A template designed by the developer that requires the log to record the operation time, operation type, operation result, user ID, and claim ID.

[0194] Evaluation result: The log content complies with the preset template and the evaluation result is "good".

[0195] Determine the second log evaluation result based on the log storage location and output format:

[0196] Log storage location: Logs are stored on the local server, not in a third-party storage.

[0197] Log output format: Logs are output according to their importance and are divided into levels such as INFO, WARNING, and ERROR.

[0198] Evaluation result: The log storage location and output format meet the requirements, and the evaluation result is "good".

[0199] Final log evaluation results:

[0200] Log evaluation result: Combining the first log evaluation result and the second log evaluation result, the final evaluation result is "good".

[0201] The embodiments of the present application can improve the efficiency of log management and ensure the integrity and readability of logs by monitoring the log storage location, content, and output format. By evaluating the integrity and accuracy of log content, application problems can be quickly located and diagnosed, reducing troubleshooting time. By determining whether logs are stored in third-party storage, the security and compliance of log data can be ensured. By evaluating the log output format, it is ensured that logs are output according to their importance, thereby improving the readability and usability of logs. Detailed log evaluation results are provided to help the operation and maintenance team quickly locate and solve problems, simplifying operation and maintenance management.

[0202] In a cloud-native environment, services are the centerpiece, not the IT infrastructure. Maintaining overall service readiness is far more important than individual service replicas. Overall stability and local dynamism coexist. Therefore, the most effective approach to data collection is to extract or select data (metric information) that represents a specific characteristic of an application system or service. This data can be extracted by the application itself or through auxiliary containers, and representative results are then calculated and reported.

[0203] See also Figure 6 In some embodiments, the target application includes at least one business; when the first information includes business indicator information, step S101 includes but is not limited to steps S601 to S603:

[0204] Step S601: obtaining the occurrence volume of the target business corresponding to the target application; wherein the target business is determined by the developer according to the requirements of the target application;

[0205] Step S602, obtaining the interface performance corresponding to the target service corresponding to the target application;

[0206] Step S603: deriving a future usage trend of the target business corresponding to the target application by analyzing historical business data; wherein the business indicator information includes the occurrence volume, the interface performance, and the future usage trend;

[0207] In step S601 of some embodiments, the number of occurrences of the target service within a certain period of time is recorded.

[0208] In step S602 of some embodiments, performance indicators such as response time and throughput of the interface corresponding to the target service are monitored.

[0209] In step S603 of some embodiments, the occurrence volume of the target service, interface performance data, user activity, etc. in the past period of time are obtained, and the future usage trend of the target service is predicted through data analysis.

[0210] In the embodiment of the present application, it should be noted that the target service may be determined by manual selection or a preset mapping table. For example, in an insurance system, the target service may be a claim application service.

[0211] The occurrence volume can be the number of times the target business occurs within a certain period of time, reflecting the frequency of use of the business.

[0212] Interface performance can be performance indicators such as response time and throughput of the interface corresponding to the target service, reflecting the efficiency and stability of the interface.

[0213] The future usage trend can be the future usage and performance requirements of the target business predicted by analyzing historical business data.

[0214] Business indicator information may include information on occurrence volume, interface performance, and future usage trends.

[0215] In some implementations, the occurrence volume, interface performance, and future usage trends may be weighted averaged or directly combined to determine the indicator evaluation result.

[0216] In other implementations, an evaluation system can be constructed, including occurrence volume, interface performance, and future usage trends. The weights of each indicator are determined through expert scoring, hierarchical analysis, and other methods. A comprehensive evaluation score is calculated based on the scores and weights of each indicator.

[0217] For example, let's assume a claims management application in an insurance system, which is responsible for processing customer claims. The claims management application is evaluated as the target application.

[0218] Step 1: Monitor the first information of the target application.

[0219] Get the occurrence volume of the target business:

[0220] Number of claims: In the past month, the number of claims applications was 1,000.

[0221] Get the interface performance corresponding to the target business:

[0222] Interface performance: The average response time of the claims application interface is 200ms, and the throughput is 100 requests per second.

[0223] Analyze historical business data to obtain future usage trends:

[0224] Claims application volume and interface performance data for the past 12 months. Data analysis predicts that claims application volume will increase by 20% in the next month, and the interface performance needs to support 120 requests per second.

[0225] Step 2: Determine the second information based on the first information.

[0226] Take a weighted average of the occurrence volume, interface performance, and future usage trends:

[0227] Set weights: occurrence weight: 30%, interface performance weight: 40%, future usage trend weight: 30%.

[0228] Standardized data: Occurrence volume: 1000 times (standardized value: 0.8), interface performance: 200ms (standardized value: 0.7), future usage trend: 20% growth (standardized value: 0.9).

[0229] Weighted average calculation: Index evaluation result = (0.3×0.8)+(0.4×0.7)+(0.3×0.9)=0.24+0.28+0.27=0.79.

[0230] Determine the indicator evaluation results: The indicator evaluation result is 0.79 (out of 1.0), indicating that the overall performance of the claims application business is good.

[0231] The embodiments of this application comprehensively evaluate the performance of the target business by comprehensively considering the occurrence volume, interface performance, and future usage trends. Based on the prediction of future usage trends, resource allocation can be planned and optimized in advance to ensure that the application system can cope with future business growth. By evaluating interface performance, performance bottlenecks can be identified and resolved, thereby improving the overall performance of the application system. By providing detailed business indicator evaluation results, data support is provided for business decision-making and helps to formulate reasonable business strategies.

[0232] See also Figure 7 , an embodiment of the present application further provides an evaluation device for applications in a cloud-native environment, which can implement the above-mentioned evaluation method for applications in a cloud-native environment, and the device includes:

[0233] Monitoring module 701, configured to monitor first information of a target application; wherein the target application is any application in an application system; the first information includes at least one of the following: service availability information, resource usage information, log information, and business indicator information;

[0234] Determining module 702 is configured to determine second information based on the first information; wherein the second information includes at least one of the following: an availability evaluation result of the target application determined based on the service availability information; a resource usage result of the target application determined based on the resource usage information; a log evaluation result of the target application determined based on the log information; or an indicator evaluation result of the target application determined based on the business indicator information.

[0235] The generating module 703 is configured to generate an observability assessment report based on the second information.

[0236] The specific implementation of the evaluation device applied in the cloud native environment is basically the same as the specific implementation of the evaluation method applied in the above-mentioned cloud native environment, and will not be repeated here.

[0237] The present application also provides an electronic device comprising a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned method for evaluating applications in a cloud-native environment. The electronic device can be any smart terminal, including a tablet computer and an in-vehicle computer.

[0238] See also Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0239] The processor 801 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0240] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the evaluation method for applications in the cloud native environment of the embodiments of this application;

[0241] Input / output interface 803, used to implement information input and output;

[0242] Communication interface 804, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0243] Bus 805 , which transmits information between various components of the device (e.g., processor 801 , memory 802 , input / output interface 803 , and communication interface 804 );

[0244] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0245] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the evaluation method for applications in the above-mentioned cloud-native environment.

[0246] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0247] An embodiment of the present application also provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the evaluation method for applications in the above-mentioned cloud-native environment.

[0248] The embodiment of the present application provides an evaluation method, device, equipment, medium and product for applications in a cloud-native environment, which monitors the first information of the target application; wherein the target application is any application in the application system; the first information includes at least one of the following: service availability information, resource usage information, log information and business indicator information; based on the first information, the second information is determined; wherein the second information includes at least one of the following: the available evaluation result of the target application determined based on the service availability information; the resource usage result of the target application determined based on the resource usage information; the log evaluation result of the target application determined based on the log information; the indicator evaluation result of the target application determined based on the business indicator information; based on the second information, an observability evaluation report is generated. The present application obtains the service availability information, and / or resource usage information, and / or log information, and / or business indicator information corresponding to the target application, and uses this information to obtain the evaluation result of the corresponding application. Furthermore, based on multiple or one of the evaluation results of the corresponding application, an observability evaluation report is generated. In this way, the evaluation capability of applications in a cloud-native environment is improved. The observability assessment report can provide key clues for operation and maintenance personnel. By analyzing the observability assessment report, they can quickly locate the specific applications or links involved in the problem, and then perform targeted repairs and optimizations to ensure the normal operation of the application system, improve the system maintenance capabilities in the cloud-native environment, and further save maintenance costs.

[0249] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0250] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0251] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0252] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0253] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0254] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0255] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0256] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0257] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0258] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0259] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A method for evaluating applications in a cloud-native environment, characterized in that: Applied to an electronic device, the electronic device is provided with a cloud-native environment, and the electronic device is installed with an application system running in the cloud-native environment, the application system including at least one application; the method includes: Monitoring first information of a target application; wherein the target application is any application in the application system; the first information includes at least one of the following: service availability information, resource usage information, log information, and business indicator information; Determine second information based on the first information; wherein the second information includes at least one of the following: an availability evaluation result of the target application determined based on the service availability information; a resource usage result of the target application determined based on the resource usage information; a log evaluation result of the target application determined based on the log information; or an indicator evaluation result of the target application determined based on the business indicator information; Generate an observability assessment report based on the second information.

2. The method according to claim 1, characterized in that The service availability information includes: response information of the target application to the service call request; The determining the second information according to the first information includes: In the case where the response information is monitored, determining that the availability assessment result is available; In the case that the response information is not monitored, it is determined that the available evaluation result is unavailable.

3. The method according to claim 2, characterized in that The first information of the monitoring target application includes: In the case that the target application does not have a corresponding public service interface and private service interface, sending the service call request by using an automation platform; Monitor the response information of the target application to the service call request.

4. The method according to claim 1, wherein The resource usage information includes: resource usage of the target application, and a response rate of the target application corresponding to the service call request; The determining the second information according to the first information includes: Comparing the resource usage with a benchmark performance indicator to determine a first resource usage result; wherein the benchmark performance indicator is determined based on a performance test or an average value of resource usage over a preset time period; determining a second resource usage result based on the response rate; The resource usage result is determined based on the first resource usage result and / or the second resource usage result.

5. The method according to claim 1, wherein The log information includes the log storage location, log content and log output format; The determining the second information according to the first information includes: Determining a first log evaluation result based on a degree of relevance between the log content of the log and a preset log content; wherein the preset log content is a content template designed by a developer; Determining a second log evaluation result based on the log storage location and the log output format of the log; wherein the log storage location is used to determine whether the log of the target application is stored by a third party; and the log output format is used to determine whether the log of the target application is output according to importance; The log evaluation result is determined based on the first log evaluation result and the second log evaluation result.

6. The method according to claim 1, characterized in that The target application includes at least one business; when the first information includes the business indicator information, the first information of the monitored target application includes: Obtaining the occurrence volume of a target business corresponding to the target application; wherein the target business is determined by a developer based on the requirements of the target application; Obtaining interface performance corresponding to a target service corresponding to the target application; By analyzing historical business data, a future usage trend of the target business corresponding to the target application is obtained; wherein the business indicator information includes the occurrence volume, the interface performance and the future usage trend.

7. An evaluation device for applications in a cloud native environment, characterized in that: The device comprises: A monitoring module configured to monitor first information of a target application; wherein the target application is any application in an application system; and the first information includes at least one of the following: service availability information, resource usage information, log information, and business indicator information; a determination module, configured to determine second information based on the first information; wherein the second information includes at least one of the following: an availability evaluation result of the target application determined based on the service availability information; a resource usage result of the target application determined based on the resource usage information; a log evaluation result of the target application determined based on the log information; or an indicator evaluation result of the target application determined based on the business indicator information; A generation module is used to generate an observability assessment report based on the second information.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the evaluation method for applications in a cloud native environment as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the evaluation method for applications in a cloud-native environment according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the evaluation method for applications in a cloud-native environment as described in any one of claims 1 to 6.

Images