Method, device and equipment for determining quality of cloud-native intelligent agent and medium

Abstract

The application relates to a quality determination method, device and equipment of a cloud native intelligent agent and a medium. The method comprises the following steps: in response to an evaluation request for a cloud native intelligent agent, obtaining observation data of the cloud native intelligent agent; based on the observation data and an index evaluation model, determining quality parameter values of each quality index corresponding to the cloud native intelligent agent; each of the quality indexes comprises at least two of a stability evaluation index, a safety evaluation index, an expansion evaluation index and a business efficiency index; and based on the quality parameter values of each of the quality indexes, determining a quality evaluation result of the cloud native intelligent agent. The above method not only focuses on the algorithm and decision-making ability of the cloud native intelligent agent, but also covers reliability, safety and scalability, so as to reflect the ability of the intelligent agent in fault recovery, safety protection and environmental adaptability, pay more attention to the distributed and dynamic characteristics in the cloud native environment, and more comprehensively and accurately determine the quality of the intelligent agent.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence, in particular to a quality determination method, device and equipment of a cloud-native agent and a medium. BACKGROUND

[0002] With the rapid development of cloud computing technology and the increasing popularity of artificial intelligence applications, cloud-native agents (Agents) have emerged. Cloud-native Agents rely on the elasticity, scalability and distributed characteristics of cloud-native architecture, combined with the autonomous decision-making, learning and interaction capabilities of Agents, to exhibit great application potential in intelligent operation and maintenance, intelligent customer service, industrial automation and many other fields.

[0003] Cloud-native technology emphasizes the use of containerization, microservices architecture, automated deployment and management to build and run applications to fully leverage the advantages of cloud computing. Cloud-native Agents deeply integrate the above technologies with Agent technology to achieve more efficient and flexible intelligent services. However, the quality of cloud-native Agents is directly related to service effectiveness and business stability, and how to accurately and comprehensively determine the quality of cloud-native Agents has become a key problem to be solved. SUMMARY

[0004] Therefore, it is necessary to provide a quality determination method, device and equipment of a cloud-native agent that can accurately and comprehensively determine the quality of cloud-native Agents.

[0005] In a first aspect, the application provides a quality determination method of a cloud-native agent, comprising:

[0006] In response to an evaluation request for a cloud-native Agent, obtaining observation data of the cloud-native Agent, the observation data being running state data and / or behavior data of executing a business of the cloud-native Agent.

[0007] Based on the observation data and an index evaluation model, determining quality parameter values of each quality index corresponding to the cloud-native Agent; each of the quality indexes includes at least two of a stability evaluation index, a security evaluation index, an expansion evaluation index and a business efficiency index, the stability evaluation index being used to represent the fault recovery capability of the cloud-native Agent in executing a business, the security evaluation index being used to represent the security protection capability of the cloud-native Agent, the expansion evaluation index being used to represent the adaptability of the cloud-native Agent when the environment changes, and the business efficiency index being used to represent the function completion capability and performance of the cloud-native Agent in executing a business.

[0008] Based on the quality parameter values of each of the quality indexes, determining a quality evaluation result of the cloud-native Agent.

[0009] In one of the embodiments, the determining of the quality parameter values of the quality indicators corresponding to the cloud native intelligent agent based on the observation data and the indicator evaluation model comprises:

[0010] The quality parameter values of the quality indicators of a lowest layer in a preset hierarchical structure of the quality indicators are determined based on the observation data and the indicator evaluation model; the preset hierarchical structure comprises at least two levels, each level comprises at least one group of quality indicators, and each group of quality indicators comprises at least one quality indicator; two adjacent levels in the at least two levels are an adjacent upper level and an adjacent lower level, and any upper level quality indicator of the adjacent upper level corresponds to a group of lower level quality indicators of the adjacent lower level.

[0011] The quality parameter values of the quality indicators of a target layer in the preset hierarchical structure are determined based on the quality parameter values of the lowest layer; the target layer is any layer in the preset hierarchical structure.

[0012] In one of the embodiments, the determining of the quality parameter values of the quality indicators of the target layer in the preset hierarchical structure based on the quality parameter values of the lowest layer comprises:

[0013] In a case where the target layer is not the lowest layer, relative weight parameter values of the quality indicators of each lower layer corresponding to the target layer are obtained; the relative weight parameter of the quality indicator is obtained by performing hierarchical analysis on each quality indicator according to the relative importance between the quality indicators in the same group.

[0014] The quality parameter values of the quality indicators of the target layer are determined based on the quality parameter values of the lowest layer and the relative weight parameter values; the quality parameter value of any upper level quality indicator of any adjacent upper level is determined according to the quality parameter value and the relative weight parameter value of the lower level quality indicator corresponding to the upper level quality indicator.

[0015] In one of the embodiments, the obtaining of the relative weight parameter values of the quality indicators of each lower layer corresponding to the target layer comprises:

[0016] For each quality indicator group in each lower layer corresponding to the target layer, the relative importance between each quality evaluation indicator in the quality indicator group is corrected based on the data associated with the quality indicator group in the observation data, to obtain a corrected relative importance.

[0017] The hierarchical analysis on each quality indicator is performed according to the corrected relative importance between each quality indicator in the same group, to obtain the relative weight parameter values of the quality indicators.

[0018] In one of the embodiments, for each quality indicator group in each lower layer corresponding to the target layer, the relative importance between each quality evaluation indicator in the quality indicator group is corrected based on the data associated with the quality indicator group in the observation data, to obtain a corrected relative importance, including:

[0019] For each quality indicator group, the key attribute value of each quality indicator in the quality indicator group is obtained based on the data associated with the quality indicator group in the observation data; the key attribute value is used to represent the key attribute of the indicator that is adapted to the business scenario of the cloud native agent.

[0020] The relative importance between each quality indicator in the quality indicator group is corrected using the key attribute value, to obtain a corrected relative importance.

[0021] In one of the embodiments, the quality evaluation result of the cloud native agent is determined based on the quality parameter value of each quality indicator, including:

[0022] The quality parameter value and the absolute weight parameter value corresponding to each quality indicator of the target layer are operated and processed to obtain a target quality value of the cloud native agent; the absolute weight parameter value corresponding to each quality indicator of the target layer is determined according to the relative weight parameter value of each quality indicator from the target layer to the uppermost layer.

[0023] The quality evaluation result of the cloud native agent is determined using the membership relationship between the target quality value of the cloud native agent and at least one preset quality range.

[0024] In a second aspect, the present application further provides a quality determination device of a cloud native agent, including:

[0025] The acquisition module is configured to acquire observation data of the cloud native agent in response to an evaluation request for the cloud native agent; the observation data is running state data and / or behavior data of executing a business of the cloud native agent.

[0026] The determination module is used to determine the quality parameter values ​​of each quality indicator corresponding to the cloud-native intelligent agent based on the observation data and the indicator evaluation model. Each quality indicator includes at least two of the following: stability evaluation indicator, security evaluation indicator, extended evaluation indicator, and business performance indicator. The stability evaluation indicator is used to characterize the fault recovery capability of the cloud-native intelligent agent in executing business operations. The security evaluation indicator is used to characterize the security protection capability of the cloud-native intelligent agent. The extended evaluation indicator is used to characterize the adaptability of the cloud-native intelligent agent when its environment changes. The business performance indicator is used to characterize the functional completion capability and performance of the cloud-native intelligent agent in executing business operations.

[0027] The quality module is used to determine the quality assessment result of the cloud-native intelligent agent based on the quality parameter values ​​of each of the aforementioned quality indicators.

[0028] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the cloud-native intelligent agent quality determination method provided in the first aspect of this application.

[0029] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cloud-native intelligent agent quality determination method provided in the first aspect of this application.

[0030] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the cloud-native intelligent agent quality determination method provided in the first aspect of this application.

[0031] The aforementioned method, apparatus, device, and medium for determining the quality of cloud-native intelligent agents, in response to an evaluation request for a cloud-native intelligent agent, acquires observation data of the cloud-native intelligent agent, determines the quality parameter values ​​of each quality indicator corresponding to the cloud-native intelligent agent based on the observation data and the indicator evaluation model, and determines the quality evaluation result of the cloud-native intelligent agent based on the quality parameter values ​​of each quality indicator. The quality evaluation indicators of this application include at least two of the following: stability evaluation indicators, security evaluation indicators, scalability evaluation indicators, and business performance indicators. This provides a more comprehensive coverage of various quality elements of cloud-native agents under unique distributed architectures and dynamic operating modes. It not only focuses on the intelligent algorithms and decision-making capabilities of cloud-native intelligent agents, such as how well business is completed and what the performance is, but also covers reliability, security, or scalability to reflect the capabilities of cloud-native agents in fault recovery, security protection, and environmental adaptability. It pays more attention to the distributed and dynamic characteristics of the cloud-native environment, and can more comprehensively and accurately determine the quality of cloud-native agents from multiple dimensions. Attached Figure Description

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained from these drawings without creative labor.

[0033] Figure 1 An application environment diagram of the quality determination method of the cloud native intelligent agent in an embodiment.

[0034] Figure 2 A flowchart of the quality determination method of the cloud native intelligent agent in an embodiment.

[0035] Figure 3 A flowchart of determining the quality parameter value in an embodiment.

[0036] Figure 4 A schematic diagram of a preset hierarchical structure in an embodiment.

[0037] Figure 5 A flowchart of determining the quality parameter value in another embodiment.

[0038] Figure 6 A flowchart of determining the quality evaluation result in an embodiment.

[0039] Figure 7 A structural block diagram of the quality determination device of the cloud native intelligent agent in an embodiment.

[0040] Figure 8 An internal structure diagram of the computer device in an embodiment. DETAILED DESCRIPTION

[0041] In order to make the purposes, technical solutions and advantages of the present application clearer, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0042] It should be noted that the terms "first", "second", and the like used in the present application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "multiple" used in the present application refers to two and more than two. The term "and / or" used in the present application refers to one of the solutions, or any combination of multiple solutions.

[0043] The quality of cloud-native agents is directly related to application effect and business stability. How to accurately and comprehensively determine the quality of cloud-native agents has become a key problem to be solved. Accurate quality determination not only helps developers to deeply understand the performance of agents, find potential defects, and optimize the design and implementation of agents, but also provides a scientific basis for users to select appropriate cloud-native agent products or services. Therefore, it is of great theoretical and practical significance to carry out research on the quality determination method of cloud-native agents.

[0044] The quality determination method of the cloud-native agent provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 The cloud-native agent 102 communicates with the quality evaluation system 104 through the network. The data storage system can store the data required to be processed by the quality evaluation system 104. The data storage system can be integrated on the quality evaluation system 104, or placed on the cloud or other network servers. The quality evaluation system 104 can be deployed on a terminal or a server, and the terminal or the server can execute the quality determination method of the cloud-native agent provided in the embodiments of the present application alone. The quality evaluation system 104 can also be deployed on the terminal and the server at the same time, and the terminal and the server can execute the quality determination method of the cloud-native agent provided in the embodiments of the present application cooperatively.

[0045] The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, unmanned aerial vehicles, low-altitude aerial vehicles, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0046] In an exemplary embodiment, as shown in Figure 2 A quality determination method of a cloud-native agent is provided. The method is applied to Figure 1 The terminal corresponding to the quality evaluation system in the method is taken as an example for illustration, and the method includes the following steps 202 to 206. Wherein:

[0047] Step 202, in response to an evaluation request for a cloud-native agent, observation data of the cloud-native agent is obtained.

[0048] The observation data of the cloud-native agent can include running state data of the cloud-native agent, and the observation data of the cloud-native agent can also include behavior data of the cloud-native agent performing a service.

[0049] For example, the observation data of the cloud-native agent can include, but is not limited to, CPU utilization, memory occupation, network bandwidth, health status of a container or a microservice, log information, error reports, fault occurrence time and frequency, recovery time, task completion, task completion time, service response time, result accuracy, and interaction logs with other services.

[0050] For example, a user can initiate an evaluation request for a certain cloud-native agent through a graphical user interface, or the evaluation request can be triggered by an automated test process, a timing task, or a specific system event. In response to the evaluation request of the cloud-native agent, the terminal triggers an observation data collection task for the cloud-native agent, and the observation data can be observation data corresponding to each quality indicator.

[0051] At step 204, based on the observation data and the indicator evaluation model, quality parameter values of each quality indicator corresponding to the cloud-native agent are determined.

[0052] The quality indicators can include at least two of a stability evaluation indicator, a security evaluation indicator, an expansion evaluation indicator, and a service performance indicator. The indicator evaluation model can include a mapping relationship between the observation data and the quality parameter values of each quality indicator, which can be a preset mapping rule, a statistical model, or a machine learning model.

[0053] The service performance indicator of the embodiment of the application can be used to represent the function completion capability and performance of the cloud-native agent performing a service, which can measure the ability of the cloud-native agent to perform a service, i.e., complete a service requirement, and measure the efficiency of the cloud-native agent performing a service, i.e., the efficiency in the service execution process.

[0054] The stability evaluation indicator of the embodiment of the application can be used to represent the fault recovery capability of the cloud-native agent in performing a service. In a dynamic and complex cloud-native environment, the agent can face various unexpected faults, such as network interruption, node crash, or service anomaly. The stability evaluation indicator aims to measure whether the cloud-native agent can maintain service continuity when facing these challenges and whether the cloud-native agent can quickly self-repair and recover to a normal running state after a fault occurs.

[0055] The security evaluation indicator of the embodiment of the application can be used to represent the security protection capability of the cloud-native agent, which can measure whether the cloud-native agent can effectively prevent unauthorized access, data leakage, and system attacks.

[0056] The expansion evaluation index of the embodiment of the present application can be used to represent the adaptability of the cloud-native intelligent agent when the environment changes. The index can measure the elasticity of the cloud-native agent when adapting to business growth and environmental changes.

[0057] Exemplarily, after obtaining the observation data of the cloud-native intelligent agent, the terminal inputs the observation data into the index evaluation model, and uses the index evaluation model to output the quality parameter values of each quality index. For example, the observation data related to stability evaluation, such as error records in system logs and recovery timestamps, can be used to calculate the quality parameter values of the stability evaluation index; the observation data related to security, such as access logs and security scanning results, can be used to calculate the quality parameter values of the security evaluation index; the observation data related to expansion, such as resource allocation records and load fluctuation conditions, can be used to calculate the quality parameter values of the expansion evaluation index; and the observation data related to business execution, such as task completion status, response time, and throughput, can be used to calculate the quality parameter values of the business performance index. Through the pre-set index evaluation model, each observation data can be quantified to obtain the objective parameter values of each dimension quality index.

[0058] Step 206, based on the quality parameter values of each quality index, determining the quality evaluation result of the cloud-native intelligent agent.

[0059] Exemplarily, the terminal can use the weighted summation method to combine the preset weights and quality parameter values of each quality index to obtain a comprehensive quality parameter value. The terminal can also use the fuzzy comprehensive evaluation or machine learning model method to perform nonlinear combination or pattern recognition on the multi-dimensional quality parameter values, thereby outputting the quality evaluation result. In addition, the quality evaluation result can also be compared with the preset performance threshold, security baseline or user expectation to determine whether the cloud-native intelligent agent meets a specific quality standard.

[0060] In the method for determining the quality of the cloud-native intelligent agent, in response to an evaluation request for the cloud-native intelligent agent, observation data of the cloud-native intelligent agent is obtained, quality parameter values of each quality index corresponding to the cloud-native intelligent agent are determined based on the observation data and an index evaluation model, and a quality evaluation result of the cloud-native intelligent agent is determined based on the quality parameter values of each quality index. The quality evaluation indexes in the embodiment of the application include at least two of the stability evaluation index, the security evaluation index, the expansion evaluation index, and the business efficiency index, which can more comprehensively cover various quality elements of the cloud-native Agents under the unique distributed architecture and dynamic operation mode. The cloud-native Agents not only focus on the intelligent algorithm and decision-making ability of the cloud-native Agents, such as how to complete the business and how the performance is, but also cover the reliability, security, or scalability to reflect the ability of the cloud-native Agents in fault recovery, security protection, and environmental adaptability, and pay more attention to the distributed and dynamic characteristics in the cloud-native environment, so that the quality of the cloud-native Agents can be more comprehensively and accurately determined from multiple dimensions.

[0061] In actual applications, in order to more accurately reflect the complex characteristics of the cloud-native Agents, the quality indexes of the cloud-native Agents can be divided into multiple levels in the embodiment of the application, and a multi-level quality index system is used to more flexibly determine the comprehensive quality of the cloud-native Agents.

[0062] In one exemplary embodiment, as shown in Figure 3 Step 204 includes steps 302 to 304. Wherein:

[0063] In step 302, quality parameter values of each quality index in the lowest level in the preset hierarchical structure are determined based on the observation data and the index evaluation model.

[0064] The preset hierarchical structure includes at least two levels, each level includes at least one group of quality indexes, and each group of quality indexes includes at least one quality index. The two adjacent levels in the at least two levels are an adjacent upper level and an adjacent lower level, and any upper level quality index in the adjacent upper level corresponds to a group of lower level quality indexes in the adjacent lower level.

[0065] The quality indexes of the cloud-native Agents are divided into multiple levels in the embodiment of the application, and any quality index in the upper level can be supported to be disassembled into a group of quality indexes in the adjacent lower level, so that the quality index in the upper level can be gradually refined into the lower level index which can be directly observed and quantified. The quality parameter values of the quality indexes in the lowest level can be directly calculated from the original observation data (such as running state data or business behavior data) and the corresponding formula model of the index evaluation model.

[0066] For example, the lowest-level quality indicators corresponding to business performance indicators may include average response time and throughput. The terminal obtains the initial quality parameter values ​​of average response time and throughput based on observation data such as request processing time and number of requests, using a preset calculation formula or model. Then, the initial quality parameter values ​​are updated using a standardization method to obtain the quality parameter values ​​of average response time and throughput.

[0067] Step 304: Based on the quality parameter values ​​corresponding to the lowest level, determine the quality parameter values ​​of each quality indicator in the target layer of the preset hierarchical structure.

[0068] The target layer can be any layer in the preset hierarchical structure.

[0069] Understandably, when the target layer is the lowest level, the terminal can directly determine the quality parameter values ​​of each quality indicator in the target layer. When the target layer is not the lowest level, for each quality indicator in the target layer, the terminal can calculate its quality parameter value based on the quality parameter values ​​of a set of lower-level quality indicators in its adjacent lower layers, combined with preset aggregation calculation rules or models, such as weighted average, fuzzy comprehensive evaluation, or machine learning models. This process can proceed layer by layer upwards, aggregating lower-level quantification results into higher-level quantification results until the desired target layer is reached.

[0070] In this embodiment, by adopting a multi-level quality index system, the quality parameter values ​​of each quality index at any level can be flexibly determined based on observation data and index evaluation models, which can reflect the complex characteristics of cloud-native agents in a more detailed and accurate manner.

[0071] The following example, using a two-tiered quality indicator system, further illustrates this application.

[0072] In one exemplary embodiment, such as Figure 4 As shown, the preset hierarchical structure of the quality indicators for cloud-native agents in this application embodiment includes two levels: an upper level and a lower level, or an adjacent upper level and an adjacent lower level. Among these, business performance indicators may include functional integrity indicators and performance indicators. Figure 4The reliability and stability can represent the stability evaluation index, the security can represent the security evaluation index, and the scalability can represent the expansion evaluation index. The upper level in the preset hierarchy includes a group of quality indexes, and each group of quality indexes includes a functional integrity index, a performance index, a stability evaluation index, a security evaluation index, and an expansion evaluation index. The lower level in the preset hierarchy includes five groups of quality indexes, the first group of quality indexes is a task completion capability index, a result accuracy rate index, and a function coverage range index corresponding to the functional integrity index; the second group of quality indexes is a response time index, a throughput index, and a resource utilization index corresponding to the performance index; the third group of quality indexes is a fault rate index, a recovery time index, and a fault tolerance capability index corresponding to the stability evaluation index; the fourth group of quality indexes is a data encryption index, an access control index, and a vulnerability detection index corresponding to the security evaluation index; and the fifth group of quality indexes is a node expansion capability index and a load balancing capability index corresponding to the expansion evaluation index.

[0073] The functional integrity index in the upper level of the embodiment of the application is the basis for the cloud native Agents to meet user needs.

[0074] Among them, the task completion capability index in the lower level is one of the key indexes for measuring functional integrity, and is closely related to the characteristics of the cloud native environment in the context of cloud native Agents.

[0075] Taking the cloud native Agents for intelligent logistics scheduling as an example, in the scene of multiple parallel tasks and dynamic resource allocation, the task completion capability index in the index evaluation model can be calculated by the following formula model:

[0076] Task completion success rate = .

[0077] Average task completion time = .

[0078] In the formula, n is the total number of tasks.

[0079] Among them, the result accuracy rate index in the lower level needs to be quantified according to the specific requirements of the task. For example, in the order allocation task, the result accuracy rate is represented by the order allocation accuracy rate, which can be calculated by the following formula model:

[0080] Order allocation accuracy rate = .

[0081] The next level of function coverage range indicator reflects the matching degree of Agent function and business demand. In the cloud native environment, business demand may change frequently, and Agent needs to have the ability to adapt quickly. Assuming that m function requirements are defined in the business demand specification, the cloud native Agent provides n functions, and k functions match the business demand. Then the function coverage range can be calculated by the following formula model:

[0082] Function coverage range = .

[0083] The performance indicator in the previous level of the embodiment of the application is crucial for the efficient operation of cloud native Agents in the cloud native environment.

[0084] The next level of response time indicator is the time interval from the reception of the request to the response given by the Agent. Due to the distributed nature of the cloud native environment, requests may flow between different nodes, making the calculation of response time more complex. Taking the distributed e-commerce customer service cloud native Agent as an example, the response time can be calculated by the following formula model:

[0085] Average response time = .

[0086] Maximum response time = .

[0087] Wherein, l is the total number of requests.

[0088] The next level of throughput indicator represents the number of requests processed by the Agent per unit time. In the cloud native environment, Agents may need to handle a large number of concurrent requests, and the stability and improvement of throughput is crucial. Assuming that Agent processed N requests in time T, then the throughput can be calculated by the following formula model:

[0089] Throughput = N / T.

[0090] The next level of resource utilization indicator focuses on the consumption of CPU, memory, network bandwidth and other resources during the running of Agent. In the cloud native platform, relevant data can be obtained with the help of monitoring tools provided by the platform. Taking CPU utilization as an example, it can be calculated by the following formula model:

[0091] CPU utilization = .

[0092] Memory occupancy rate can be calculated by the following formula model:

[0093] Memory occupancy rate = .

[0094] The stability evaluation index in the previous level of the embodiments of the present application is the key to ensure the continuous operation of cloud-native Agents.

[0095] Among them, the failure rate index of the next level is measured by counting the number of failures of the Agent within a certain period of time. In the cloud-native environment, Agents may face various types of failures, such as node failures, network interruptions, etc. Assuming that the Agent fails f times within time t, the failure rate can be calculated by the following formula model:

[0096] Failure rate = f / t.

[0097] Among them, the recovery time index of the next level refers to the time required for the Agent to recover to normal operation after failure, reflecting the self-repairing ability of the Agent. In the cloud-native environment, Agents can use container orchestration tools (such as Kubernetes) to achieve rapid recovery. The recovery time can be directly obtained by monitoring the time interval from failure to normal operation of the Agent.

[0098] Among them, the fault tolerance capability index of the next level reflects the ability of the Agent to maintain basic functions when some components or nodes fail. In the cloud-native multi-node deployment scenario, the fault tolerance capability can be evaluated by simulating different proportions of node failures. For example, after simulating x% node failure, the success rate of the Agent in completing the key task is calculated, and this is used as the fault tolerance capability, i.e. the fault tolerance capability value is the success rate of the key task after node failure.

[0099] The security index in the previous level of the embodiments of the present application relates to the information security and system stability of cloud-native Agents.

[0100] Among them, the data encryption of the next level is used to evaluate the encryption processing of the Agent during the storage and transmission of sensitive data. In the cloud-native environment, data may be stored and transmitted between multiple nodes, and the security of encryption is particularly important. The effectiveness of data encryption can be measured by evaluating the strength of the encryption algorithm and the security of key management. For example, for an Agent using the AES encryption algorithm, the encryption strength can be evaluated by checking the key length (such as AES-256 using a 256-bit key). At the same time, the security of key management can be comprehensively evaluated by the key replacement frequency f key and the key storage security score s key (the value range is 0-10, 10 represents the highest security level). The data encryption evaluation value can be calculated by the following formula model:

[0101] Data encryption value = .

[0102] where the key length score can be set according to industry standards, for example, the key length score of AES-256 is 10.

[0103] where the next level of access control indicators focuses on the management of Agent's access permissions for different users and system components.

[0104] In a cloud-native multi-tenant environment, it is crucial to ensure that each tenant can only access its authorized resources. The effectiveness of access control can be measured by calculating the unauthorized access interception rate:

[0105] Unauthorized access interception rate = .

[0106] where the next level of vulnerability detection indicators can use professional security vulnerability scanning tools to periodically scan the Agent. In a cloud-native environment, the software dependencies and network exposure of the Agent can change at any time, so frequent vulnerability detection is needed. The vulnerability detection result can be calculated by counting the number of detected vulnerabilities v and the preset vulnerability severity score s vi (for example, high-risk vulnerability score is 10, medium-risk vulnerability score is 5, and low-risk vulnerability score is 1):

[0107] Vulnerability detection evaluation value = .

[0108] The scalability indicator in the previous level of the embodiments of the present application is used to represent whether the cloud-native Agent can adapt to business growth and changes.

[0109] where the next level of node expansion capability indicators is used to represent the convenience and efficiency of adding new nodes to increase processing capacity in a cloud environment. In a cloud-native containerized deployment, node expansion involves operations such as quickly creating containers, configuring, and joining clusters. Node expansion capability can be calculated by calculating the node expansion time t expand (from the time the expansion request is initiated to the new node successfully joins the cluster and starts processing tasks) and the node expansion success rate r expand :

[0110] Node expansion capability evaluation value = .

[0111] where the next level of load balancing capability indicators measures the effectiveness and balance of the load balancing algorithm. In a cloud-native multi-node load balancing scenario, the load balancing effect can be evaluated by calculating the standard deviation of node load. Assuming there are n nodes, each node's load is L i , and the average load is . Then the formula for calculating the load balancing standard deviation is:

[0112] Load balancing standard deviation .

[0113] The smaller the standard deviation, the stronger the load balancing capability.

[0114] It can be understood that the above formula model can calculate the initial quality parameter value of each next level index. The embodiments of the present application can update the initial quality parameter value by standardizing the initial quality parameter value to obtain the quality parameter value of each next level index, so as to unify the dimensions of different indexes.

[0115] In the embodiment, by setting the quality indexes covering function, performance, reliability, security and scalability, multi-dimensional comprehensive evaluation of cloud native Agents is realized.

[0116] In some embodiments, in the process of determining the quality parameter value of each quality index corresponding to the cloud native intelligent agent, and / or in the process of determining the quality parameter value of each quality index corresponding to the cloud native intelligent agent, the terminal can combine the weight parameter of each quality index. The weight parameter of each quality index of the embodiments of the present application can be determined by using the analytic hierarchy process.

[0117] In one of the embodiments, the weight parameter of each quality index can include a relative weight parameter. For each quality index, the relative weight parameter value of the quality index can be obtained by performing analytic hierarchy processing on each quality index in the same group according to the relative importance between the quality indexes.

[0118] For example, for each group of quality indexes, the terminal can obtain the relative importance between each quality index in the group set in advance, and construct a judgment matrix based on the relative importance. For example, the element at position (i, j) in the judgment matrix represents the relative importance of the i-th evaluation index in the same group compared with the j-th evaluation index. The terminal calculates the maximum eigenvalue according to the judgment matrix, and judges whether the judgment matrix satisfies the consistency test based on the maximum eigenvalue. If it satisfies, the characteristic vector corresponding to the maximum eigenvalue is obtained, and the relative weight parameter value corresponding to each quality index in the group is determined based on the characteristic vector.

[0119] In order to fully consider the dynamic characteristics of the cloud native environment, and combined with the multi-source data feedback in the running process of Agents, dynamic weight adjustment can be realized by combining the observation data of cloud native Agents, so as to more accurately determine the weight of each index.

[0120] In one of the embodiments, for each quality indicator, the relative weight parameter value of the quality indicator can also be based on the observation data of the respective quality indicators in the same group where the quality indicator is located, to correct the relative importance between the respective quality indicators, and then based on the corrected relative importance between the respective quality indicators, to obtain the relative weight parameter value of the respective quality indicators through analytic hierarchy process.

[0121] For example, for each group of quality indicators, the terminal can obtain the data associated with the group of quality indicators in the observation data and the relative importance between the respective quality indicators in the group, correct the relative importance through the data associated with the group of quality indicators to obtain the corrected relative importance, and based on the corrected relative importance between the respective quality indicators in the group, perform analytic hierarchy process on the respective quality indicators in the group to obtain the relative weight parameter value of the respective quality indicators.

[0122] In a possible implementation, for each group of quality indicators, the terminal can obtain the key attribute value of each quality indicator in the group of quality indicators based on the data associated with the group of quality indicators in the observation data, where the key attribute value is used to represent the key attribute of the indicator that is suitable for the business scenario of the cloud native agent. The terminal can correct the relative importance between the respective quality evaluation indicators in the quality evaluation indicator group by using the key attribute value to obtain the corrected relative importance.

[0123] For example, the previous level in the preset hierarchy includes a group of quality indicators, and the group of quality indicators includes a functional integrity indicator, a performance indicator, a stability evaluation indicator, a security evaluation indicator, and an expansion evaluation indicator. When determining the relative weight parameter value of the group of quality indicators, in the case that the business scenario of the cloud native agent to be determined is a high concurrency business scenario, the terminal selects the indicator fluctuation degree as the key attribute of the indicator suitable for the business scenario, and according to the observation data associated with the group of quality indicators, obtains the indicator fluctuation degree corresponding to each quality indicator in the group of quality indicators, and corrects the pre-set relative importance by using the indicator fluctuation degree to obtain the corrected relative importance.

[0124] For example, the terminal can determine the correction factor of the quality indicator through the indicator fluctuation degree of the quality indicator, the correction factor is proportional to the indicator fluctuation degree, and the pre-set relative importance is corrected through the correction factor corresponding to each quality indicator. For example, the correction factor of the functional integrity indicator is divided by the correction factor of the performance indicator, and then multiplied by the relative importance of the functional integrity indicator with respect to the performance indicator to obtain the corrected relative importance.

[0125] For example, the next level of the preset hierarchy includes a set of functional integrity indicators corresponding to quality indicators, respectively, task completion capability indicator, result accuracy indicator and functional coverage indicator. When the terminal determines the relative weight parameter value of the set of quality indicators, in the case that the cloud native intelligent agent to be determined quality belongs to the business function expansion business scenario, the terminal selects the sensitivity of business demand change as the key attribute of the index adapted to the business scenario, and obtains the sensitivity of business demand change corresponding to each quality indicator in the set of quality indicators according to the observation data associated with the set of quality indicators (for example, the ratio of the number of new demand function points to the number of original function points), and modifies the pre-set relative importance degree through the sensitivity of business demand change to obtain the modified relative importance degree.

[0126] In one of the embodiments, the embodiments of the present application can also use the reflection triggering mechanism to modify the relative importance between the quality indicators, and determine the relative weight parameter value of each quality indicator based on the modified relative importance between the quality indicators.

[0127] For example, the terminal updates the judgment matrix, obtains the potential deviation in the current criterion weight configuration, and performs reflection-driven modification when the consistency ratio (CR) of the judgment matrix obviously rises or the repeated pattern of decision failure appears (such as continuous multiple task failures and concentration in a certain criterion).

[0128] For each set of quality indicators, the terminal can obtain the pre-set initial relative importance between the quality indicators in the set, and the terminal constructs an initial judgment matrix according to the initial relative importance between the quality indicators in the set. If the consistency ratio (CR) of the initial judgment matrix is greater than a preset consistency threshold, the terminal obtains the historical deviation degree of each quality indicator in the set, which is the imbalance frequency or deviation degree of each quality indicator in the historical task execution. The historical deviation degree of the quality indicator is proportional to the number of times it is underestimated in the historical task execution. The terminal determines the dynamic correction factor between the quality indicators according to the historical deviation degree of each quality indicator, and modifies the initial relative importance between the quality indicators using the dynamic correction factor to obtain the modified relative importance.

[0129] For example, the next level of the preset hierarchy, such as the criterion indicator layer, includes a set of quality indicators, respectively, functional integrity F, performance P, reliability and stability R, security S, and scalability E, and the number of quality indicators m=5. When first running or historical data is incomplete, expert questionnaire or Delphi method can be used for paired scoring to determine the initial relative importance between any two quality indicators in the criterion indicator layer, and then obtain the initial judgment matrix, which can be represented as , aij represents the initial relative importance of the i-th quality indicator with respect to the j-th quality indicator. If the consistency ratio (CR) of the initial judgment matrix is greater than a preset consistency threshold or the decision failure appears a repeated pattern, the terminal updates the initial relative importance to update the initial judgment matrix.

[0130] To more reasonably update the relative importance, first introduce a reflection memory vector , which is used to record the imbalance frequency or deviation degree of each quality indicator in the historical task execution. When the quality indicator i is frequently underestimated (less than the average value) in the past multiple rounds of evaluation, its f i will be gradually accumulated, representing the severity of its neglect. Based on the memory vector, the historical deviation degree difference between two quality indicators is defined as:

[0131] .

[0132] wherein, is the historical deviation degree difference between the i-th quality indicator and the j-th quality indicator.

[0133] Then introduce a dynamic correction factor (dynamic adjustment coefficient):

[0134] .

[0135] wherein, is the learning rate, used to control the adjustment amplitude, and the learning rate is inversely proportional to CR. For example, , η0 is a preset initial learning rate or maximum learning rate, when CR approaches 0.1 (upper limit), it means that the matrix tends to be inconsistent, and η should be reduced for conservative adjustment, otherwise it can be improved, so that the correction direction of the judgment matrix is consistent with the historical feedback.

[0136] The present application can replace the traditional linear correction model, and adopt an exponential weighted nonlinear update function:

[0137] .

[0138] wherein, is the relative importance of the i-th quality indicator with respect to the j-th quality indicator after correction.

[0139] The terminal can use the corrected judgment matrix to calculate the relative weight parameter value of each quality indicator in the criterion indicator layer. By solving the linear equation group , the characteristic vector of the corrected judgment matrix is calculated, and the power method is used to iteratively solve the maximum eigenvalue and the corresponding characteristic vector of the matrix:

[0140] .

[0141] wherein, is an identity matrix, is the estimated value of the eigenvector at the kth iteration, and the initial value may be randomly selected as a normalized vector. When ( is a preset precision, such as ), the iteration is stopped, and the weight vector is obtained.

[0142] Then, the terminal calculates the consistency index , and according to the value of , obtains the random consistency index from a related table. Then, the consistency ratio is calculated. If , it is judged that the judgment matrix has acceptable consistency; otherwise, the judgment matrix needs to be adjusted.

[0143] For a specific index layer, taking the task completion capability under the functional integrity F in the criterion index layer, the order allocation accuracy , and the functional coverage range as examples, the judgment matrix ( is the number of lower-level indexes corresponding to the quality index, and here ), and the values of b ij are determined by using the 1-9 scale method.

[0144] Considering the dynamic evolution of cloud-native business, the importance of indexes will change with business needs. For example, the expansion of business functions may increase the weight of coverage range, and the increase in the frequency of sudden failures may highlight the importance of task completion capability. Triggering conditions include but are not limited to: when the business needs have a structural change (such as version update, function range expansion, call path change, etc.), or when key indicators such as task completion rate and delay trigger alarm thresholds, that is, the index layer reflection mechanism is triggered.

[0145] According to the changes in business needs in the cloud-native environment:

[0146] .

[0147] .

[0148] wherein, is the difference value of the historical deviation degree between the quality index pairs, is the learning rate.

[0149] Similar to the indicator criteria layer, a revised judgment matrix is ​​constructed based on the revised relative importance. Its eigenvectors are solved iteratively using the power method. This means obtaining the relative weight parameters of the specific indicator layer:

[0150] .

[0151] After obtaining the relative weight vectors of each quality indicator at the specific indicator layer... Then, combine the weight vector of the criterion indicator layer. Calculate the absolute weight parameters of each quality indicator in the specific indicator layer relative to the target layer. This is based on the task completion capability under functional integrity. For example, its absolute weight The calculation is as follows:

[0152] .

[0153] in, The weight of functional integrity quality indicators in the criterion indicator layer (from...) ), It is the weight of task completion capability under functional integrity at the specific indicator level (from...) ).

[0154] In one embodiment, the weight parameters of each quality indicator may also include absolute weight parameters, which can be determined based on the relative weight parameters of the quality indicators at the level to which each quality indicator is located and the quality indicators above it.

[0155] The relative weight parameter value of the top-level quality indicator is the absolute weight parameter value. The absolute weight parameter value of the lower-level quality indicator is obtained by multiplying the absolute weight parameter value of the upper-level quality indicator with the relative weight parameter value of the lower-level quality indicator.

[0156] For example, the failure rate, recovery time, and fault tolerance indicators corresponding to the stability assessment indicators are determined by the absolute weight parameter value of the failure rate indicator, which is based on the absolute weight parameter value of the stability assessment indicator and the relative weight parameter value of the failure rate indicator. Similarly, the absolute weight parameter value of the recovery time indicator and the absolute weight parameter value of the fault tolerance indicator are also determined by these factors.

[0157] In one exemplary embodiment, such as Figure 5 As shown, step 304 includes steps 502 to 504. Wherein:

[0158] Step 502, in the case where the target layer is not the lowest layer, obtaining the relative weight parameter value of the quality index of each lower layer corresponding to the target layer.

[0159] Wherein, each lower layer corresponding to the target layer does not include the target layer, for example, in the case where the preset layer hierarchy is three layers, each lower layer of the uppermost layer is the adjacent lower layer of the uppermost layer and the lowermost layer.

[0160] Wherein, the relative weight parameter of the quality index is obtained by performing analytic hierarchy processing on each quality index according to the relative importance between each quality index in the same group of quality indexes.

[0161] In one possible implementation, for each quality index group in each lower layer corresponding to the target layer, the relative importance between each quality evaluation index in the quality index group is corrected based on the data associated with the quality index group in the observation data, to obtain the corrected relative importance; and the analytic hierarchy processing is performed on each quality index based on the corrected relative importance between each quality index in the same group, to obtain the relative weight parameter value of each quality index.

[0162] The determination of the relative weight parameter value in the embodiment of the application can refer to the determination method of the relative weight parameter value in the above-mentioned embodiments.

[0163] Step 504, determining the quality parameter value of each quality index of the target layer based on the quality parameter value and the relative weight parameter value of the lowest layer.

[0164] Wherein, the quality parameter value of any upper layer quality index of any adjacent upper layer is determined according to the quality parameter value and the relative weight parameter value of the lower layer quality index corresponding to the upper layer quality index.

[0165] For example, in a group of quality indexes in the adjacent lower layer, the quality parameter value of the adjacent upper layer functional integrity index can be obtained by using the sum of the product of the quality parameter value of the task completion capability index and the relative weight parameter, the product of the quality parameter value of the result accuracy rate index and the relative weight parameter, and the product of the quality parameter value of the functional coverage range index and the relative weight parameter.

[0166] In one exemplary embodiment, as shown in Figure 6 Step 206 includes step 602 and step 604, wherein:

[0167] Step 602, performing operation processing on the quality parameter value and the absolute weight parameter value of each quality index corresponding to the target layer to obtain the target quality value of the cloud native intelligent agent.

[0168] The absolute weight parameter value corresponding to each quality indicator of the target layer is determined according to the relative weight parameter values of each quality indicator from the target layer to the uppermost layer.

[0169] The determination of the absolute weight parameter value in the embodiment of the application can refer to the determination method of the absolute weight parameter value in the above-mentioned embodiment.

[0170] For example, the target layer is the uppermost layer, and the terminal performs summation operation on the product of the quality parameter value and the absolute weight parameter value of the functional integrity indicator, the product of the quality parameter value and the absolute weight parameter value of the performance indicator, the product of the quality parameter value and the absolute weight parameter value of the stability evaluation indicator, the product of the quality parameter value and the absolute weight parameter value of the security evaluation indicator, and the product of the quality parameter value and the absolute weight parameter value of the expansion evaluation indicator in the uppermost layer, to obtain the target quality value of the cloud native agent.

[0171] In step 604, the target quality value of the cloud native agent is used to determine the quality evaluation result of the cloud native agent according to the membership relation with at least one preset quality range.

[0172] For example, the preset quality range includes a preset quality threshold, and in the case that the target quality value of the cloud native agent is greater than the preset quality threshold, the cloud native agent is determined to be of high quality. In the case that the target quality value of the cloud native agent is less than or equal to the preset quality threshold, the cloud native agent is determined to be of low quality, wherein the high quality is superior to the low quality.

[0173] In a possible implementation, the target quality value can include a comprehensive quality value, a key quality value and a basic quality value, and each quality indicator can include a key quality indicator and a basic quality indicator. For example, the terminal can determine the key quality indicator in each quality indicator according to the business scenario and the life cycle of the cloud native agent. For example, in the initial stage of product online, the key quality indicator can include the functional integrity indicator, the performance indicator and each lower indicator belonging thereto, and the basic quality indicator can include the stability evaluation indicator, the security evaluation indicator, the expansion evaluation indicator and each lower indicator belonging thereto.

[0174] The terminal performs operation processing on the quality parameter value and the absolute weight parameter value corresponding to each quality indicator of the target layer to obtain the comprehensive quality value of the cloud native agent; the terminal performs operation processing on the quality parameter value and the absolute weight parameter value corresponding to each key quality indicator of the target layer to obtain the key quality value of the cloud native agent; and the terminal performs operation processing on the quality parameter value and the absolute weight parameter value corresponding to each basic quality indicator of the target layer to obtain the basic quality value of the cloud native agent.

[0175] In a case where the comprehensive quality value of the cloud-native agent is greater than or equal to a first preset quality threshold, the cloud-native agent is determined to be high quality; in a case where the comprehensive quality value of the cloud-native agent is less than or equal to a second preset quality threshold, the cloud-native agent is determined to be low quality; in a case where the comprehensive quality value of the cloud-native agent is less than the first preset threshold and greater than the second preset threshold, and the key quality value is greater than or equal to a preset key threshold, the cloud-native agent is determined to be medium quality; in a case where the comprehensive quality value of the cloud-native agent is less than the first preset threshold and greater than the second preset threshold, and the key quality value is less than the preset key threshold, the cloud-native agent is determined to be low quality. The high quality is better than the medium quality, and the medium quality is better than the low quality.

[0176] In the present implementation, the performance of the cloud-native agent in the business scenario and the ability that the user is most concerned about can be determined more accurately and flexibly, and the good performance of the secondary indicators can be avoided.

[0177] In an exemplary embodiment, a method for determining the quality of a cloud-native agent is provided, and the method comprises:

[0178] Step A1, obtaining a preset hierarchical structure of quality indicators of the cloud-native agent.

[0179] In the preset hierarchical structure, the upper level includes a group of quality indicators, and each group of quality indicators includes a functional integrity indicator, a performance indicator, a stability evaluation indicator, a security evaluation indicator, and an expansion evaluation indicator. The lower level includes five groups of quality indicators. The first group of quality indicators is a task completion capability indicator, a result accuracy indicator, and a functional coverage range indicator corresponding to the functional integrity indicator. The second group of quality indicators is a response time indicator, a throughput indicator, and a resource utilization indicator corresponding to the performance indicator. The third group of quality indicators is a fault rate indicator, a recovery time indicator, and a fault tolerance capability indicator corresponding to the stability evaluation indicator. The fourth group of quality indicators is a data encryption indicator, an access control indicator, and a vulnerability detection indicator corresponding to the security evaluation indicator. The fifth group of quality indicators is a node expansion capability indicator and a load balancing capability indicator corresponding to the expansion evaluation indicator.

[0180] Step A2, determining the quality parameter value of each quality indicator corresponding to the lowest level based on the observation data and the indicator evaluation model.

[0181] Step A3, obtaining the relative weight parameter value of each quality indicator by using an improved analytic hierarchy process.

[0182] In the quality determination of the cloud-native agent, in order to more accurately determine the weight of each indicator, the improved AHP algorithm is used in the embodiments of the present application, the dynamic characteristics of the cloud-native environment are fully considered, and the dynamic weight adjustment is realized by combining the multi-source data feedback in the running process of the agent.

[0183] Firstly, the terminal constructs a hierarchical model. The hierarchy of cloud-native Agent quality determination is divided into the overall goal (cloud-native Agent quality evaluation Z), the criterion index layer (function integrity F, performance P, reliability and stability R, security S, and scalability E), and the specific index layer (specific indexes under each criterion layer, such as task completion capability F1 and function coverage range F2 under function integrity).

[0184] The terminal constructs a judgment matrix according to the relative importance between each index. On the basis of constructing a judgment matrix in the traditional AHP, the elements of the judgment matrix are modified considering the dynamic nature of the cloud-native environment. Taking the judgment matrix of the criterion layer as an example (m is the number of criterion layer indexes, and here m = 5) a ij represents the ratio of the importance of the i-th criterion (such as function integrity) to the overall goal and the importance of the j-th criterion (such as performance) to the overall goal layer, which is assigned using the 1-9 scale method. However, considering the dynamic nature of the cloud-native environment, a ij is modified as follows:

[0185] .

[0186] where the left side of the equation is the modified judgment matrix element, the right side of the equation is the modified judgment matrix element, and the right side of the equation is a dynamic adjustment coefficient determined according to the fluctuation of the indexes related to each criterion in the cloud-native environment (for example, in a high-concurrency business scenario, the performance-related index fluctuates greatly, and the corresponding is larger), represents the evaluation time interval.

[0187] The terminal calculates the characteristic vector and the characteristic value by solving the linear equation group to calculate the characteristic vector of the modified judgment matrix , where is the modified judgment matrix, is the maximum characteristic value, is the unit matrix. In order to calculate more accurately, the power method iterative formula is used:

[0188] .

[0189] where, is the estimated value of the characteristic vector at the n th iteration, and the initial value can be randomly selected as a normalized vector. When ​For a preset precision, such as ), the iteration stops, and the eigenvector at this time is , which is the required eigenvector .

[0190] The terminal performs consistency checking, calculates the consistency index , and according to the value of , obtains the random consistency index from the relevant table. Then the consistency ratio is calculated. If , it is judged that the matrix has acceptable consistency; otherwise, the judgment matrix needs to be adjusted.

[0191] The terminal calculates the specific index layer weight. For the specific index layer, taking the task completion ability F1 and the function coverage F2 under the functional integrity criterion as an example, a judgment matrix is constructed (n is the number of indexes under the criterion, here ). The value of b ij is determined by the 1-9 scale method, and is modified according to the change of business demand in the cloud native environment:

[0192] .

[0193] Wherein, is the modified element of the index layer judgment matrix, is the adjustment coefficient based on the influence of the change of business demand on the relative importance of the index (for example, when the business function is expanded, the importance of the function coverage increases, and the changes accordingly), represents the quantitative value of the degree of change of business demand (which can be measured by the ratio of the number of new demand function points to the number of original function points).

[0194] Similar to the criterion index layer, the power method is used to iteratively calculate the eigenvector of the modified judgment matrix of the specific index layer, and the relative weight vector of the specific index under each criterion index layer is obtained. Then, combined with the weight vector of the criterion layer, the combined weight of each index of the specific index layer relative to the total target is calculated. Taking the task completion ability under the functional integrity as an example, its combined weight is calculated as follows:

[0195] .

[0196] Wherein, is the weight of the functional integrity criterion in the criterion layer (from ), is the weight of the task completion ability under the functional integrity in the index layer (from v).

[0197] The terminal can employ a feedback adjustment mechanism to ensure the rationality and dynamic adaptability of the weights. This mechanism is introduced based on quality data monitoring results during the actual operation of cloud-native agents (such as task success rate in the functional integrity metric). Average response time in performance-related metrics ), and the preset threshold (task success rate threshold) Average response time threshold The deviation rate was calculated by comparing the results with those of other parameters (e.g., [etc.]).

[0198] .

[0199] .

[0200] The weights are dynamically adjusted based on the deviation rate. If the task success rate is below a threshold, the weights of functional integrity-related indicators are increased. The adjustment formula is as follows:

[0201] .

[0202] in, It is the adjusted weight of the task completion capability indicator. This is the weight adjustment range coefficient, which can be set according to the actual situation, for example ( =0.1).

[0203] Through the above series of formulas and methods, the relative weight parameter values ​​of quality indicators can be dynamically determined in combination with the characteristics of cloud-native environment. This can more accurately reflect the importance of each indicator in the determination of cloud-native agent quality in different scenarios.

[0204] Step A4: Based on the quality parameter values ​​of each quality indicator corresponding to the lowest layer and the relative weight parameter values ​​of each quality indicator, the quality assessment value of the cloud-native agent of the cloud-native agent is obtained by weighted calculation.

[0205] Assuming that in the cloud-native agent quality evaluation index system, the weights of the criteria layer indicators F (functional integrity), P (performance), R (reliability and stability), S (security), and E (scalability) are ω respectively F ω P ω R ω S and ω E The weights of the indicators at each criterion level are as follows: the weight of task completion capability F1 under functional integrity is ω. F1 The weight of the functional coverage range F2 is ω. F2 The weight of response time P1 under performance conditions is ω. P1, the weight of throughput P2 is ω P2 , the weight of resource utilization P3 is ω P3 ; the weight of failure rate R1 under reliability and stability is ω R1 , the weight of recovery time R2 is ω R2 , the weight of fault tolerance R3 is ω R3 ; the weight of data encryption S1 under security is ω S1 , the weight of access control S2 is ω S2 , the weight of vulnerability detection S3 is ω S3 ; the weight of node expansion capability E1 under scalability is ω E1 , the weight of load balancing capability E2 is ω E2 .

[0206] The quality parameter values of each index after standardization are as follows: the quality parameter value of task completion capability under functional integrity is Score F1 , the quality parameter value of functional coverage is Score F2 ; the quality parameter value of response time under performance is Score P1 , the quality parameter value of throughput is Score P2 , the quality parameter value of resource utilization is Score P3 ; the quality parameter value of failure rate under reliability and stability is Score R1 , the quality parameter value of recovery time is Score R2 , the quality parameter value of fault tolerance is Score R3 ; the quality parameter value of data encryption under security is Score S1 , the quality parameter value of access control is Score S2 , the quality parameter value of vulnerability detection is Score S3 ; the quality parameter value of node expansion capability under scalability is Score E1 , the quality parameter value of load balancing capability is Score E2 .

[0207] The calculation formula of the cloud native Agent quality evaluation value TotalScore is as follows:

[0208] .

[0209] Through this quality determination algorithm, the performance of cloud native Agents in various aspects and the weights of the corresponding indexes are comprehensively considered, and a quality evaluation value reflecting the quality level can be obtained. The quality evaluation value can be used for quality comparison between different cloud native Agents, and also helps to intuitively understand the quality status of the Agent, providing a quantitative basis for optimizing and selecting cloud native Agents.

[0210] In this embodiment, cloud-native Agents play a key role in complex and changing digital scenarios as a technology that combines cloud-native technology and Agent architecture. The embodiments of the application focus on the quality evaluation of cloud-native Agents and construct a comprehensive, scientific and practical evaluation system and method. Through in-depth analysis of the current research status, an evaluation index system covering function, performance, reliability, security and scalability is determined, and the analytic hierarchy process (AHP) and other methods are used to determine the index weight. At the same time, the evaluation method of the total score is proposed, providing a multi-dimensional solution for the quality evaluation of cloud-native Agents. The application effectively supports the quality improvement of cloud-native Agents and promotes their wide application.

[0211] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the application.

[0212] Based on the same inventive concept, the embodiments of the application also provide a cloud-native agent quality determination apparatus for implementing the cloud-native agent quality determination method described above. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, so the specific limitations in one or more cloud-native agent quality determination apparatus embodiments provided below can refer to the limitations of the cloud-native agent quality determination method in the above text, which will not be repeated here.

[0213] In one exemplary embodiment, as shown in Figure 7 A cloud-native agent quality determination apparatus is provided, comprising: an acquisition module 702, a determination module 704, and a quality module 706, wherein:

[0214] The acquisition module 702 is configured to acquire observation data of the cloud-native agent in response to an evaluation request for the cloud-native agent; the observation data is running state data and / or behavior data of executing a service of the cloud-native agent.

[0215] The determining module 704 is configured to determine quality parameter values of each quality index corresponding to the cloud native agent based on the observation data and the index evaluation model. The quality index includes at least two of a stability evaluation index, a security evaluation index, an expansion evaluation index, and a business performance index. The stability evaluation index is used to represent the fault recovery capability of the cloud native agent in performing a business. The security evaluation index is used to represent the security protection capability of the cloud native agent. The expansion evaluation index is used to represent the adaptability of the cloud native agent when the environment changes. The business performance index is used to represent the function completion capability and performance of the cloud native agent in performing a business.

[0216] The quality module 706 is configured to determine a quality evaluation result of the cloud native agent based on the quality parameter values of each quality index.

[0217] The modules in the cloud native agent quality determination apparatus can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be called and executed by a processor to perform operations corresponding to the modules.

[0218] In an exemplary embodiment, a computer device, which can be a server, is provided. An internal structure diagram of the computer device can be as shown in Figure 8 The computer device includes a processor, a memory, an input / output interface, and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a cloud native agent quality determination method.

[0219] Those skilled in the art can understand that Figure 8 The structure shown in the above

[0220] In an example embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method for determining the quality of a cloud-native intelligent agent provided by the embodiments of the application when executing the computer program.

[0221] In an example embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the steps of the method for determining the quality of a cloud-native intelligent agent provided by the embodiments of the application when executed by a processor.

[0222] In an example embodiment, a computer program product is provided, comprising a computer program, and the computer program implementing the steps of the method for determining the quality of a cloud-native intelligent agent provided by the embodiments of the application when executed by a processor.

[0223] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0224] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0225] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0226] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for determining the quality of cloud-native intelligent agents, characterized in that, The method includes: In response to an evaluation request for a cloud-native intelligent agent, the observation data of the cloud-native intelligent agent is obtained, wherein the observation data is the operational status data and / or the behavioral data of the cloud-native intelligent agent performing business. Based on the observation data and indicator evaluation model, the quality parameter values ​​of each quality indicator corresponding to the cloud-native intelligent agent are determined. Each quality indicator includes at least two of the following: stability evaluation indicator, security evaluation indicator, extended evaluation indicator, and business performance indicator. The stability evaluation indicator is used to characterize the fault recovery capability of the cloud-native intelligent agent in executing business operations. The security evaluation indicator is used to characterize the security protection capability of the cloud-native intelligent agent. The extended evaluation indicator is used to characterize the adaptability of the cloud-native intelligent agent when its environment changes. The business performance indicator is used to characterize the functional completion capability and performance of the cloud-native intelligent agent in executing business operations. Based on the quality parameter values ​​of each of the aforementioned quality indicators, the quality assessment result of the cloud-native intelligent agent is determined; The determination of quality parameter values ​​for each quality indicator corresponding to the cloud-native intelligent agent based on the observation data and indicator evaluation model includes: Based on the observation data and the indicator evaluation model, the quality parameter values ​​of each quality indicator in the lowest level of the preset hierarchical structure of the quality indicators are determined; the preset hierarchical structure includes at least two levels, each level includes at least one set of quality indicators, and each set of quality indicators includes at least one quality indicator; the two adjacent levels in the at least two levels are respectively the adjacent upper level and the adjacent lower level, and any upper level quality indicator of the adjacent upper level corresponds to a set of lower level quality indicators of the adjacent lower level; Based on the quality parameter value corresponding to the lowest layer, determine the quality parameter value of each quality index of the target layer in the preset hierarchical structure; the target layer is any layer in the preset hierarchical structure.

2. The method according to claim 1, characterized in that, The observation data of the cloud-native intelligent agent includes the memory usage, network bandwidth, health status of containers or microservices, log information, error reports, failure occurrence time and frequency, recovery time, task completion status, task completion time, business response time, result accuracy, and interaction logs with other services.

3. The method according to claim 1, characterized in that, The step of determining the quality parameter values ​​of each quality indicator in the target layer of the preset hierarchical structure based on the quality parameter values ​​corresponding to the lowest layer includes: If the target layer is not the lowest layer, obtain the relative weight parameter values ​​of the quality indicators of each lower layer corresponding to the target layer; the relative weight parameters of the quality indicators are obtained by performing hierarchical analysis on each quality indicator based on the relative importance between each quality indicator in the same group. Based on the quality parameter value corresponding to the lowest layer and the relative weight parameter value, the quality parameter value of each quality indicator of the target layer is determined; wherein, the quality parameter value of any upper-layer quality indicator of any adjacent upper layer is determined according to the quality parameter value and relative weight parameter value of the lower-layer quality indicator corresponding to the upper-layer quality indicator.

4. The method according to claim 3, characterized in that, The step of obtaining the relative weight parameter values ​​of the quality indicators of each lower layer corresponding to the target layer includes: For each quality indicator group in each lower layer corresponding to the target layer, the relative importance between each quality assessment indicator in the quality indicator group is corrected based on the data associated with the quality indicator group in the observation data, so as to obtain the corrected relative importance. Based on the corrected relative importance among the quality indicators within the same group, hierarchical analysis is performed on each quality indicator to obtain the relative weight parameter values ​​of each quality indicator.

5. The method according to claim 4, characterized in that, For each quality indicator group in each lower layer corresponding to the target layer, based on the data associated with the quality indicator group in the observation data, the relative importance between each quality assessment indicator in the quality indicator group is corrected to obtain the corrected relative importance, including: For each quality indicator group, based on the data associated with the quality indicator group in the observation data, the key attribute values ​​of each quality indicator in the quality indicator group are obtained; the key attribute values ​​are used to characterize the key attributes of the indicators that are adapted to the business scenarios of the cloud-native intelligent agent. The relative importance among the quality indicators within the quality indicator group is corrected using the key attribute values ​​to obtain the corrected relative importance.

6. The method according to any one of claims 2 to 5, characterized in that, The process of determining the quality assessment result of the cloud-native intelligent agent based on the quality parameter values ​​of each of the aforementioned quality indicators includes: The quality parameter values ​​and absolute weight parameter values ​​corresponding to each quality indicator of the target layer are processed to obtain the target quality value of the cloud-native intelligent agent; the absolute weight parameter values ​​corresponding to each quality indicator of the target layer are determined based on the relative weight parameter values ​​of each quality indicator from the target layer to the top layer. The quality assessment result of the cloud-native intelligent agent is determined by utilizing the membership relationship between the target quality value of the cloud-native intelligent agent and at least one preset quality range.

7. A device for determining the quality of cloud-native intelligent agents, characterized in that, The device includes: The acquisition module is used to acquire observation data of the cloud-native intelligent agent in response to an evaluation request for the cloud-native intelligent agent; the observation data is the operational status data and / or behavioral data of the cloud-native intelligent agent performing business. The determination module is used to determine the quality parameter values ​​of each quality indicator corresponding to the cloud-native intelligent agent based on the observation data and the indicator evaluation model. Each quality indicator includes at least two of the following: stability evaluation indicator, security evaluation indicator, extended evaluation indicator, and business performance indicator. The stability evaluation indicator is used to characterize the fault recovery capability of the cloud-native intelligent agent in executing business. The security evaluation indicator is used to characterize the security protection capability of the cloud-native intelligent agent. The extended evaluation indicator is used to characterize the adaptability of the cloud-native intelligent agent when the environment changes. The business performance indicator is used to characterize the functional completion capability and performance of the cloud-native intelligent agent in executing business. The quality module is used to determine the quality assessment result of the cloud-native intelligent agent based on the quality parameter values ​​of each of the aforementioned quality indicators. The determination of quality parameter values ​​for each quality indicator corresponding to the cloud-native intelligent agent based on the observation data and indicator evaluation model includes: Based on the observation data and the indicator evaluation model, the quality parameter values ​​of each quality indicator in the lowest level of the preset hierarchical structure of the quality indicators are determined; the preset hierarchical structure includes at least two levels, each level includes at least one set of quality indicators, and each set of quality indicators includes at least one quality indicator; the two adjacent levels in the at least two levels are respectively the adjacent upper level and the adjacent lower level, and any upper level quality indicator of the adjacent upper level corresponds to a set of lower level quality indicators of the adjacent lower level; Based on the quality parameter value corresponding to the lowest layer, determine the quality parameter value of each quality index of the target layer in the preset hierarchical structure; the target layer is any layer in the preset hierarchical structure.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images