Cloud resource pool performance evaluation method, device, equipment, medium and product

Abstract

The application provides a cloud resource pool performance evaluation method, device, equipment, medium and product, wherein the method comprises: calculating the priority of each cloud resource pool based on the index data of a plurality of normal performance indexes of each cloud resource pool; acquiring fault index data of a plurality of fault performance indexes collected when performing fault performance testing on a target cloud resource pool; inferring each fault index data to obtain the weight of a plurality of fault index features included in each fault performance index; selecting a plurality of target fault index features with the highest importance from each fault performance index based on the weight; and performing fault performance evaluation on the remaining cloud resource pool based on the index data of the target fault index features of the remaining cloud resource pool. The application tests the remaining cloud resource pool by extracting the target fault index features, simplifies the test content, reduces the labor cost, and improves the test efficiency.

Description

Technical Field

[0001] This application belongs to the field of cloud computing technology, specifically relating to a method, apparatus, equipment, medium, and product for evaluating the performance of cloud resource pools. Background Technology

[0002] With the development of technology and the improvement of digitalization, both enterprises and individuals have an ever-increasing demand for information resources. In order to resolve the contradiction between people's growing demand for information services and the cost of building private physical servers, the concept of cloud computing has emerged. Cloud services are provided by service providers who build physical equipment such as servers and offer computing or storage services to customers through the network, without requiring customers to build their own physical server rooms. The servers built to provide cloud services are often clustered in different locations in reality, and are also known as cloud resource pools.

[0003] Due to differences in physical locations and included hardware, the performance of different cloud resource pools varies, primarily in terms of energy consumption, task processing speed, network load capacity, and storage load capacity. These performance differences affect the efficiency and stability of cloud services. Therefore, to meet customer needs in different scenarios and improve user experience, it is essential to conduct a reasonable performance evaluation of cloud resource pools. Reliable and comprehensive evaluation results can also help cloud service providers optimize the performance and control costs of cloud resource pools, as well as uncover hidden vulnerabilities and enhance security.

[0004] When there are multiple cloud resource pools, the existing technology usually performs comprehensive testing on each cloud resource pool without discrimination. This comprehensive testing approach has the disadvantages of complicated testing content, low testing efficiency, and high cost. Summary of the Invention

[0005] The embodiments of this application propose a cloud resource pool performance evaluation method, apparatus, device, medium, and product. By acquiring fault indicator data of the fault performance indicators of the target cloud resource pool, multiple target fault indicator features with the highest importance are selected from the fault performance indicators. Based on the target fault indicator features, the fault performance of the remaining cloud resource pool is tested, which reduces the testing content of the remaining cloud resource pool, improves testing efficiency, and helps to save costs and resources.

[0006] The first aspect of this application proposes a method for evaluating the performance of a cloud resource pool, including:

[0007] Based on the acquired data of multiple normal performance indicators for each cloud resource pool, the priority of each cloud resource pool is calculated, and the priority is used to characterize the degree of importance that users attach to the cloud resource pool.

[0008] The fault indicator data of multiple fault performance indicators collected during the fault performance test of the target cloud resource pool are obtained, wherein the target cloud resource pool is one of the cloud resource pools except for the one with the lowest priority.

[0009] A pre-trained model is used to infer the fault index data to obtain the weights of multiple fault index features included in each fault performance index. The weights are used to characterize the importance of each fault index feature in evaluating the performance of the target cloud resource pool.

[0010] Based on the weights, select the most important target fault indicator features from each fault performance indicator to obtain the multiple target fault indicator features included in each fault performance indicator.

[0011] When it is necessary to perform fault performance testing on the remaining cloud resource pool, the fault performance of the remaining cloud resource pool is evaluated based on the obtained index data of the target fault index characteristics. The remaining cloud resource pool is the cloud resource pool other than the target cloud resource pool among the various cloud resource pools.

[0012] In one or more optional embodiments, the priority of each cloud resource pool is calculated based on the acquired index data of multiple normal performance indicators for each cloud resource pool, including:

[0013] A pre-trained indicator prediction model is used to infer the indicator data of multiple normal performance indicators for each of the cloud resource pools, and to predict the future indicator data of each of the normal performance indicators for each of the cloud resource pools within a future preset time range; and the indicator data of each of the normal performance indicators are weighted and normalized to obtain the weight of each of the normal performance indicators.

[0014] Based on the future indicator data and the weights corresponding to the future indicator data, the priority of each cloud resource pool is calculated.

[0015] In one or more optional embodiments, before acquiring fault indicator data of multiple fault performance indicators collected during fault performance testing of the target cloud resource pool, the method further includes:

[0016] From the remaining cloud resource pool, select a backup cloud resource pool with a lower priority than the target cloud resource pool;

[0017] The data stored in the target cloud resource pool is backed up as backup data to the backup cloud resource pool;

[0018] Once the fault performance test of the target cloud resource pool is completed, the backup data stored in the backup cloud resource pool is used to verify the data stored in the target cloud resource pool.

[0019] In one or more optional embodiments, where the remaining cloud resource pool has the lowest priority and the fault testing order is the latest, before performing a fault performance evaluation on the remaining cloud resource pool based on the obtained indicator data of the target fault indicator characteristics, the method further includes:

[0020] Obtain the fault performance evaluation results of all cloud resource pools except the remaining cloud resource pools in each cloud resource pool;

[0021] Based on the fault performance evaluation results, the cloud resource pool with the best evaluation results is selected from the cloud resource pools as the backup cloud resource pool for the remaining cloud resource pools.

[0022] The data stored in the remaining cloud resource pool is backed up as backup data to the backup cloud resource pool of the remaining cloud resource pool;

[0023] Once it is confirmed that the fault performance test of the remaining cloud resource pool has been completed, the backup data stored in the backup cloud resources is used to verify the data stored in the remaining cloud resource pool.

[0024] In one or more optional embodiments, based on the obtained index data of the remaining cloud resource pool regarding the target fault index characteristics, a fault performance assessment of the remaining cloud resource pool is performed, including:

[0025] Based on the indicator data of multiple target fault indicator features included in each of the fault performance indicators, an indicator matrix is ​​generated; and based on the weights of the fault indicator features included in each of the fault performance indicators, the weights of each of the fault performance indicators are obtained, and a weight matrix is ​​generated based on the weights of each of the fault performance indicators.

[0026] Calculate the product of the index matrix and the weight matrix, and use the result of the product as the fault performance evaluation result of the remaining cloud resource pool.

[0027] In one or more optional embodiments, obtaining the weight of each fault performance index based on the weight of the fault index features included in each fault performance index includes:

[0028] The average value is calculated for the weights of the fault indicator features included in each of the aforementioned fault performance indicators;

[0029] The result of the mean calculation is used as the weight of each of the fault performance indicators.

[0030] In one or more optional embodiments, the calculation process for the fault performance evaluation result of the target cloud resource pool includes:

[0031] Based on the collected fault performance index data, an index data matrix is ​​generated; and based on the weight of each fault performance index, a weight matrix is ​​generated.

[0032] Calculate the product of the indicator data matrix and the weight matrix, and use the result of the product as the performance evaluation result of the target cloud resource pool.

[0033] In one or more optional embodiments, the normal performance indicators include at least one of the following: recent access frequency, recent modification frequency, cloud resource pool storage capacity, and cloud resource pool storage utilization rate;

[0034] The fault performance indicators include at least one of the following: computing resources, storage resources, network resources, database services, and security and authentication services.

[0035] An embodiment of the second aspect of this application provides a performance evaluation apparatus, comprising:

[0036] The calculation module is used to calculate the priority of each cloud resource pool based on the acquired index data of multiple normal performance indicators of each cloud resource pool. The priority is used to characterize the degree of importance that users attach to the cloud resource pool.

[0037] The acquisition module is used to acquire fault indicator data of multiple fault performance indicators collected when performing fault performance testing on the target cloud resource pool, wherein the target cloud resource pool is one of the cloud resource pools other than the one with the lowest priority.

[0038] The weight inference module is used to infer the fault index data using a pre-trained model to obtain the weights of multiple fault index features included in each fault performance index. The weights are used to characterize the importance of each fault index feature in evaluating the performance of the target cloud resource pool.

[0039] The selection module is used to select multiple target fault indicator features with the highest importance from each fault performance indicator based on the weight, thereby obtaining multiple target fault indicator features included in each fault performance indicator.

[0040] The evaluation module is used to evaluate the fault performance of the remaining cloud resource pool based on the obtained index data of the remaining cloud resource pool in the target fault index characteristics when it is necessary to conduct fault performance testing on the remaining cloud resource pool. The remaining cloud resource pool is the cloud resource pool other than the target cloud resource pool among the various cloud resource pools.

[0041] An embodiment of the third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0042] An embodiment of the fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first aspect above.

[0043] An embodiment of the fifth aspect of this application provides a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method described in the first aspect above.

[0044] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0045] (1) Based on the fault indicator data of the target cloud resource pool, the most important fault indicator features are selected as target fault indicator features. Fault performance testing of the remaining cloud resource pool is then conducted based on these target fault indicator features. This not only reduces the number of test items, saves time and resources, and improves testing efficiency, but also ensures the reliability of testing the remaining cloud resource pool because the target fault indicator features come from the highest priority or higher-priority target cloud resource pool. This achieves a dual benefit of improving testing efficiency and ensuring test results. By conducting comprehensive fault performance testing on the highest priority or higher-priority target cloud resource pool, a comprehensive and reliable fault performance evaluation result can be obtained, increasing the importance attached to the target cloud resource pool and contributing to maintaining its security and stability over the future.

[0046] (2) Before the fault performance test, data backup of the target cloud resource pool and the remaining cloud resource pool can reduce the risk of data damage or loss in the cloud resource pool due to the introduction of faults. The backup data can be used to verify and restore the cloud resource pool, ensuring the integrity and reliability of the data, which is conducive to protecting user interests and improving security.

[0047] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0048] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0049] In the attached diagram:

[0050] Figure 1 This invention illustrates a flowchart of a cloud resource pool performance evaluation method provided in an embodiment of this application.

[0051] Figure 2 This invention illustrates a flowchart of a cloud resource pool data backup process provided in an embodiment of this application.

[0052] Figure 3 This invention illustrates a flowchart of a cloud resource pool data verification process provided in an embodiment of this application.

[0053] Figure 4 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0054] Figure 5 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0056] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0057] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0058] Research has found that when there are multiple cloud resource pools, existing technologies typically perform comprehensive, indiscriminate testing on each of the multiple cloud resource pools. This indiscriminate, comprehensive testing approach has drawbacks such as complex testing content, low testing efficiency, and high cost.

[0059] Based on the above research, this application provides a cloud resource pool performance evaluation method. By employing a fault-based testing approach, the performance evaluation of the cloud resource pool is more comprehensive and reliable. This facilitates the early detection of security and systemic vulnerabilities in the cloud resource pool, as well as the early exposure of architectural risks, thereby enabling better optimization and improvement of the cloud resource pool. By conducting comprehensive fault performance testing on the target cloud resource pool with the highest priority or higher priority, a comprehensive and reliable fault performance evaluation result can be obtained, increasing the importance placed on the target cloud resource pool and contributing to maintaining its security and stability over a future period. Simultaneously, based on the obtained fault indicator data of the target cloud resource pool, the most important fault indicator features are selected as target fault indicator features. Fault performance testing of the remaining cloud resource pools is then conducted based on these target fault indicator features, reducing the number of fault indicator features that need to be tested, saving time, manpower, and material resources, and improving testing efficiency.

[0060] To facilitate understanding of this embodiment, a detailed description of the cloud resource pool performance evaluation method disclosed in this application embodiment will be provided first. The execution subject of the cloud resource pool performance evaluation method provided in this application embodiment is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the cloud resource pool performance evaluation method can be implemented by the processor calling computer-readable instructions stored in memory.

[0061] like Figure 1The diagram shows a flowchart of a cloud resource pool performance evaluation method provided in this application embodiment. The method includes the following steps 1-5:

[0062] Step 1: Based on the acquired data of multiple normal performance indicators for each cloud resource pool, calculate the priority of each cloud resource pool.

[0063] Priority is used to characterize the user's level of attention to the cloud resource pool. Normal performance indicators include at least one of the following: recent access frequency, recent modification frequency, cloud resource pool storage capacity, and cloud resource pool storage utilization rate. It should be noted that, taking an example with two cloud resource pools and the above four normal performance indicators, when obtaining the indicator data for each cloud resource pool, the indicator data for indicator 1, indicator 2...indicator 4 of the first cloud resource pool are statistically analyzed, as are the indicator data for indicator 1, indicator 2...indicator 4 of the second cloud resource pool. Here, indicator 1 represents recent access frequency, indicator 2 represents recent modification frequency, indicator 3 represents cloud resource pool storage capacity, and indicator 4 represents cloud resource pool storage utilization rate.

[0064] The specific implementation of priority calculation can be as follows:

[0065] Method 1: Based on the above-mentioned normal performance indicators, manually set or indicate the priority of each cloud resource pool;

[0066] Method 2: Specifically includes the following steps:

[0067] 1-1: A pre-trained metric prediction model is used to infer the metric data of multiple normal performance metrics for each cloud resource pool, predicting the future metric data of each normal performance metric within a preset future time range. Furthermore, the metric data of each normal performance metric is weighted and normalized to obtain the weights of each normal performance metric. Specifically, the metric prediction model can be a time series model (such as ARIMA or Prophet), combined with the four normal performance metrics mentioned above. The future metric data includes future access frequency, future modification frequency, future cloud resource pool storage volume, and future cloud resource pool storage utilization. The preset future time range can be the next week or the next month; users can adjust the specific time range according to their actual needs. The specific implementation method for calculating the weights can be the analytic hierarchy process (AHP).

[0068] 1-2: Calculate the priority of each cloud resource pool based on future indicator data and the corresponding weights. Specifically, the formula for calculating the priority of a cloud resource pool can be:

[0069] I = α*FF + β*FR + γ*FS + θ*FSU

[0070] Where I represents the priority of the cloud resource pool, α, β, γ, and θ represent the weights corresponding to recent access frequency, modification frequency, cloud resource pool storage volume, and cloud resource pool storage utilization rate, respectively, FF represents the future access frequency of the cloud resource pool, FR represents the future modification frequency of the cloud resource pool, FS represents the future storage volume of the cloud resource pool, and FSU represents the future storage utilization rate of the cloud resource pool.

[0071] Step 2: Obtain fault indicator data for multiple fault performance metrics collected during fault performance testing of the target cloud resource pool. The target cloud resource pool is the cloud resource pool other than the one with the lowest priority among all the cloud resource pools. Assuming there are 5 cloud resource pools, the target cloud resource pool can be selected from the top 4 with higher priority as a better option. The target cloud resource pool is the cloud resource pool with the highest priority. That is, the cloud resource pool that will be used most frequently in the future, has the largest storage capacity, and the highest usage rate. Prioritizing its fault performance testing not only reduces the impact on future use but also allows for faster vulnerability patching and functional improvement, ensuring the normal operation of the highest priority cloud resource pool and improving user evaluation.

[0072] Specifically, fault performance metrics include at least one of the following: computing resources, storage resources, network resources, database services, and security and authentication services. Computing resources include, but are not limited to, virtual machine instances, container instances, and compute instances; storage resources include, but are not limited to, object storage, block storage, and file storage; network resources include, but are not limited to, virtual networks, load balancing, and DNS services; database services include, but are not limited to, relational databases and NoSQL databases; and security and authentication services include, but are not limited to, identity and access management, encryption services, and security auditing. Each fault performance metric includes multiple fault indicator characteristics. Taking computing resources as an example, computing resources may include four fault indicator characteristics: highest load rate, longest continuous runtime, self-restart success rate, and fault impact scope. Furthermore, users can modify the fault indicator characteristics included in each fault performance metric according to their actual needs.

[0073] Step 3: Using a pre-trained model, inference is performed on the fault indicator data to obtain the weights of multiple fault indicator features included in each fault performance indicator. These weights characterize the importance of each fault indicator feature in evaluating the performance of the target cloud resource pool. Specifically, a random forest regression model can be selected to infer the fault indicator data.

[0074] The specific steps involved in evaluating the performance of a target cloud resource pool are as follows:

[0075] S1: Based on the fault index data of multiple fault performance indicators collected in step 2, generate an index data matrix; and based on the calculated weights of each fault performance indicator, generate a weight matrix.

[0076] S2: Calculate the product of the indicator data matrix and the weight matrix, and use the result of the product as the performance evaluation result of the target cloud resource pool. Referring to step 2 of this embodiment, the formula for calculating the fault performance evaluation result of the target cloud resource pool is as follows:

[0077]

[0078] Where M is the performance evaluation score of the target cloud resource pool, W is the indicator data matrix storing the fault performance indicator μ, and X is the weight matrix storing the weight σ. T μ is the transpose of matrix X. 计算 σ represents the fault index data of the target cloud resource pool in terms of computing resources. 计算 The weight of the computing resources in the target cloud resource pool can be represented as follows:

[0079]

[0080]

[0081] The remaining parameters are described in the same manner, and will not be elaborated upon in this embodiment.

[0082] Step 4: Based on the weights obtained in Step 3, select the most important target fault indicator features from each fault performance indicator to obtain the multiple target fault indicator features included in each fault performance indicator. Note that for each fault performance indicator, the number of target fault indicator features included is less than the number of fault indicator features. Taking "computing resources" as an example from Step 2, assuming the weights of the four fault indicator features included in computing resources are in the following order: highest load rate > longest continuous runtime > self-restart success rate > fault impact range, 1-3 of the four fault indicator features with higher weights can be selected as target fault indicator features according to actual needs. Assuming the top three with higher weights are selected as target fault indicator features, then when performing fault performance testing on the remaining cloud resource pool, the computing resource fault performance indicator will include three target fault indicator features. Other fault performance indicators (network resources, storage resources, etc.) follow a similar analogy, and will not be elaborated further in this embodiment.

[0083] Step 5: If fault performance testing of the remaining cloud resource pool is required, conduct a fault performance evaluation of the remaining cloud resource pool based on the obtained indicator data of the target fault indicator characteristics. The remaining cloud resource pool is one of the cloud resource pools other than the target cloud resource pool. The specific steps include:

[0084] Step 5.1: Generate an indicator matrix based on the indicator data of the acquired target fault indicator features; and obtain the weight of each fault performance indicator based on the weight of the fault indicator features included in each fault performance indicator, and generate a weight matrix based on the weight of each fault performance indicator.

[0085] The method for obtaining the weights of each fault performance index can be:

[0086] (1) Select the largest or smallest weight among the weights of the fault indicator features included in each fault performance indicator as the weight of each fault performance indicator.

[0087] (2) Calculate the average weight of each fault performance indicator's features, and use the average as the weight of each fault performance indicator. Taking "computing resources" as an example, Mean σ 计算 The weight for "computing resources".

[0088] Step 5.2: Calculate the product of the index matrix and the weight matrix, and use the result of the product as the fault performance evaluation result of the remaining cloud resource pool. Referring to steps 2 and 3 of this embodiment, the formula for calculating the fault performance evaluation result of the remaining cloud resource pool is as follows:

[0089]

[0090] Where M is the performance evaluation score of the remaining cloud resource pool, W is the indicator data matrix storing the fault performance indicator μ, and X is the weight matrix storing the weight σ. T This is the transpose of matrix X. Taking "computing resources" as an example, a fault performance metric, μ... 计算 σ represents the fault index data of the remaining cloud resource pool in terms of computing resources. 计算 The weight of the remaining cloud resource pool computing resources can be represented as follows:

[0091]

[0092] It should be noted that the descriptions of other fault performance indicators are similar, and will not be repeated in this embodiment.

[0093] Research has found that existing technologies for fault testing of cloud resource pools typically employ methods that directly introduce faults. Due to the complexity and diversity of hardware devices and program architectures within cloud resource pools, this method of introducing faults poses a potential risk to the cloud resource pools, potentially leading to irreversible system failures, resulting in the loss or damage of stored data. This would severely harm customer interests and lower user ratings.

[0094] Therefore, the difference between this embodiment and the previous embodiments is as follows:

[0095] S1: Before step 2 of embodiment 1, the method further includes: backing up the data stored in the target cloud resource pool to a backup cloud resource pool as backup data. The backup cloud resource pool has a lower priority than the target cloud resource pool; more preferably, the backup cloud resource pool can be the cloud resource pool with the lowest priority. Backing up the target cloud resource pool before conducting fault performance testing reduces the risk of data corruption or loss due to introduced faults, ensuring the integrity and reliability of stored data, protecting user interests, and improving security. Figure 2 As shown, the specific process may include the following steps:

[0096] S1.1: Obtain authentication and operation authorization for the target cloud resource pool and the backup cloud resource pool respectively, and establish a connection and transmission channel between the two cloud resource pools.

[0097] S1.2: Set backup strategy, which may include: selecting appropriate backup tools and setting backup parameters; wherein, backup parameters may include backup content, backup cycle, data transmission encryption, etc.

[0098] S1.3: Based on the set backup strategy, manually or automatically trigger the backup task, execute data transfer, and transfer data from the target cloud resource pool to the backup cloud resource pool. During the transfer process, data can be stored in a designated area within the backup cloud resource pool, and a monitoring program can be set to monitor the transferred data in real time to ensure data integrity and security and prevent harmful data from intruding into the cloud resource pool.

[0099] S1.4: After the data backup is complete, verify the backed-up data on the backup cloud resource pool to ensure that the data is complete and can be accessed normally; and perform a recovery test on the backed-up data to verify that the backed-up data can be successfully restored.

[0100] S1.5: After successful verification, record the backup process in the system log of the executing entity for subsequent search and optimization; disconnect the connection between the two resource pools before performing fault performance testing on the target cloud resource pool.

[0101] S2: Following step 2 of embodiment 1, the method further includes: after confirming the completion of the fault performance test on the target cloud resource pool, verifying the data stored in the target cloud resource pool using backup data stored in the backup cloud resource pool, to check whether any abnormalities such as data loss or damage have occurred in the data stored in the target cloud resource pool after the fault performance test, so as to ensure the integrity and security of the data. Figure 3 As shown, the specific steps may include:

[0102] S2.1: Establish a connection and transmission channel between the target cloud resource pool and the backup cloud resource pool.

[0103] S2.2: Retrieve the backup data stored in the backup cloud resource pool and verify it against the original data stored in the target cloud resource pool. Check whether there are any abnormalities such as data changes, loss, or damage in the target cloud resource pool after the fault performance test. If so, export the backup data to the target cloud resource pool and restore the data in the target cloud resource pool according to the backup data. Once the restoration is complete, the verification is considered successful. If there are no abnormalities, the verification is successful directly.

[0104] S2.3: After successful verification, the backup data stored in the backup cloud resource pool is transferred to the recycle area. This can be configured to be deleted immediately or retained for a period before deletion. Furthermore, the storage space in the designated area of ​​the backup cloud resource pool used for storing backup data can be increased or decreased based on the amount of data stored in the next remaining cloud resource pool to be tested. This space is used to receive backup data from the next remaining cloud resource pool to be tested.

[0105] S2.4: Record the actions of changing backup data in the system log of the executing entity.

[0106] S3: Before step 5 of embodiment 1, the method further includes: before performing fault performance testing on the remaining cloud resource pool, backing up the data stored in the remaining cloud resource pool to the backup cloud resource pool as backup data.

[0107] This reduces the risk of data corruption or loss in the remaining cloud resource pool due to introduced faults, ensuring the integrity and reliability of stored data, protecting user interests, and improving security. Specific steps and procedures can be found in steps S1.1 to S1.5 of this embodiment, and will not be repeated here.

[0108] It should be noted in this embodiment that the target cloud resource pool and the remaining cloud resource pool can be the same or different cloud resource pools as backup cloud resource pools during data backup. Users can configure the specific backup scheme according to their actual needs. This embodiment also provides backup schemes for reference in the following two special cases:

[0109] Scenario 1: If the remaining cloud resource pool has the lowest priority and the latest fault testing order, meaning it is the last cloud resource pool to be tested, indicating that all other cloud resource pools have completed fault performance testing and obtained fault performance evaluation results, then the specific backup plan for the remaining cloud resource pool may include the following steps:

[0110] 1-1: Obtain the fault performance evaluation results of all cloud resource pools except the remaining cloud resource pools in each cloud resource pool;

[0111] 1-2: Based on the fault performance evaluation results, select the cloud resource pool with the best evaluation results as the backup cloud resource pool for the remaining cloud resource pools;

[0112] 1-3: Repeat steps S1.1 to S1.5 to back up the data stored in the remaining cloud resource pool to the backup cloud resource pool of the remaining cloud resource pool as backup data;

[0113] 1-4: After confirming that the fault performance test of the remaining cloud resource pool has been completed, repeat steps S2.1 to S2.4 to verify the data stored in the remaining cloud resource pool using the backup data stored in the backup cloud resources.

[0114] Scenario 2: When it is necessary to perform fault performance testing on the backup cloud resource pool, the specific backup scheme for the backup cloud resource pool can be:

[0115] Option 1: Back up the data stored in the backup cloud resource pool to be tested to another backup cloud resource pool. The specific backup steps can be repeated from S1.1 to S1.5. After confirming that the fault performance test of the backup cloud resource pool to be tested has been completed, repeat steps S2.1 to S2.4 and use the backup data stored in the backup cloud resource pool to verify the data stored in the backup cloud resource pool to be tested.

[0116] Option 2: Back up the data stored in the backup cloud resource pool to be tested to a cloud resource pool with lower priority. The specific backup steps can be repeated from S1.1 to S1.5. After confirming that the fault performance test of the backup cloud resource pool to be tested has been completed, repeat steps S2.1 to S2.4 and use the backup data stored in the backup cloud resource pool to verify the data stored in the backup cloud resource pool to be tested.

[0117] Option 3: Back up the data stored in the backup cloud resource pool to be tested to the cloud resource pool with the best evaluation results after the test has been completed. The specific backup steps can be repeated from S1.1 to S1.5.

[0118] Once the fault performance test of the backup cloud resource pool to be tested is completed, repeat steps S2.1 to S2.4 to verify the data stored in the backup cloud resource pool using the backup data stored in the backup cloud resource pool.

[0119] This embodiment provides a method for evaluating the performance of cloud resource pools, which specifically includes the following steps:

[0120] Step 1: Based on the acquired performance metrics of each cloud resource pool, calculate the priority of each cloud resource pool. Step 1 specifically includes the following steps:

[0121] Step 1.1: Obtain multiple normal performance metrics for each cloud resource pool:

[0122] In this embodiment, the normal performance indicators include at least one of the following: recent access frequency, recent modification frequency, cloud resource pool storage capacity, and cloud resource pool storage utilization. This embodiment uses the above four normal performance indicators as an example for explanation and illustration. The indicator data of the four normal performance indicators of each cloud resource pool are obtained in sequence. Step 1.1 may specifically include the following steps:

[0123] Access the central console of the cloud resource pool, which can be the control system of the executing entity, and retrieve the access logs, operation logs and storage capacity logs of each cloud resource pool recorded in the central console;

[0124] The retrieved access logs, operation logs, and storage capacity logs were preprocessed, and the log data was cleaned and organized to ensure the integrity and accuracy of the logs. Any outliers, missing data, and erroneous records were handled. The cleaned logs are shown in Table 1.

[0125]

[0126] Table 1

[0127] Extract normal performance metrics from the preprocessed logs, namely, recent access frequency, recent modification frequency, cloud resource pool storage volume, and cloud resource pool storage utilization, as shown in Table 2:

[0128] F = A / x R = B / x S = CD SU = (CD) / C

[0129] Table 2

[0130] Where x represents the number of recent days selected, A represents the number of recent external accesses, B represents the number of recent internal modifications, C represents the total storage capacity of the cloud resource pool, and D represents the remaining storage space of the cloud resource pool.

[0131] Step 1.2: Based on the normal performance index data extracted in Step 1.1, a pre-trained index prediction model is used to infer the index data of multiple normal performance indices for each cloud resource pool, and to predict the future access frequency, future modification frequency, future storage amount and future storage utilization rate of each cloud resource pool within a preset time range.

[0132] Specifically, the indicator prediction model can be a time series model (such as ARIMA or Prophet). The model is trained using historical data from each cloud resource pool, and its reliability and rationality are evaluated through cross-validation or by reserving a subset of data as a validation set. Considering indicators such as the accuracy, mean squared error, and prediction error of the indicator prediction model, the trained model is then used to predict the usage and storage volume of cloud resource pools in the future. Based on the prediction results, cloud resource pools that are likely to be frequently used and have large storage volumes in the future are identified and located.

[0133] Step 1.3: The recent access frequency, recent modification frequency, cloud resource pool storage capacity, and cloud resource pool storage utilization rate extracted in Step 1.1 are weighted and normalized to determine the weight of each indicator. This embodiment includes, but is not limited to, calculating the weights using the following methods:

[0134] The relative importance is measured using the analytic hierarchy process (AHP), one of the most common methods in existing statistical analysis. Initial weighting is performed first. Recent access frequency represents the frequency of recent use by customers; considering that the cloud resource pool primarily considers customers, recent access frequency is weighted at 4. Recent modification frequency represents the frequency of recent internal operations on it, and is weighted at 3. Cloud resource pool storage rate determines whether there is excessive idle space, and is weighted at 2. Storage volume determines the amount of content stored in the cloud resource pool, and is weighted at 1. Based on the extracted data of recent access frequency, recent modification frequency, cloud resource pool storage volume, and cloud resource pool storage utilization rate, Table 3 is obtained:

[0135]

[0136] Table 3

[0137] Specifically, this embodiment uses the analytic hierarchy process (AHP). Taking the data "4 / 3" in Table 3 as an example, it represents the ratio of recent access frequency F to recent modification frequency R as 4 / 3. The sum of these values ​​is the value for that column. The initial weight vector calculated from Table 3 is shown in Table 4.

[0138] Recent modifications to frequency R and initial weights ＝(3 / 4+1+3+3 / 2) / 4≈1.562 Cloud resource pool storage S initial weight ＝(1 / 4+1 / 3+1+1 / 2) / 4≈0.521 Cloud resource pool storage utilization (SU) initial weight ＝(1 / 2+2 / 3+2+1) / 4≈1.042

[0139] Table 4

[0140] After normalizing the initial weight vector in Table 4, the weights corresponding to each index are shown in Table 5:

[0141] Recent modifications to frequency R weight β ＝1.562 / (2.083+1.562+0.521+1.042)≈0.300 Cloud resource pool storage S weight γ ＝0.521 / (2.083+1.562+0.521+1.042)≈0.100 Cloud resource pool storage utilization SU weight θ ＝1.042 / (2.083+1.562+0.521+1.042)≈0.200

[0142] Table 5

[0143] It should be noted that the weight calculation method in this embodiment is only used to explain the technical solution, and the calculated weights can be adjusted according to actual needs.

[0144] Step 1.4: Based on the future access frequency, future modification frequency, future storage volume and future storage utilization of each cloud resource pool within a preset time range predicted in Step 1.2, and combined with the weights determined in Step 1.3, calculate the importance of the cloud resource pool, i.e., the priority. The priority is used to characterize the degree of importance that users attach to the cloud resource pool.

[0145] Specifically, the formula for calculating the priority of a cloud resource pool is as follows:

[0146] I=α*FF+β*FR+γ*FS+θ*FSU,

[0147] Where I represents the priority of the cloud resource pool, α, β, γ, and θ represent the weights corresponding to recent access frequency, modification frequency, cloud resource pool storage volume, and cloud resource pool storage utilization rate, respectively, FF represents the future access frequency of the cloud resource pool, FR represents the future modification frequency of the cloud resource pool, FS represents the future storage volume of the cloud resource pool, and FSU represents the future storage utilization rate of the cloud resource pool.

[0148] Step 2: Select the target cloud resource pool from each cloud resource pool, and select the backup cloud resource pool from each cloud resource pool.

[0149] Based on the priority obtained in step 1, a test sequence of cloud resource pools can be established, and each cloud resource pool undergoes fault performance testing sequentially according to the test sequence. In this embodiment, the cloud resource pool with the highest priority is selected as the target cloud resource pool, and the cloud resource pool with the lowest priority is selected as the backup cloud resource pool. The highest priority cloud resource pool represents the cloud resource pool that will be used most frequently in the future, has the largest storage capacity, and the highest usage rate. Prioritizing its fault performance testing not only reduces the impact on future use but also allows for faster vulnerability patching and functional improvement, ensuring the normal operation of the highest priority cloud resource pool and improving user evaluation. Selecting the lowest priority cloud resource pool as the backup cloud resource pool not only improves the storage space utilization of the lowest priority cloud resource pool but also saves resources and costs.

[0150] Step 3: Before conducting fault performance testing on the target cloud resource pool, back up the data stored in the target cloud resource pool to the backup cloud resource pool. Backing up the target cloud resource pool before fault performance testing reduces the risk of data corruption or loss due to introduced faults, ensuring the integrity and reliability of stored data, protecting user interests, and improving security. Step 3 specifically includes the following steps:

[0151] Step 3.1: Obtain authentication and operation authorization for the target cloud resource pool and the backup cloud resource pool respectively, and establish a connection and transmission channel between the two cloud resource pools.

[0152] Step 3.2: Set up a backup strategy, which may include: selecting appropriate backup tools and setting backup parameters; the backup parameters may include backup content, backup cycle, data transmission encryption, etc.

[0153] Step 3.3: Based on the set backup strategy, manually or automatically trigger the backup task to execute data transfer, transferring data from the target cloud resource pool to the backup cloud resource pool. During the transfer process, data can be stored in a designated area within the backup cloud resource pool, and a monitoring program can be set up to monitor the transferred data in real time to ensure data integrity and security, and prevent harmful data from intruding into the cloud resource pool.

[0154] Step 3.4: After the data backup is complete, verify the backed-up data on the backup cloud resource pool to ensure that the data is complete and accessible; and perform a recovery test on the backed-up data to verify that the backed-up data can be successfully restored.

[0155] Step 3.5: After successful verification, record the backup process in the system log of the executing entity for subsequent lookup and optimization; disconnect the connection between the two resource pools before performing fault performance testing on the target cloud resource pool.

[0156] Step 4: Perform fault performance testing on the target cloud resource pool, and obtain fault indicator data for multiple fault performance indicators collected during the fault performance testing of the target cloud resource pool. In this embodiment, the fault performance indicators include: computing resources, storage resources, network resources, database services, and security and authentication services. Each fault performance indicator includes multiple fault indicator characteristics. Step 4 specifically includes the following steps:

[0157] Step 4.1: Perform fault testing on the computing resources of the cloud resource pool; computing resources include: virtual machine instances, container instances, and compute instances, etc.

[0158] Specifically, the test content can be selected as follows:

[0159] Stability and load capacity: Testing the stability and performance of computing resources under high load conditions;

[0160] Automated recovery capability: Simulate computing resource failures and observe the system's automatic recovery capability, such as automatic restart and failover.

[0161] Resource isolation: Verify the isolation performance between different computing resources to prevent the failure of one resource from affecting other resources.

[0162] Based on the above test content, the fault indicators of computing resources can be selected as: highest load rate, longest continuous runtime, self-restart success rate, and fault impact range.

[0163] Step 4.2: Perform fault testing on the storage resources of the cloud resource pool; storage resources include: object storage, block storage, and file storage, etc.

[0164] Specifically, the test content can be selected as follows:

[0165] Data integrity and reliability: Test the data integrity and reliability of the storage system, and simulate data recovery performance under failure conditions.

[0166] Capacity and Performance: Test the capacity limits and performance metrics of storage resources, including read / write speed and response time.

[0167] Backup and Recovery: Verify backup and recovery systems to ensure data can be quickly recovered in the event of a failure.

[0168] Based on the above test content, the fault indicators of storage resources can be selected as: data integrity rate, maximum capacity, response time, data recovery rate, etc.

[0169] Step 4.3: Perform fault testing on the network resources of the cloud resource pool; network resources include: virtual networks, load balancers, DNS services, etc.

[0170] Specifically, the test content can be selected as follows:

[0171] Load balancing and high availability: Test the load balancing system to ensure the stability of network services under different loads.

[0172] Network Isolation and Security: Verify network isolation performance and security policies to ensure isolation between different networks and protect systems from network attacks.

[0173] Failover and Routing: Simulate network device failures to test routing and failover mechanisms and ensure service availability.

[0174] Based on the above test content, the fault indicators of network resources can be selected as: stability rate under different loads, isolation performance, and failover success rate.

[0175] Step 4.4: Perform fault testing on the database services of the cloud resource pool; database services include: relational databases, NoSQL databases, etc.

[0176] Specifically, the test content can be selected as follows:

[0177] Data consistency and recovery: Test the database's data consistency and recovery capabilities under failure conditions, including data integrity and data backup.

[0178] Performance stability: Test the performance stability of the database, especially its performance under high load and large-scale query conditions.

[0179] Scalability and Partitioning: Verify the scalability of the database, including horizontal and vertical scaling, and test partitioning and sharding capabilities.

[0180] Based on the above test content, the fault indicators of the database service can be selected as: data integrity rate, data recovery success rate, runtime, etc.

[0181] Step 4.5: Perform fault testing on the security and authentication services of the cloud resource pool; the security and authentication services include: identity and access management, encryption services, security auditing, etc.

[0182] Specifically, the test content can be selected as follows:

[0183] Authentication and Authorization: Test the accuracy and security of the authentication system to ensure that only authorized users can access resources.

[0184] Vulnerability and security scanning: Perform vulnerability scanning and security testing to ensure that the service is free of security vulnerabilities.

[0185] Incident Response and Monitoring: Verify the monitoring and response mechanisms for security incidents, as well as the system's performance under security threats.

[0186] Based on the above test content, the fault indicators of security and authentication services can be selected as: authentication pass rate, number of vulnerabilities scanned, response time, etc.

[0187] It should be noted that the test content and fault indicator characteristics selected in this embodiment only include the important parts in the usual definition. Depending on the actual situation and different needs, the test content and fault indicator characteristics can be modified, and different test schemes can be selected. This step is only for the purpose of obtaining fault indicator data under fault performance test.

[0188] Step 5: After confirming the completion of the fault performance test on the target cloud resource pool, verify the data stored in the target cloud resource pool using the backup data stored in the backup cloud resource pool. Step 5 specifically includes the following steps:

[0189] Step 5.1: Establish a connection and transmission channel between the target cloud resource pool and the backup cloud resource pool.

[0190] Step 5.2: Retrieve the backup data stored in the backup cloud resource pool and verify it with the original data stored in the target cloud resource pool. Check whether there are any abnormalities such as data changes, data loss, or data corruption in the target cloud resource pool after the fault performance test. If so, export the backup data to the target cloud resource pool and restore it according to the backup data. The verification is successful after the restoration is completed. If not, the verification is successful.

[0191] Step 5.3: After successful verification, transfer the backup data stored in the backup cloud resource pool to the recycle area. This can be configured for immediate deletion or retention for a period before deletion. Additionally, based on the amount of data stored in the remaining cloud resource pool to be tested, increase or decrease the storage space in the designated area of ​​the backup cloud resource pool used to store backup data, to receive backup data from the next remaining cloud resource pool to be tested.

[0192] Step 5.4: Record the action of changing the backup data in the system log of the executing entity.

[0193] Step 6: Using a pre-trained model, inference is performed on the fault indicator data obtained in Step 4 to obtain the weights of multiple fault indicator features included in each fault performance indicator, and the fault performance evaluation result of the target cloud resource pool is calculated. Step 6 specifically includes the following steps:

[0194] Step 6.1: Classify and store the fault index data of each fault index feature obtained in Step 4 into a test dataset, denoted as TheMostImportant_test_data;

[0195] Step 6.2: Select appropriate feature engineering to analyze the importance (i.e., weights) of the fault indicator features in the test dataset. Specifically, this embodiment selects a random forest regression model to train the test dataset. The implementation method can be the scikit-learn library in Python to process the data and calculate the weight of each fault indicator feature.

[0196] The calculation process for the fault performance evaluation results of the target cloud resource pool specifically includes the following steps:

[0197] 1-1: Based on the fault index data of multiple fault performance indicators collected in step 4, generate an index data matrix; and based on the calculated weights of each fault performance indicator, generate a weight matrix.

[0198] 1-2: Calculate the product of the indicator data matrix and the weight matrix, and use the result of the product as the performance evaluation result of the target cloud resource pool.

[0199] Referring to step 4 of this embodiment, the formula for calculating the fault performance evaluation result of the target cloud resource pool is as follows:

[0200]

[0201] Where M is the performance evaluation score of the target cloud resource pool, W is the indicator data matrix storing the fault performance indicator μ, i.e., the matrix form of the test dataset TheMostImportant_test_data; X is the matrix storing the weights σ. T μ is the transpose of matrix X. 计算 σ refers to the test data of the target cloud resource pool in terms of computing resources. 计算 This refers to the weight of the performance index of computing resource failure in the target cloud resource pool, specifically:

[0202]

[0203]

[0204] The remaining fault performance indicators are described in the same way, and will not be elaborated upon in this embodiment.

[0205] Step 7: Based on the weights obtained in Step 6, select the multiple target fault indicator features with the highest weights from each fault performance indicator to obtain the multiple target fault indicator features included in each fault performance indicator. The number of target fault indicator features included in each fault performance indicator is less than the number of fault indicator features. Taking "computing resources" as an example, assuming that the weights of the fault indicator features included in computing resources are in the following order: highest load rate > longest continuous runtime > self-restart success rate > fault impact range, users can select 1-3 of the four fault indicator features with higher weights as target fault indicator features according to their actual needs. Other fault performance indicators (network resources, storage resources, etc.) follow the same principle, which will not be elaborated further in this embodiment.

[0206] Step 7 may also include the following steps:

[0207] After obtaining the target fault indicator features, they are stored in a feature dataset, denoted as TheMostFeature. A suitable algorithm is used to build a test model, which is then used to sequentially test the fault performance of the remaining cloud resource pools. In this embodiment, a neural network is preferably chosen as the implementation method for building the model. In practical applications, parameters such as the structure, number of layers, number of neurons, and activation function of the neural network can be adjusted as needed to achieve better performance of the test model. The trained model can be evaluated using a test set to observe indicators such as accuracy, and adjustments and optimizations can be made as needed.

[0208] Based on the fault indicator characteristics obtained after testing the target cloud resource pool (i.e., the highest priority cloud resource pool), several important ones are extracted as target fault indicator characteristics, making the number of target fault indicator characteristics less than the number of fault indicator characteristics. A test model is established based on the target fault indicator characteristics, and the fault performance of the remaining cloud resource pool is tested through the test model. This not only reduces the number of fault indicator characteristics that need to be tested, saving costs and resources and improving testing efficiency, but also, since the target cloud resource pool is the highest priority cloud resource pool, the priority reflects the importance of the cloud resource pool. Therefore, the various indicators of the target cloud resource pool can also reflect the parts that users value most. The target fault indicator characteristics selected based on the content that users value most are the most representative. In fact, this simplifies the testing content of the remaining cloud resource pool while also comprehensively considering the more important fault indicator characteristics.

[0209] Step 8: Before conducting fault performance testing on the remaining cloud resource pool, back up the data stored in the remaining cloud resource pool to the backup cloud resource pool as backup data. For specific backup plans, refer to steps 3.1 to 3.5. The remaining cloud resource pool is one of the cloud resource pools other than the target cloud resource pool. All remaining cloud resource pools are backed up and subjected to fault performance testing sequentially according to the test sequence. This means setting the test order based on the user's level of importance to each cloud resource pool not only reduces the impact on future use but also allows for the rapid patching of vulnerabilities and functional improvements to cloud resource pools that are of higher importance to the user, maintaining the normal operation of the cloud resource pools and improving user feedback.

[0210] Step 9: Based on the target fault indicator characteristics, conduct a fault performance assessment of the remaining cloud resource pool.

[0211] The calculation process for the fault performance assessment results of the remaining cloud resource pool specifically includes the following steps:

[0212] 1-1: Generate an index matrix based on the index data of multiple target fault index features included in each fault performance index; and obtain the weight of each fault performance index based on the weight of the fault index features included in each fault performance index, and generate a weight matrix based on the weight of each fault performance index.

[0213] Calculate the product of the indicator matrix and the weight matrix, and use the result of the product as the fault performance evaluation result of the remaining cloud resource pool.

[0214] Using steps 4 and 6 of this embodiment as examples, refer to the calculation formula for the fault performance evaluation score of the target cloud resource pool:

[0215]

[0216] Where M is the performance evaluation score of the remaining cloud resource pool, W is a matrix storing the fault performance index μ, and X is a matrix storing the weights σ. T Let X be the transpose of matrix X.

[0217] Referring to step 4 of this embodiment, μ 计算 σ refers to the indicator data representing the target failure characteristics of cloud resource pools in terms of computing resources. 计算 This refers to the weight of the corresponding cloud resource pool computing resources. Taking the "computing resource" fault performance indicator as an example, "highest load rate," "longest continuous runtime," and "self-restart success rate" were selected as target fault indicator features, while "fault impact range" and other fault indicator features with less impact were removed.

[0218]

[0219] Specifically, the weights of each fault performance index are obtained based on the weights of the fault indicator features included in each fault performance index. This includes averaging the weights of the respective fault indicator features included in each fault performance index. The average value is used as the weight for each of the aforementioned fault performance indicators; that is, the average of these weights is the weight of the fault performance indicator "computing resources". The remaining parameters are explained by analogy, and will not be elaborated upon in this embodiment.

[0220] By selecting target fault indicator features to evaluate the fault performance of the remaining cloud resource pool, the number of matrix rows of storage fault performance indicator μ is reduced, and fault indicator features with low impact are removed, which helps to improve testing efficiency. At the same time, taking the average weight as the weight of each fault performance indicator helps to ensure the consistency and reliability of the test results.

[0221] Step 10: After confirming that the fault performance test of the remaining cloud resource pool has been completed, use the backup data stored in the backup cloud resource pool to verify the data stored in the remaining cloud resource pool. For details, please refer to steps 5.1 to 5.4.

[0222] This application also provides a performance evaluation apparatus, including:

[0223] The calculation module is used to calculate the priority of each cloud resource pool based on the acquired indicator data of multiple normal performance indicators of each cloud resource pool. The priority is used to characterize the user's importance to the cloud resource pool.

[0224] The acquisition module is used to acquire fault indicator data of multiple fault performance indicators collected when performing fault performance testing on the target cloud resource pool. The target cloud resource pool is one of the cloud resource pools except for the one with the lowest priority.

[0225] The weight inference module is used to infer the data of each fault indicator using a pre-trained model, and obtain the weights of multiple fault indicator features included in each fault performance indicator. The weights are used to characterize the importance of each fault indicator feature in evaluating the performance of the target cloud resource pool.

[0226] The selection module is used to select multiple target fault indicator features with the highest importance from each fault performance indicator based on the weight, thereby obtaining multiple target fault indicator features included in each fault performance indicator.

[0227] The evaluation module is used to evaluate the fault performance of the remaining cloud resource pool based on the obtained indicator data of the remaining cloud resource pool in terms of the target fault indicator characteristics when it is necessary to conduct fault performance testing on the remaining cloud resource pool. The remaining cloud resource pool is a cloud resource pool other than the target cloud resource pool among all cloud resource pools.

[0228] This application also provides an electronic device for performing the above-described cloud resource pool performance evaluation method. Please refer to... Figure 4 It illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 4 As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802, and a communication interface 803. The processor 800, the communication interface 803, and the memory 801 are connected through the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the cloud resource pool performance evaluation method provided in any of the foregoing embodiments of this application.

[0229] The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0230] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs. After receiving an execution instruction, the processor 800 executes the program. The cloud resource pool performance evaluation method disclosed in any of the foregoing embodiments of this application can be applied to the processor 800, or implemented by the processor 800.

[0231] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0232] The electronic device provided in this application embodiment and the cloud resource pool performance evaluation method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0233] This application also provides a computer-readable storage medium corresponding to the cloud resource pool performance evaluation method provided in the foregoing embodiments. Please refer to... Figure 5 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the cloud resource pool performance evaluation method provided in any of the aforementioned embodiments.

[0234] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0235] The computer-readable storage medium provided in the above embodiments of this application and the cloud resource pool performance evaluation method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0236] This application also provides a computer program product that carries program code. The instructions included in the program code can be used to execute the steps of the cloud resource pool performance evaluation method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0237] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0238] It should be noted that:

[0239] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0240] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0241] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0242] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for evaluating the performance of a cloud resource pool, characterized in that, include: Based on the acquired data of multiple normal performance indicators for each cloud resource pool, the priority of each cloud resource pool is calculated, and the priority is used to characterize the degree of importance that users attach to the cloud resource pool. The fault indicator data of multiple fault performance indicators collected during the fault performance test of the target cloud resource pool are obtained, wherein the target cloud resource pool is one of the cloud resource pools except for the one with the lowest priority. A pre-trained model is used to infer the fault index data to obtain the weights of multiple fault index features included in each fault performance index. The weights are used to characterize the importance of each fault index feature in evaluating the performance of the target cloud resource pool. Based on the weights, select the most important target fault indicator features from each fault performance indicator to obtain the multiple target fault indicator features included in each fault performance indicator. When it is necessary to perform fault performance testing on the remaining cloud resource pool, the fault performance of the remaining cloud resource pool is evaluated based on the obtained index data of the target fault index characteristics. The remaining cloud resource pool is the cloud resource pool other than the target cloud resource pool among the various cloud resource pools.

2. The cloud resource pool performance evaluation method according to claim 1, characterized in that, Based on the acquired performance metrics data of each cloud resource pool, the priority of each cloud resource pool is calculated, including: A pre-trained indicator prediction model is used to infer the indicator data of multiple normal performance indicators for each of the cloud resource pools, and to predict the future indicator data of each of the normal performance indicators for each of the cloud resource pools within a future preset time range; and the indicator data of each of the normal performance indicators are weighted and normalized to obtain the weight of each of the normal performance indicators. Based on the future indicator data and the weights corresponding to the future indicator data, the priority of each cloud resource pool is calculated.

3. The cloud resource pool performance evaluation method according to claim 1 or 2, characterized in that, Before acquiring the fault metric data of multiple fault performance metrics collected during fault performance testing of the target cloud resource pool, the following steps are also included: From the remaining cloud resource pool, select a backup cloud resource pool with a lower priority than the target cloud resource pool; The data stored in the target cloud resource pool is backed up as backup data to the backup cloud resource pool; Once the fault performance test of the target cloud resource pool is completed, the backup data stored in the backup cloud resource pool is used to verify the data stored in the target cloud resource pool.

4. The cloud resource pool performance evaluation method according to claim 1 or 2, characterized in that, If the remaining cloud resource pool has the lowest priority and the latest fault testing order, before performing a fault performance evaluation on the remaining cloud resource pool based on the obtained index data of the target fault index characteristics, the following steps are also included: Obtain the fault performance evaluation results of all cloud resource pools except the remaining cloud resource pools in each cloud resource pool; Based on the fault performance evaluation results, the cloud resource pool with the best evaluation results is selected from the cloud resource pools as the backup cloud resource pool for the remaining cloud resource pools. The data stored in the remaining cloud resource pool is backed up as backup data to the backup cloud resource pool of the remaining cloud resource pool; Once it is confirmed that the fault performance test of the remaining cloud resource pool has been completed, the backup data stored in the backup cloud resources is used to verify the data stored in the remaining cloud resource pool.

5. The cloud resource pool performance evaluation method according to claim 1, characterized in that, Based on the acquired index data of the remaining cloud resource pool regarding the target fault index characteristics, a fault performance evaluation is performed on the remaining cloud resource pool, including: Based on the indicator data of multiple target fault indicator features included in each of the fault performance indicators, an indicator matrix is ​​generated; and based on the weights of the fault indicator features included in each of the fault performance indicators, the weights of each of the fault performance indicators are obtained, and a weight matrix is ​​generated based on the weights of each of the fault performance indicators. Calculate the product of the index matrix and the weight matrix, and use the result of the product as the fault performance evaluation result of the remaining cloud resource pool.

6. The cloud resource pool performance evaluation method according to claim 5, characterized in that, The weights of each fault performance index are obtained based on the weights of the fault index features included in each of the fault performance indices, including: The average value is calculated for the weights of the fault indicator features included in each of the aforementioned fault performance indicators; The result of the mean calculation is used as the weight of each of the fault performance indicators.

7. The cloud resource pool performance evaluation method according to claim 1, characterized in that, The calculation process for the fault performance evaluation result of the target cloud resource pool includes: Based on the collected fault performance index data, an index data matrix is ​​generated; and based on the weight of each fault performance index, a weight matrix is ​​generated. Calculate the product of the indicator data matrix and the weight matrix, and use the result of the product as the performance evaluation result of the target cloud resource pool.

8. The cloud resource pool performance evaluation method according to claim 1, characterized in that, The normal performance indicators include at least one of the following: recent access frequency, recent modification frequency, cloud resource pool storage capacity, and cloud resource pool storage utilization rate. The fault performance indicators include at least one of the following: computing resources, storage resources, network resources, database services, and security and authentication services.

9. A computer cloud resource pool performance evaluation device, characterized in that, include: The calculation module is used to calculate the priority of each cloud resource pool based on the acquired index data of multiple normal performance indicators of each cloud resource pool. The priority is used to characterize the degree of importance that users attach to the cloud resource pool. The acquisition module is used to acquire fault indicator data of multiple fault performance indicators collected when performing fault performance testing on the target cloud resource pool, wherein the target cloud resource pool is one of the cloud resource pools other than the one with the lowest priority. The weight inference module is used to infer the fault index data using a pre-trained model to obtain the weights of multiple fault index features included in each fault performance index. The weights are used to characterize the importance of each fault index feature in evaluating the performance of the target cloud resource pool. The selection module is used to select multiple target fault indicator features with the highest importance from each fault performance indicator based on the weight, thereby obtaining multiple target fault indicator features included in each fault performance indicator. The evaluation module is used to evaluate the fault performance of the remaining cloud resource pool based on the obtained index data of the remaining cloud resource pool in the target fault index characteristics when it is necessary to conduct fault performance testing on the remaining cloud resource pool. The remaining cloud resource pool is the cloud resource pool other than the target cloud resource pool among the various cloud resource pools.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method as described in any one of claims 1-8.

Images