Heterogeneous cloud resource load dynamic analysis and optimization method and system

Abstract

The invention relates to the technical field of heterogeneous cloud resource loads, and discloses a heterogeneous cloud resource load dynamic analysis and optimization method and system, and the method comprises the steps: collecting heterogeneous cloud resource environment data, and carrying out the data analysis; constructing a load prediction model through an improved deep learning algorithm, and dynamically adjusting the prediction model according to real-time data; constructing a multi-objective optimization strategy, and generating an optimal resource adjustment and allocation scheme; and carrying out resource adjustment, and carrying out real-time adjustment on the resource adjustment and distribution scheme. The invention provides a more advanced and efficient resource load dynamic analysis and optimization method which is particularly suitable for resource management and load balancing in a heterogeneous cloud environment, the method can significantly improve the resource use efficiency, reduce the cost and improve the overall service quality, and through intelligent analysis, prediction capability and automatic resource management, the resource management efficiency is greatly improved. The system is ensured to operate efficiently and reliably, and the operation cost is reduced at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of heterogeneous cloud resource load, and in particular to a method and system for dynamic analysis and optimization of heterogeneous cloud resource load. Background Art

[0002] Resource management challenges in heterogeneous cloud environments. With the development of cloud computing technology, enterprises and organizations are increasingly adopting hybrid cloud and multi-cloud architectures to meet their business needs. These complex cloud environments usually contain multiple types of cloud resources such as public clouds, private clouds, and edge computing. However, there are a series of challenges in achieving effective resource management and load balancing in these environments: Complexity of resource allocation: In heterogeneous cloud environments, different types of cloud platforms have different characteristics and resource limitations, which makes resource allocation more complicated; Dynamic changes in workloads: Fluctuations in business demand lead to uncertainty in workloads, and traditional static resource configuration methods often cannot effectively cope with this uncertainty; Difficulty in cost control: In a multi-cloud environment, over- or under-configuration of resources will lead to increased costs, especially when resources need to be migrated or expanded across cloud platforms.

[0003] The limitations of static resource configuration. Traditional resource configuration methods are usually based on preset rules or policies, which will not change once the configuration is completed. This method has the following problems: Resource waste: When the workload is low, the preset resource allocation may lead to excess resources, resulting in waste; Performance bottleneck: When the workload suddenly increases, the fixed resource configuration may not provide enough resources to support the normal operation of the application, resulting in performance bottlenecks; Cost increase: Due to the rigidity of resource allocation, fixed costs must be paid even when so many resources are not needed, which increases the operating costs of the enterprise.

[0004] Lack of intelligent analysis and optimization. Most current resource management solutions lack effective intelligent analysis methods and cannot accurately predict future resource requirements. This leads to the following deficiencies: Prediction accuracy: There is not enough powerful data analysis capabilities to predict future workload requirements, making it difficult to match resource allocation with actual needs; Automatic adjustment capabilities: The lack of automatic resource adjustment capabilities requires administrators to manually intervene to adjust resource allocation, which is not only time-consuming but also prone to errors; Cost optimization: There is not enough mechanism to optimize resource usage, resulting in the inability to effectively control costs.

[0005] Cost control challenges: In a hybrid cloud or multi-cloud environment, over- or under-configuration of resources will lead to unnecessary expenses. In addition, the complexity of resource management also makes cost control more difficult: Over-configuration of resources: In order to ensure high availability and high performance, resources are often over-configured, but this will lead to unnecessary expenses; Under-configuration of resources: If resources are insufficiently configured, it may lead to application performance degradation or even failure, which will also affect business continuity and user experience; Cost prediction and control: The lack of an effective cost prediction and control mechanism makes it difficult for enterprises to accurately estimate future costs and to optimize costs between different cloud platforms; Existing technologies have obvious limitations in resource management and load balancing in heterogeneous cloud environments.

[0006] Traditional static resource configuration methods often cannot effectively respond to changing workload demands, resulting in resource waste and increased costs. Therefore, developing a system that can intelligently analyze and automatically adjust resource allocation is crucial to improving resource utilization efficiency, reducing costs, and improving service quality. Summary of the invention

[0007] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0008] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a method for dynamic analysis and optimization of heterogeneous cloud resource load to solve the above problems.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] In a first aspect, the present invention provides a method for dynamic analysis and optimization of heterogeneous cloud resource loads, comprising:

[0011] Collect heterogeneous cloud resource environment data and perform data analysis;

[0012] Build a load forecasting model through an improved deep learning algorithm and dynamically adjust the forecasting model based on real-time data;

[0013] Based on the environmental data, a multi-objective optimization strategy is constructed to obtain the optimal solution of the load forecasting model under multiple constraints, and the optimal resource adjustment allocation plan is generated by adjusting the solution through reinforcement learning;

[0014] The resource adjustment is performed based on the optimal resource adjustment allocation plan, and the resource adjustment allocation plan is safety verified. The resource status is continuously monitored to adjust the resource adjustment allocation plan in real time.

[0015] As a preferred solution of the method for dynamic analysis and optimization of heterogeneous cloud resource loads described in the present invention, collecting heterogeneous cloud resource environment data and performing data analysis includes:

[0016] The heterogeneous cloud resource environment data is collected through an adaptive collection algorithm, the adaptive collection algorithm automatically adjusts the collection strategy according to the load of the current cloud environment, adjusts the collection frequency, collection granularity and collection content, and builds a message queue model to accelerate the reading and writing operations of messages through memory-mapped files and shared memory mechanisms;

[0017] The data analysis includes analyzing the correlation between data nodes, building a highly correlated data model, and using real-time data analysis based on streaming computing to optimize data processing performance by dynamically adjusting the window size and sliding step size.

[0018] As a preferred solution of the method for dynamic analysis and optimization of heterogeneous cloud resource loads described in the present invention, building a load prediction model includes:

[0019] Dynamically adjusting the learning rate of the load prediction model through an adaptive model adjustment algorithm, wherein the adaptive model adjustment algorithm adjusts model parameters according to changes in real-time data;

[0020] The load forecasting model is constructed by using the improved hybrid CNN-LSTM algorithm. The local features of the time series are extracted by the CNN layer, and the long-term dependencies between these features are learned by the LSTM layer. The load forecasting model is expressed as follows:

[0021] h t =LSTM(h t-1 ,x t )

[0022] y t =CNN(x t )+h t

[0023] Among them, h t represents the hidden state of the LSTM unit at time t, x t Represents the value of the input data at time t, y t Represents the final output.

[0024] As a preferred solution of the method for dynamic analysis and optimization of heterogeneous cloud resource loads described in the present invention, constructing a multi-objective optimization strategy includes:

[0025] Define multiple objective functions f i (x) To deal with multi-objective optimization problems, we set the non-dominated sorting NS k, find multiple non-dominated solution sets and use the crowding distance d i Maintaining population diversity;

[0026] The multiple objective functions f i (x) is expressed as:

[0027]

[0028] Among them, f i (x) represents the i-th objective function, w j represents the weight of the jth variable, x j represents the decision variable;

[0029] The non-dominated sorting NS k It is expressed as:

[0030]

[0031] Among them, NS k represents the non-dominated solution set of the kth level, P represents the current population, S(x) represents the set of all individuals dominated by x, and R(y) represents the number of individuals dominating x;

[0032] The crowding distance d i It is expressed as:

[0033]

[0034] Among them, d i represents the crowding distance of individual i, f ij represents the performance of individual i on target j, Indicates the minimum value of target j, min j and max j They represent the minimum and maximum values ​​on feature j respectively, and m represents the number of features.

[0035] As a preferred solution of the method for dynamic analysis and optimization of heterogeneous cloud resource loads described in the present invention, adjustment by reinforcement learning includes:

[0036] By training the agent to learn the optimal action sequence, the resource allocation plan is automatically adjusted according to the current state and historical behavior, and the optimal action sequence of the resource allocation plan is learned;

[0037] Among them, the optimal action sequence of learning resource allocation schemes is automatically adjusted under changing business needs based on the current state and historical behavior;

[0038] In each step, the agent observes the current state s, takes action a, and updates the Q value based on the immediate reward r(s,a) and the new state s′;

[0039] The experience replay buffer D stores the quadruple of the state, action, reward, and new state that the agent has experienced in the past. It optimizes the learning process through the advantage function and value function, calculates the pros and cons of each action relative to the average behavior of the current state and the value of the state, and guides the update of the Q value.

[0040] As a preferred solution of the method for dynamic analysis and optimization of heterogeneous cloud resource loads described in the present invention, the security verification of the resource adjustment allocation solution includes:

[0041] Dynamically adjust resources according to business load, perform security and feasibility verification before execution, consider load prediction, load type, and resource efficiency factors, calculate the number of containers, and dynamically adjust the number of containers according to real-time load through a custom controller;

[0042] For each state s and action a, define safety constraints and use formal verification to determine whether it is safe to take action a in state s at time t. If it is safe, continue to monitor; if it is not safe, regenerate the optimal resource adjustment allocation plan.

[0043] As a preferred solution of the method for dynamic analysis and optimization of heterogeneous cloud resource loads described in the present invention, the method of continuously monitoring resource status and adjusting the resource adjustment allocation scheme in real time includes:

[0044] Through the feedback regulation mechanism based on the PID controller, the control signal is adjusted according to the deviation between the set value and the actual measured value, and the resource allocation plan is continuously optimized according to the real-time feedback to dynamically balance the resource usage;

[0045] Among them, the PID controller adjusts the control signal through the proportion, integration and differentiation, and dynamically adjusts the differential gain for real-time optimization.

[0046] In a second aspect, the present invention provides a heterogeneous cloud resource load dynamic analysis and optimization system, comprising:

[0047] The collection and analysis module is used to collect heterogeneous cloud resource environment data and perform data analysis;

[0048] Build a model to build a load forecasting model through an improved deep learning algorithm and dynamically adjust the forecasting model based on real-time data;

[0049] A generation module is used to construct a multi-objective optimization strategy based on the environmental data, obtain the optimal solution of the load forecasting model under multiple constraints, and adjust it through reinforcement learning to generate an optimal resource adjustment allocation plan;

[0050] The verification and adjustment module is used to adjust resources based on the optimal resource adjustment allocation plan, perform security verification on the resource adjustment allocation plan, and continuously monitor the resource status to make real-time adjustments to the resource adjustment allocation plan.

[0051] In a third aspect, the present invention provides an electronic device, comprising:

[0052] Memory and processor;

[0053] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for dynamic analysis and optimization of heterogeneous cloud resource loads are implemented.

[0054] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for dynamic analysis and optimization of heterogeneous cloud resource loads.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides a more advanced and efficient resource load dynamic analysis and optimization method, which is particularly suitable for resource management and load balancing in heterogeneous cloud environments. This method can significantly improve resource utilization efficiency, reduce costs, and improve the overall service quality. Through intelligent analysis, predictive capabilities and automated resource management, it ensures that the system can operate efficiently and reliably, while reducing operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0057] Figure 1 A schematic diagram of the overall process of a method for dynamic analysis and optimization of heterogeneous cloud resource loads according to an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of a heterogeneous cloud resource optimization management process flow based on closed-loop control of a heterogeneous cloud resource load dynamic analysis and optimization method according to an embodiment of the present invention;

[0059] Figure 3 A schematic diagram of a process flow of optimizing application resources across geographical regions of a method for dynamically analyzing and optimizing heterogeneous cloud resource loads according to an embodiment of the present invention;

[0060] Figure 4The present invention is a schematic diagram of a resource optimization process flow for intelligently scheduling computing tasks of a method for dynamically analyzing and optimizing heterogeneous cloud resource loads according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0063] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0064] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0065] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0066] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0067] Reference Figure 1-Figure 3 , is an embodiment of the present invention, and provides a method for dynamic analysis and optimization of heterogeneous cloud resource loads, such as Figure 1 As shown, including:

[0068] S101, collect heterogeneous cloud resource environment data and perform data analysis;

[0069] S102, building a load prediction model through an improved deep learning algorithm, and dynamically adjusting the prediction model according to real-time data;

[0070] S103, based on the environmental data, construct a multi-objective optimization strategy, obtain the optimal solution of the load forecasting model under multiple constraints, and adjust it through reinforcement learning to generate the optimal resource adjustment allocation plan;

[0071] S104, adjusting resources based on the optimal resource adjustment allocation plan, performing security verification on the resource adjustment allocation plan, and continuously monitoring resource status to adjust the resource adjustment allocation plan in real time.

[0072] It should be noted that, in this embodiment, Figure 2 As shown in the figure, there is a heterogeneous cloud environment including public cloud and private cloud. By deploying agents on each cloud platform, key indicators such as CPU usage, memory usage, disk I / O, etc. can be collected; the data analysis engine uses this data to identify peak load periods and predict resource requirements in the future through machine learning algorithms; based on the prediction results, the system will generate dynamic adjustment strategies, such as increasing resources in advance before predicting the peak load period; when predicting the trough period, releasing excess resources, thereby achieving cost optimization.

[0073] In this embodiment, if Figure 3 As shown in the figure, we also consider an application deployed across multiple geographical locations. By analyzing the resource usage in different geographical areas, computing tasks can be intelligently scheduled to areas with lower loads to relieve pressure on other areas. In addition, dynamic configuration of the load balancer can be achieved through automated tools to ensure that traffic is reasonably distributed.

[0074] In this embodiment, if Figure 4 As shown, it is also possible to manage a globally distributed geographic region deployment, with the goal of optimizing resource usage by intelligently scheduling computing tasks to ensure that the system can operate efficiently in the face of changing business needs.

[0075] In a preferred embodiment, step S101, collecting heterogeneous cloud resource environment data and performing data analysis includes:

[0076] Step 1: Collect heterogeneous cloud resource environment data through an adaptive collection algorithm. The adaptive collection algorithm automatically adjusts the collection strategy according to the load of the current cloud environment, adjusts the collection frequency, collection granularity and collection content, and builds a message queue model to accelerate the reading and writing of messages through memory-mapped files and shared memory mechanisms.

[0077] Step 2: Data analysis includes analyzing the correlation between data nodes, building a highly correlated data model, and using real-time data analysis based on streaming computing to optimize data processing performance by dynamically adjusting the window size and sliding step size.

[0078] It should be noted that this step proposes a lightweight data collection agent and its deployment method, which can automatically detect and adapt to the characteristics of different cloud environments, reduce the impact on the production system, and deploy lightweight data collection agents on key nodes of each cloud platform to collect resource usage, network traffic, application performance indicators, etc., and ensure real-time data transmission and reduce delays through efficient message queue technology.

[0079] In step 1 of S101, a container-based adaptive collection agent is proposed, which can automatically adjust the collection frequency, collection granularity and collection content according to the runtime environment, and an adaptive collection algorithm is introduced, which can automatically adjust the collection strategy according to the load of the current cloud environment to minimize interference with the production system.

[0080] In an optional implementation, in order to realize the function of the adaptive collection agent, an adaptive collection algorithm is proposed, which can automatically adjust the collection frequency according to the load of the current cloud environment to minimize the interference with the production system. The formula is as follows:

[0081]

[0082] Among them, Sampling Rate is the adjusted sampling rate, LoadAverage is the average system load, Max LoadThreshold is the maximum allowable load threshold, and Base Sampling Rate is the base sampling rate.

[0083] When the system load is high, the Load Average approaches or exceeds the Max Load Threshold. When the ratio is close to or equal to 1, the sampling rate f (Sampling Rate) will be close to the smaller value of the Base Sampling Rate, thereby reducing the acquisition frequency and alleviating interference to the system; when the system load is low, the Load Average is much lower than the Max Load Threshold. If the ratio is small, the sampling rate f (Sampling Rate) will be close to the larger value of the Base Sampling Rate, thereby increasing the acquisition frequency to capture the system behavior more finely.

[0084] The above formula reflects the relationship between the sampling rate and the average system load. When the system load is high, the sampling rate will automatically decrease to reduce interference to the system. Conversely, when the system load is low, the sampling rate will increase to capture the system behavior more finely.

[0085] Among them, Sampling Rate: indicates the frequency at which the collection agent collects data. The higher the sampling rate, the denser the collected data, but the interference to the system may also be greater.

[0086] Load Average: Indicates the average level of the current system load. The higher the load, the more tasks or requests the system is processing.

[0087] Max Load Threshold: This is a preset value that defines the maximum load level that the system can withstand. When the system load approaches or exceeds this threshold, the sampling rate should be reduced to reduce the system burden.

[0088] Base Sampling Rate: The default sampling rate of the acquisition agent. When the system load is low, the acquisition frequency will be close to or equal to this base value.

[0089] It should be noted that in this way, the collection agent can minimize the interference with the production system while ensuring the quality of data collection. This adaptive collection algorithm is very suitable for dynamically changing cloud environments because it can automatically adjust the collection strategy according to the actual load conditions to ensure the efficiency and accuracy of data collection.

[0090] By implementing resource isolation and adaptive scheduling at the container level, we ensure that the collection agent can run efficiently without interfering with the production system. The algorithm ensures that when the system load is high, the collection frequency will be reduced accordingly, thereby reducing the impact of the collection process on system performance. On the contrary, when the load is low, the collection frequency will be increased to capture the system behavior more finely.

[0091] In an optional implementation, a message queue model is constructed by using efficient message queue technology. An efficient message queue model based on zero-copy technology is used to accelerate the reading and writing operations of messages by using memory-mapped files and shared memory mechanisms. A zero-copy message passing algorithm is used, which can minimize the number of copies of data from disk to memory and then to the network, thereby significantly improving the throughput of the message queue. The zero-copy message passing algorithm is expressed as:

[0092]

[0093] Among them, T transfer is the data transmission time, Data Size is the data size, Bandwidth is the bandwidth, Overhead is the additional overhead, and CPU Speed ​​is the CPU speed.

[0094] This formula is used to calculate the total time it takes to transfer data from disk to memory and then to the network. By using an efficient message queue model based on zero-copy technology, the data transmission time is significantly shortened. This model uses memory-mapped files and shared memory mechanisms to reduce the number of times data is copied between user mode and kernel mode, thereby improving the efficiency of data transmission.

[0095] Data transmission time T transfcr It is mainly affected by data size and bandwidth. The larger the data size, the longer the transmission time; the larger the bandwidth, the shorter the transmission time; overhead will also increase the transmission time, but this time is related to the CPU speed. The faster the CPU speed, the shorter the time required to process the overhead and the shorter the transmission time.

[0096] Then use memory-mapped files and shared memory advanced functions to achieve efficient data transmission and processing. By using memory-mapped files and shared memory mechanisms, data can be mapped directly from the disk to the process space, thereby eliminating the copying of data between user state and kernel state, reducing unnecessary CPU usage, and thus improving overall system performance.

[0097] It should be noted that by combining memory-mapped files and shared memory mechanisms, true zero-copy technology is implemented, which greatly improves the efficiency of data transmission. The formula comprehensively considers multiple factors such as data size, bandwidth, additional overhead and CPU speed, and provides a comprehensive method for calculating data transmission time. The model can flexibly configure parameters according to different application scenarios to achieve the best data transmission effect.

[0098] In an optional implementation, in step 2, building a highly correlated data model includes:

[0099] The unified data modeling algorithm based on the graph database can adaptively process data from different cloud platforms and build a highly connected data model through the complex network theory in graph theory, which can be expressed as:

[0100]

[0101] Among them, C(i,j) represents the correlation coefficient between nodes i and j, Γ(i) and Γ(j) represent the neighbor sets of nodes i and j respectively, and |·| represents the number of elements in the set.

[0102] This formula is used to calculate the correlation coefficient C(i,j) between node i and node j, which is an indicator to measure the degree of association between two nodes. By calculating the correlation coefficient, we can understand the similarity and association between different nodes, and thus build a highly correlated data model.

[0103] Among them, C(i,j) (correlation coefficient): represents the correlation coefficient between node i and node j. The value range of this coefficient is usually between [0,1]. The larger the value, the higher the degree of association between the nodes.

[0104] Γ(i) and Γ(j) (neighbor sets): represent the neighbor sets of node i and node j respectively. The neighbor set refers to the set of other nodes connected to a given node.

[0105] |·|(number of elements in a set): represents the number of elements in a set. In this formula, |Γ(i)| and |Γ(j)| represent the number of elements in the neighbor sets of node i and node j, respectively.

[0106] The calculation of the correlation coefficient C(i,j) is based on the neighbor sets Γ(i) and Γ(j) of nodes i and j. The numerator |Γ(i)∩Γ(j)| in the calculation formula represents the number of common neighbors of the two nodes, that is, the number of elements in the intersection of the two sets. The denominator It is the geometric mean of the number of elements in the two sets, which is used to standardize the value of the numerator. The calculated correlation coefficient C(i,j) not only takes into account the number of common neighbors, but also the size of the node's own neighbor set.

[0107] It should be noted that through graph databases and complex network theory, we can efficiently store and query multi-source data. The algorithm measures the degree of association between two nodes by calculating the correlation coefficient between them. The higher the correlation coefficient, the closer the connection between the two nodes in the graph, which helps to build a more accurate data model.

[0108] In an optional implementation, step 2 of S101 uses real-time data analysis based on streaming computing, multi-source data fusion technology, and uses window functions and event-driven architecture to achieve efficient processing of data streams. The algorithm can optimize data processing performance by dynamically adjusting the window size and sliding step size. By comprehensively considering the data arrival rate, processing delay and required accuracy, the algorithm can automatically adjust the window size and sliding step size to achieve efficient processing of data streams. The algorithm finds the balance between processing efficiency and accuracy by calculating the optimal window size and sliding step size, thereby improving the overall performance of data stream processing. The design of this method can adapt well to data streams of different sizes and types, and can efficiently process both small-scale and large-scale data.

[0109] The multi-source data fusion algorithm based on streaming computing is expressed as:

[0110]

[0111] Among them, W optimal is the optimal window size, S optimal is the optimal sliding step size, DataRate is the data arrival rate, ProcessingLatency is the processing delay, and Required Precision is the required accuracy.

[0112] This formula is used to calculate the optimal window size W given the data arrival rate, processing delay, and required accuracy. optimal and the optimal sliding step size S optimal By optimizing the window size and sliding step size, the processing efficiency of the data stream can be improved and the delay can be reduced. The optimal window size W optimal It is determined by calculating the product of the data arrival rate and the processing delay, and then dividing it by the required accuracy; a larger window size can reduce the number of calculations but may sacrifice accuracy, and a smaller window size can improve accuracy but may increase computational overhead. optimal By setting the optimal window size W optimal Determined by dividing by 2, the choice of sliding step size affects the processing density and calculation frequency of the data stream.

[0113] It should be noted that this embodiment uses the window function and event-driven architecture of the Apache Flink streaming computing framework to achieve real-time processing and fusion of multi-source data streams, thereby achieving higher processing efficiency and lower latency. The algorithm achieves efficient processing of data streams by calculating the optimal window size and sliding step size - a larger window size can reduce the number of calculations, but may sacrifice accuracy; a smaller window size can improve accuracy, but may increase computing overhead. By comprehensively considering the data rate, processing latency, and required accuracy, we find a balance point to achieve the best data processing performance.

[0114] The data model constructed in this step stores data collected from different cloud platforms in a standardized manner, and uses big data analysis technology and machine learning algorithms to identify resource usage patterns and abnormal behaviors. It adopts a cross-platform unified data model construction method to achieve effective integration and standardized processing of data from different sources.

[0115] In a preferred embodiment, in step S102, building a load prediction model includes:

[0116] Dynamically adjust the learning rate of the load forecasting model through an adaptive model adjustment algorithm, which adjusts model parameters according to changes in real-time data;

[0117] The load forecasting model is constructed by using the improved hybrid CNN-LSTM algorithm. The local features of the time series are extracted through the CNN layer, and the long-term dependencies between these features are learned through the LSTM layer. The load forecasting model is expressed as:

[0118] h t =LSTM(h t-1 ,x t )

[0119] y t =CNN(x t )+h t

[0120] Among them, h t represents the hidden state of the LSTM unit at time t, x t Represents the value of the input data at time t, y t Represents the final output.

[0121] It should be noted that the dynamic load prediction model based on deep learning in this step selects a suitable prediction model according to business characteristics and historical data, can dynamically adjust the prediction model according to real-time data, and regularly adjust the model parameters according to the actual prediction error to improve the prediction accuracy.

[0122] In an optional implementation, an adaptive model adjustment algorithm based on an online gradient descent method is used, which can quickly adjust model parameters according to changes in real-time data, so that the update efficiency of model parameters can be improved by dynamically adjusting the learning rate to improve prediction accuracy, which is expressed as:

[0123] θ( t+1 )=θ( t )-η t ·g( t)

[0124] Among them, θ (t) is the model parameter at step t, η t is the learning rate at step t, g (t) is the gradient of step t, θ (t+1) represents the model parameters at the t+1th step (or iteration).

[0125] Dynamic learning rate adjustment η t It is expressed as:

[0126]

[0127] Among them, η0 is the initial learning rate, α is the learning rate decay factor, t is the number of iterations, and β represents the basic factor of learning rate decay.

[0128] It should be noted that this formula is used to quickly adjust the model parameters according to the changes in real-time data to improve the prediction accuracy. Through the online gradient descent method, the model parameters can be adjusted according to the changes in real-time data to better adapt to the changing business needs; the learning rate η t The dynamic adjustment of helps the model converge quickly and avoid overfitting. As the number of iterations increases, the learning rate gradually decreases, which helps the model finally converge to a stable solution. The learning rate attenuation factor α is dynamically adjusted according to the number of iterations t to further optimize the learning rate attenuation process.

[0129] Specifically, the first step is to calculate the current model parameters θ (t) , learning rate η t and the gradient g (t) To update the model parameters, θ (t+1) =θ (t) -η t ·g (t) ;

[0130] The second step is to adjust the learning rate according to the initial learning rate η0, the learning rate decay factor α and the number of iterations t;

[0131] The third step is to dynamically adjust the learning rate decay factor α according to the number of iterations t and the base factor β.

[0132] Through the online gradient descent method, model parameters can be quickly adjusted according to changes in real-time data, so that the model can better adapt to changing business needs. The algorithm dynamically adjusts the learning rate to adapt to the speed of data changes, thereby ensuring rapid convergence of the model while avoiding overfitting. As the number of iterations increases, the learning rate gradually decreases, which helps the model eventually converge to a stable solution.

[0133] Specifically, a load forecasting model is constructed and a hybrid CNN-LSTM model is designed. The model extracts local features of the time series through the CNN layer, and then learns the long-term dependencies between these features through the LSTM layer. The model can efficiently process time series data while capturing short-term and long-term dependencies to achieve accurate predictions. This formula is used to describe the prediction process of a hybrid CNN-LSTM model. The model first extracts local features of the time series data through the CNN layer, and then learns the long-term dependencies between these features through the LSTM layer. This structure can efficiently process time series data while capturing short-term and long-term dependencies to achieve accurate predictions.

[0134] The first step is to calculate the hidden state h through the LSTM unit t , that is, h t =LSTM(h t-1 ,x t ), this step uses the characteristics of LSTM units to capture long-term dependencies in time series;

[0135] The second step is to extract the input data x through the CNN layer t These features are then combined with the hidden state h of the LSTM unit t Combined to form the final output y t , that is, y t =CNN(x t )+h t .

[0136] In a preferred embodiment, in step S103, constructing a multi-objective optimization strategy includes:

[0137] Define multiple objective functions f i (x) To deal with multi-objective optimization problems, we set the non-dominated sorting NS k , find multiple non-dominated solution sets and use the crowding distance d i Maintaining population diversity;

[0138] Multiple objective functions f i (x) is expressed as:

[0139]

[0140] Among them, f i (x) represents the i-th objective function, w j represents the weight of the jth variable, x j represents the decision variable;

[0141] Non-dominated sorting NS k It is expressed as:

[0142]

[0143] Among them, NS k represents the non-dominated solution set of the kth level, P represents the current population, S(x) represents the set of all individuals dominated by x, and R(y) represents the number of individuals dominating x;

[0144] Crowding distance d i It is expressed as:

[0145]

[0146] Among them, d i represents the crowding distance of individual i, f ij represents the performance of individual i on target j, Indicates the minimum value of target j, min j and max j They represent the minimum and maximum values ​​on feature j respectively, and m represents the number of features.

[0147] It should be noted that this embodiment designs a multi-objective optimization resource allocation strategy generation algorithm, which can find the optimal solution under multiple constraints, and automatically adjust the resource allocation strategy using technologies such as reinforcement learning to adapt to changing business needs. Multi-objective optimization combines a multi-objective genetic algorithm that maintains population diversity and a non-dominated sorting strategy, and proposes an improved NSGA-II algorithm that can more efficiently handle complex multi-objective optimization problems and find the optimal solution under multiple constraints. It is used to describe an improved NSGA-II algorithm that combines a population diversity maintenance and a non-dominated sorting strategy, which can more efficiently handle complex multi-objective optimization problems and find the optimal solution under multiple constraints. By defining multiple objective functions f i (x), the algorithm can handle multi-objective optimization problems by non-dominated sorting NS k , the algorithm can find multiple non-dominated solution sets and pass the crowding distance d i to maintain population diversity.

[0148] Specifically, the first step is to define multiple objective functions f i (x) to describe the multi-objective optimization problem. The second step is to use non-dominated sorting NS kTo find multiple non-dominated solution sets, these solution sets contain the optimal solution in the current population P. The third step is to calculate the crowding distance d of each individual i To maintain the diversity of the population and thus find a balance between multiple goals.

[0149] In a preferred embodiment, in step S103, adjusting by reinforcement learning includes:

[0150] By training the agent to learn the optimal action sequence, the resource allocation plan is automatically adjusted according to the current state and historical behavior, and the optimal action sequence of the resource allocation plan is learned;

[0151] Among them, the optimal action sequence of learning resource allocation schemes is automatically adjusted under changing business needs based on the current state and historical behavior;

[0152] In each step, the agent observes the current state s, takes action a, and updates the Q value based on the immediate reward r(s,a) and the new state s′;

[0153] The experience replay buffer D stores the quadruple of the state, action, reward, and new state that the agent has experienced in the past. It optimizes the learning process through the advantage function and value function, calculates the pros and cons of each action relative to the average behavior of the current state and the value of the state, and guides the update of the Q value.

[0154] It should be noted that by proposing a reinforcement learning based on deep Q network (DQN), using DQN to train the intelligent agent to learn the optimal action sequence, the resource allocation strategy can be automatically adjusted according to the current state and historical behavior, which is used to learn the optimal action sequence of the resource allocation strategy and realize the automatic adjustment of the resource allocation strategy.

[0155] The reinforcement learning algorithm is expressed as:

[0156]

[0157] Where Q(s,a) represents the expected return of taking action a in state s, s represents the current state, a represents the action taken in the current state s, r(s,a) represents the immediate reward after taking action a in state s, γ is a discount factor used to measure the importance of future rewards, 0<γ≤1, s′ represents the new state after taking action a, a′ represents any action among all actions that can be taken in the new state s′, max a′ Q(s′,a′) represents the Q value of the action a′ that can bring the highest expected benefit among all possible actions a′ in the new state s′.

[0158] Among them, the max in the formula a′Q(s′,a′) represents the action that can bring the highest expected benefit among all possible actions a′ in the new state s′. This maximum value is determined by comparing the Q values ​​of all possible actions a′ in the new state s′, and the action a′ with the largest Q value is selected as the best action for the next step.

[0159] Experience replay is expressed as:

[0160] D={(s,a,r(s,a),s')}

[0161] The Q value update is expressed as:

[0162]

[0163] Here, α is the learning rate.

[0164] It should be noted that the above process describes a reinforcement learning method based on a deep Q network (DQN), which can learn the optimal action sequence of the resource allocation strategy based on the current state and historical behavior, thereby achieving automatic adjustment under changing business needs. Reinforcement learning optimizes strategies through continuous trial and learning to maximize long-term benefits. In each step, the agent observes the current state s, takes action a, and updates the Q value based on the immediate reward r(s,a) and the new state s′. The experience replay buffer D stores the quadruple of state, action, reward, and new state experienced by the agent in the past, which is used to improve the stability and efficiency of learning. Q value update formula ΔQ(s,a)=α[r(s,a)+γmax a′ Q(s′,a′)-Q(s,a)] describes how to update the Q value based on the current experience data. The introduction of the advantage function Advantage(s,a) and the value function V(s) aims to further optimize the learning process by calculating the pros and cons of each action relative to the average behavior of the current state and the value of the state to more accurately guide the update of the Q value.

[0165] Specifically, the first step is to update the Q value based on the immediate reward and the new state, that is, ΔQ(s,a)=α[r(s,a)+γmax a′ Q(s',a')-Q(s,a)];

[0166] The second step is to use the advantage function Advantage(s,a) and the value function V(s) to further optimize the update of the Q value, that is, ΔQ(s,a)=α[Advantage(s,a)+γmax′ a Q(s′,a′)].

[0167] In a preferred implementation, in step S104, performing security verification on the resource adjustment allocation scheme includes:

[0168] Dynamically adjust resources according to business load, perform security and feasibility verification before execution, consider load prediction, load type, and resource efficiency factors, calculate the number of containers, and dynamically adjust the number of containers according to real-time load through a custom controller;

[0169] For each state s and action a, define safety constraints and use formal verification to determine whether it is safe to take action a in state s at time t. If it is safe, continue to monitor; if it is not safe, regenerate the optimal resource adjustment allocation plan.

[0170] It should be noted that automatic scaling of resources is achieved by directly interacting with the cloud service provider through the API interface, and a security verification method for automatic scaling is adopted to conduct security and feasibility verification before any resource adjustment is performed. This allows the impact of resource adjustments on the business to be evaluated before the adjustment to ensure that the operation will not affect existing business.

[0171] In an optional implementation, an enhanced automatic expansion and contraction system is proposed based on the automatic scaling mechanism of Kubernetes. A custom controller is used to implement specific expansion logic, dynamically adjust resources according to business load, and perform security and feasibility verification before execution. Factors such as load prediction, load type (such as CPU intensive or I / O intensive), and resource utilization efficiency are introduced. The formula for calculating the number of containers is:

[0172]

[0173] Where C(t) represents the number of containers at time t, L(t) represents the actual load at time t, T represents the average load that each container can handle, and C min and C max are the minimum and maximum container quantity limits, P(t) represents the predicted value of the load at time t, and E represents the resource utilization efficiency factor W represents the load weight, and different values ​​are set according to different types of loads (such as CPU intensive, I / O intensive, etc.). D represents the resource allocation delay factor, which is used to reflect the time difference from request to completion of resource allocation.

[0174] The key points include the following:

[0175] Load prediction P(t), considering that load prediction can help better prepare resources and avoid service interruptions caused by sudden peaks;

[0176] Resource efficiency E. Different applications may have different resource efficiency. More efficient programs can handle more loads with the same resources.

[0177] Load weight W. Different types of loads may require different types of resource support. For example, CPU-intensive loads and I / O-intensive loads have different resource requirements.

[0178] The resource allocation delay factor D takes into account the possible delay between the detection of load changes and the actual allocation of resources.

[0179] It should be noted that this embodiment combines multiple factors such as actual load, predicted load, resource utilization efficiency, load type and resource allocation delay, and more comprehensively reflects the strategy for adjusting the number of containers. Through a custom controller, we can dynamically adjust the number of containers C(t) according to the real-time load L(t) to ensure resource utilization while avoiding resource waste.

[0180] By adopting a formally verified automatic scaling security verification method and using model checking technology to verify the security of resource adjustment policies, this method can evaluate the impact of resource adjustments on the business before they occur, ensuring that no unexpected results occur.

[0181] First, we define the symbols:

[0182] S: state space, that is, the set of all possible states of the system.

[0183] A: Action space, which is the set of all actions the system can take.

[0184] M(s,a,t): The condition that all safety constraints are satisfied at time t when action a is taken in state s.

[0185] P(s,a,t): Business performance indicator at time t when action a is taken in state s.

[0186] (s,a,t): The i-th performance metric at time t when taking action a in state s.

[0187] The target value of the i-th performance indicator.

[0188] ∈: Maximum allowed error ratio.

[0189] V: The set of valid state-action pairs.

[0190] U(s,a,t): The uncertainty factor at time t when taking action a in state s, used to measure the degree of uncertainty.

[0191] R(s,a,t): Risk factor at time t when taking action a in state s, used to measure the risk level of taking the action.

[0192] F(s,a,t): The future predictor at time t when taking action a in state s, used to measure the impact on future states.

[0193] Safety Constraints,For each state s and action a, we define the safety constraints as follows:

[0194]

[0195] Among them, η is the threshold of the uncertainty factor, and ρ is the threshold of the risk factor.

[0196] The following formal verification is used to ensure that it is safe to take action a in state s at time t, expressed as:

[0197]

[0198] Among them, θ is the minimum value of the future prediction factor, which is used to ensure that the action taken will not have an adverse impact on the future state. The future prediction factor F(s,a,t) can be calculated based on historical data and the prediction model, which indicates the degree of impact on the future state after taking action a. The uncertainty factor U(s,a,t) can be estimated based on the probability distribution of the current system state and the changes in the external environment. The risk factor R(s,a,t) can be estimated based on the probability distribution of the negative impact that may occur after taking action a.

[0199] It should be noted that through the above process we build a comprehensive formal verification framework to ensure the security and feasibility of resource adjustment strategies, help us evaluate the impact on the business before adjusting resources, and ensure that no unexpected results will occur.

[0200] In a preferred implementation, in step S104, continuously monitoring the resource status and adjusting the resource adjustment allocation scheme in real time includes:

[0201] Through the feedback regulation mechanism based on the PID controller, the control signal is adjusted according to the deviation between the set value and the actual measured value, and the resource allocation plan is continuously optimized according to the real-time feedback to dynamically balance the resource usage;

[0202] Among them, the PID controller adjusts the control signal through the proportion, integration and differentiation, and dynamically adjusts the differential gain for real-time optimization.

[0203] It should be noted that by designing a resource optimization adjustment algorithm under closed-loop control (PID), a closed-loop control system is established to continuously monitor the resource status and adjust the strategy based on actual effect feedback. The full name of the PID controller is "Proportional-Integral-Derivative Controller", which is a proportional-integral-derivative controller. It is used to adjust the control signal according to the deviation between the set value and the actual measured value, thereby achieving stable control of the system. This embodiment uses a feedback adjustment mechanism based on the PID controller to continuously optimize the resource allocation strategy according to real-time feedback to achieve a dynamic balance in resource use.

[0204] The mathematical expression of PID controller is:

[0205]

[0206] Among them, u(t) represents the control signal at time t, which is the signal output by the PID controller and is used to adjust the behavior of the controlled object so that it reaches the set target value. e(t) represents the error at time t, that is, the difference between the set value and the actual value. The error e(t) is usually defined as e(t) = r(t) - y(t), where r(t) is the set value (or reference value), y(t) is the actual value (or measured value), and K p Represents proportional gain. The role of the proportional term is to adjust the control signal proportionally according to the current error. The larger the proportional gain, the stronger the response of the control signal to the error. i Integral Gain represents the integral gain. The function of the integral term is to eliminate the steady-state error. It is adjusted according to the accumulation of the error over time. The integral term accumulates the change of the error over time, thereby gradually adjusting the control signal until the steady-state error is zero. K d It represents the derivative gain. The role of the derivative term is to improve the response speed and stability of the system. It is adjusted according to the rate of change of the error. The derivative term can help the controller predict the changing trend of the error and respond in advance.

[0207] Integral Item: represents the integral of the error e(τ) from the initial time 0 to the current time t, which means that the controller will accumulate the change of the error over time to gradually eliminate the steady-state error;

[0208] Differential term: It represents the derivative of the error e(t) with respect to time t, that is, the rate of change of the error, which can help the controller predict the changing trend of the error and respond in advance.

[0209] Through the combination of the above items, the PID controller can effectively adjust the control signal u(t) to achieve the control purpose. The PID controller adjusts the control signal through these three different items (proportional, integral, and differential), thereby achieving effective control of the controlled object.

[0210] The error calculation is expressed as:

[0211] e(t)=r(t)-y(t)

[0212] Where r(t) is the set value at time t and y(t) is the actual value at time t.

[0213] The dynamic adjustment of the differential gain is expressed as:

[0214]

[0215] in, is the basic differential gain, α is the differential gain adjustment coefficient, and Δt is the sampling time interval.

[0216] With the PID controller, we are able to adjust the control signal based on real-time feedback to achieve a dynamic balance in resource usage, and by dynamically adjusting the derivative gain, we are able to better adapt to rapidly changing business needs.

[0217] It should be noted that, compared with the prior art, the present invention has the following beneficial effects:

[0218] (1) Resource utilization rate significantly improved

[0219] Through intelligent analysis and dynamic adjustment, we ensure that resources are always in the best use state, avoid idle and over-configuration of resources, realize dynamic optimization of resource configuration, and improve resource utilization.

[0220] (2) Significant cost reduction

[0221] Avoiding idle resources and over-configuration reduces the total cost of cloud services. Through intelligent analysis and optimization algorithms, while ensuring service quality, it minimizes resource usage costs and improves economic benefits.

[0222] (3) Stable and reliable performance

[0223] The optimized resource allocation can better support business needs, improve user experience, and increase the overall performance and response speed of the system.

[0224] (4) Heterogeneous cloud compatibility

[0225] Support for multiple cloud platforms, including AWS, Azure, Alibaba Cloud, etc., enhances the system's compatibility and applicability, and can achieve effective resource management and load balancing in hybrid cloud or multi-cloud environments.

[0226] (5) Intelligent prediction capabilities

[0227] Advanced machine learning algorithms are used to predict resource requirements, which improves prediction accuracy. Through the prediction algorithm, resources can be scheduled in advance based on the prediction results to avoid resource bottlenecks and performance degradation.

[0228] (6) Flexible optimization strategies

[0229] Optimization strategies can be customized according to different business scenarios to achieve personalized management. The multi-objective optimization algorithm can find a balance between multiple goals to ensure that resource allocation is both efficient and economical.

[0230] (7) Automated execution mechanism

[0231] It reduces human intervention, improves resource management efficiency, realizes the automated execution of resource adjustment strategies, and ensures the security and feasibility of resource adjustment.

[0232] (8) Continuous improvement mechanism

[0233] Through feedback loops, the prediction model and adjustment strategy are continuously optimized, and closed-loop control mechanisms such as PID controllers are used to adjust resource allocation strategies based on real-time monitoring data to maintain the stability of system performance.

[0234] In summary, the present invention provides a more advanced and efficient resource load dynamic analysis and optimization method, which is particularly suitable for resource management and load balancing in heterogeneous cloud environments. This method can significantly improve resource utilization efficiency, reduce costs, and improve the overall service quality. Through intelligent analysis, predictive capabilities and automated resource management, it ensures that the system can run efficiently and reliably while reducing operating costs.

[0235] The above is a schematic scheme of a method for dynamic analysis and optimization of heterogeneous cloud resource loads in this embodiment. It should be noted that the technical scheme of the system for dynamic analysis and optimization of heterogeneous cloud resource loads and the technical scheme of the above-mentioned method for dynamic analysis and optimization of heterogeneous cloud resource loads belong to the same concept. For details not described in detail in the technical scheme of the system for dynamic analysis and optimization of heterogeneous cloud resource loads in this embodiment, please refer to the description of the technical scheme of the above-mentioned method for dynamic analysis and optimization of heterogeneous cloud resource loads.

[0236] The heterogeneous cloud resource load dynamic analysis and optimization system in this embodiment includes:

[0237] The collection and analysis module is used to collect heterogeneous cloud resource environment data and perform data analysis;

[0238] Build a model to build a load forecasting model through an improved deep learning algorithm and dynamically adjust the forecasting model based on real-time data;

[0239] The generation module is used to build a multi-objective optimization strategy based on environmental data, obtain the optimal solution of the load forecasting model under multiple constraints, and adjust it through reinforcement learning to generate the optimal resource adjustment allocation plan;

[0240] The verification and adjustment module is used to adjust resources based on the optimal resource adjustment allocation plan, perform security verification on the resource adjustment allocation plan, continuously monitor the resource status and make real-time adjustments to the resource adjustment allocation plan.

[0241] This embodiment further provides an electronic device, which is applicable to the case of dynamic analysis and optimization of heterogeneous cloud resource loads, including:

[0242] Memory and processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the method for dynamic analysis and optimization of heterogeneous cloud resource loads as proposed in the above embodiment.

[0243] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for dynamically analyzing and optimizing heterogeneous cloud resource loads as proposed in the above embodiment is implemented.

[0244] The storage medium proposed in this embodiment and the method for implementing dynamic analysis and optimization of heterogeneous cloud resource load proposed in the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0245] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ReadOnly, Memory, ROM), random access memory (RandomAccess Memory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform the methods of various embodiments of the present invention.

[0246] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for dynamic analysis and optimization of heterogeneous cloud resource load, characterized in that: include: Collect heterogeneous cloud resource environment data and perform data analysis; Build a load forecasting model through an improved deep learning algorithm and dynamically adjust the forecasting model based on real-time data; Based on the environmental data, a multi-objective optimization strategy is constructed to obtain the optimal solution of the load forecasting model under multiple constraints, and the optimal resource adjustment allocation plan is generated by adjusting the solution through reinforcement learning; The resource adjustment is performed based on the optimal resource adjustment allocation plan, and the resource adjustment allocation plan is safety verified. The resource status is continuously monitored to adjust the resource adjustment allocation plan in real time.

2. The method for dynamic analysis and optimization of heterogeneous cloud resource loads according to claim 1, characterized in that: Collect heterogeneous cloud resource environment data and perform data analysis including: The heterogeneous cloud resource environment data is collected through an adaptive collection algorithm, the adaptive collection algorithm automatically adjusts the collection strategy according to the load of the current cloud environment, adjusts the collection frequency, collection granularity and collection content, and builds a message queue model to accelerate the reading and writing operations of messages through memory-mapped files and shared memory mechanisms; The data analysis includes analyzing the correlation between data nodes, building a highly correlated data model, and using real-time data analysis based on streaming computing to optimize data processing performance by dynamically adjusting the window size and sliding step size.

3. The method for dynamic analysis and optimization of heterogeneous cloud resource load according to claim 1 or 2, characterized in that: Building a load forecasting model includes: Dynamically adjusting the learning rate of the load prediction model through an adaptive model adjustment algorithm, wherein the adaptive model adjustment algorithm adjusts model parameters according to changes in real-time data; The load forecasting model is constructed by using the improved hybrid CNN-LSTM algorithm. The local features of the time series are extracted by the CNN layer, and the long-term dependencies between these features are learned by the LSTM layer. The load forecasting model is expressed as follows: h t =LSTM(h t-1 ,x t ) y t =CNN(x t )+h t Among them, h t represents the hidden state of the LSTM unit at time t, x t Represents the value of the input data at time t, y t Represents the final output.

4. The method for dynamic analysis and optimization of heterogeneous cloud resource loads according to claim 3, characterized in that: Building a multi-objective optimization strategy includes: Define multiple objective functions f i (x) To deal with multi-objective optimization problems, we set the non-dominated sorting NS k , find multiple non-dominated solution sets and use the crowding distance d i Maintaining population diversity; The multiple objective functions f i (x) is expressed as: Among them, f i (x) represents the i-th objective function, wj represents the weight of the j-th variable, and xj represents the decision variable; The non-dominated sorting NS k It is expressed as: Among them, NS k represents the non-dominated solution set of the kth level, P represents the current population, S(x) represents the set of all individuals dominated by x, and R(y) represents the number of individuals dominating x; The crowding distance d i It is expressed as: Among them, d i represents the crowding distance of individual i, f ij represents the performance of individual i on target j, Indicates the minimum value of target j, min j and max j They represent the minimum and maximum values ​​on feature j respectively, and m represents the number of features.

5. The method for dynamic analysis and optimization of heterogeneous cloud resource loads according to claim 4, characterized in that: Adjustments through reinforcement learning include: By training the agent to learn the optimal action sequence, the resource allocation plan is automatically adjusted according to the current state and historical behavior, and the optimal action sequence of the resource allocation plan is learned; Among them, the optimal action sequence of learning resource allocation schemes is automatically adjusted under changing business needs based on the current state and historical behavior; In each step, the agent observes the current state s, takes action a, and updates the Q value based on the immediate reward r(s, a) and the new state s′; The experience replay buffer D stores the quadruple of the state, action, reward, and new state that the agent has experienced in the past. It optimizes the learning process through the advantage function and value function, calculates the pros and cons of each action relative to the average behavior of the current state and the value of the state, and guides the update of the Q value.

6. The method for dynamic analysis and optimization of heterogeneous cloud resource loads according to claim 1 or 5, characterized in that: Security verification of resource adjustment and allocation plans includes: Dynamically adjust resources according to business load, perform security and feasibility verification before execution, consider load prediction, load type, and resource efficiency factors, calculate the number of containers, and dynamically adjust the number of containers according to real-time load through a custom controller; For each state s and action a, define safety constraints and use formal verification to determine whether it is safe to take action a in state s at time t. If it is safe, continue to monitor; if it is not safe, regenerate the optimal resource adjustment allocation plan.

7. The method for dynamic analysis and optimization of heterogeneous cloud resource loads according to claim 6, characterized in that: Continuously monitoring the resource status and adjusting the resource adjustment allocation plan in real time includes: Through the feedback regulation mechanism based on the PID controller, the control signal is adjusted according to the deviation between the set value and the actual measured value, and the resource allocation plan is continuously optimized according to the real-time feedback to dynamically balance the resource usage; Among them, the PID controller adjusts the control signal through the proportion, integration and differentiation, and dynamically adjusts the differential gain for real-time optimization.

8. A dynamic analysis and optimization system for heterogeneous cloud resource load, characterized in that: include, The collection and analysis module is used to collect heterogeneous cloud resource environment data and perform data analysis; Build a model to build a load forecasting model through an improved deep learning algorithm and dynamically adjust the forecasting model based on real-time data; A generation module is used to construct a multi-objective optimization strategy based on the environmental data, obtain the optimal solution of the load forecasting model under multiple constraints, and adjust it through reinforcement learning to generate an optimal resource adjustment allocation plan; The verification and adjustment module is used to adjust resources based on the optimal resource adjustment allocation plan, perform security verification on the resource adjustment allocation plan, and continuously monitor the resource status to make real-time adjustments to the resource adjustment allocation plan.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for dynamic analysis and optimization of heterogeneous cloud resource loads described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the method for dynamic analysis and optimization of heterogeneous cloud resource loads as described in any one of claims 1 to 7.

Images