Python monitoring task resource usage method and system

Abstract

The present invention provides a Python monitoring task resource usage method and system, which relates to the technical field of intelligent monitoring and resource optimization, including extracting process call stack information through a multi-layer convolutional neural network, building a process dependency graph, calculating the process criticality score, and identifying the core monitoring process. The resource usage data of the core monitoring process is obtained through the resource monitoring coroutine, and the optimal sampling period is determined by an adaptive feedback control algorithm, and the data is written into a time series database. The sliding time window algorithm is used to perform data aggregation analysis to obtain performance indicators and construct an anomaly detection model. Based on the principal component analysis method, a feature vector is constructed by dimensionality reduction, and a clustering algorithm is used to divide the load type. A differentiated process optimization strategy is generated according to the feature vector and the load type. When performance fluctuations are detected, the optimization strategy is executed, and the anomaly detection model and the deep reinforcement learning model are updated according to the performance improvement index, and finally an optimization diagnosis report is generated.

Description

Technical Field

[0001] The present invention relates to intelligent monitoring and resource optimization technology, and in particular to a Python monitoring task resource usage method and system. Background Art

[0002] In modern computer systems, as the complexity of tasks and processes continues to increase, monitoring and optimizing the use of process resources has become an important issue. Traditional resource monitoring methods mostly rely on static configuration and timed sampling, which are difficult to cope with dynamically changing resource requirements and complex process dependencies. System administrators often can only rely on manual analysis and optimization, lacking automated and intelligent management methods, resulting in inefficient resource utilization and even system performance bottlenecks.

[0003] In response to this problem, in recent years, with the development of deep learning and reinforcement learning technologies, data-driven automated monitoring and optimization solutions have gradually gained attention. Some existing studies mainly focus on analyzing and predicting system performance through machine learning methods, but these methods usually ignore the dependencies between processes and differentiated optimization strategies under different load conditions. At the same time, the anomaly detection models in traditional methods are often difficult to adjust intelligently according to different load types, resulting in inaccurate optimization strategies.

[0004] Therefore, there is an urgent need for a Python monitoring task resource usage method based on a combination of deep learning and reinforcement learning to optimize the resource sampling cycle and process performance in real time to improve the overall efficiency and stability of the system. Summary of the invention

[0005] The embodiment of the present invention provides a Python monitoring task resource usage method and system, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention,

[0007] Provides a Python monitoring task resource usage method, including:

[0008] The runtime status data of the monitoring server is obtained through the Python system call interface, and the process identification code, process startup timestamp and process call stack information are extracted from the runtime status data. The process identification code is hash matched with the pre-built target task feature library to obtain the target task process, and the call features of the process call stack information are extracted using a multi-layer convolutional neural network to obtain process dependency data. Based on the process dependency data, a process dependency graph is constructed using a minimum spanning tree algorithm, and the node degree centrality of the nodes in the process dependency graph is calculated to obtain a process criticality score. The target task process whose process criticality score is greater than a preset criticality threshold is determined as a core monitoring process, and the process identification code of the core monitoring process is written into the monitoring collection object pool;

[0009] A resource monitoring coroutine is created for each core monitoring process in the monitoring collection object pool. The resource monitoring coroutine obtains resource usage data of the core monitoring process through a system call interface, calculates the process resource volatility of the resource usage data, substitutes the process resource volatility into an adaptive feedback control algorithm to calculate an optimal sampling period, writes the resource usage data into a time series database according to the optimal sampling period, uses a sliding time window algorithm to perform aggregation analysis on the data in the time series database to obtain performance indicators, builds an anomaly detection model based on the performance indicators, uses principal component analysis to reduce the dimension of the performance indicators to construct a feature vector, and uses a clustering algorithm based on the feature vector to divide the core monitoring process into different load types;

[0010] The feature vector and load type are input into a deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data. When the anomaly detection model detects performance fluctuations, the differentiated process optimization strategy is executed, the optimized resource usage data is obtained, and the performance improvement index of the optimized resource usage data relative to the data before optimization is calculated. When the performance improvement index is greater than a preset optimization threshold, the parameters of the anomaly detection model are updated, an optimization diagnosis report is generated, and the optimization strategy, performance improvement index and load type are written into the training sample library of the deep reinforcement learning model for continuous optimization.

[0011] In an optional embodiment,

[0012] The target task process is obtained by performing hash matching on the process identification code and the pre-built target task feature library, including:

[0013] A sliding time window is used to extract features from the process identification code, the sliding time window is divided into multiple time segments, and a weighted combination of feature vectors in each time segment is calculated based on an exponential decay function to obtain an initial time series feature matrix;

[0014] Based on the data distribution characteristics of the initial time series feature matrix, multi-scale overlapping sharding is performed, the optimal sharding parameters and the size of the overlapping area are determined by calculating the feature density, and multiple groups of local sensitive hash function families are used to map the feature shards to generate a multi-dimensional hash mapping sequence, wherein the parameters of the hash function family are determined by feature similarity constraints;

[0015] The multidimensional hash mapping sequence is used to construct a spatiotemporal feature association network, the local similarity and temporal migration features between adjacent shards are calculated, the local similarity and temporal migration features are combined to form edge weights, the path features in the network are extracted through deep walking, and the hierarchical feature fingerprint is generated by combining the single point features and the path features;

[0016] A multi-level cache feature library is constructed according to the access frequency and timeliness of the hierarchical feature fingerprints, and a feature index forest is established in each level of cache. The nodes of the feature index forest store feature fingerprints, associated network structures, and adaptive matching thresholds, and search priorities are assigned to nodes based on the accuracy of historical matching results and feature stability;

[0017] According to the search priority, a parallel search path is planned in the feature index forest, and a multi-path search is performed on the feature fingerprint to be matched. The comprehensive similarity between the node and the feature to be matched is calculated on the search path. The comprehensive similarity includes the feature vector distance, the network structure similarity and the timing pattern matching degree. When the comprehensive similarity of the node exceeds the adaptive matching threshold, the process corresponding to the current node is determined as the target task process.

[0018] In an optional embodiment,

[0019] The process call stack information is extracted using a multi-layer convolutional neural network to obtain call features to obtain process dependency data, a process dependency graph is constructed based on the process dependency data using a minimum spanning tree algorithm, the node degree centrality of the nodes in the process dependency graph is calculated to obtain a process criticality score, and the target task process whose process criticality score is greater than a preset criticality threshold is determined as a core monitoring process, including:

[0020] The process call stack information is segmented using a sliding time window, the process call stack information includes the caller process ID, the callee process ID, the call timestamp and the call depth, the process call frequency, the call level and the resource occupancy features are extracted from each time window, and the extracted features are weighted and combined based on the time series correlation to obtain a time series feature matrix;

[0021] The time series feature matrix is ​​input into a multi-layer convolutional neural network, which includes a feature extraction layer, a pattern fusion layer and a relationship modeling layer, wherein the feature extraction layer includes a plurality of convolution kernels of different sizes, each convolution kernel performs a convolution operation on the time series feature matrix to obtain a feature map of a corresponding scale, and each feature map is combined to form a multi-scale feature; the pattern fusion layer performs a weighted calculation on each feature component in the multi-scale feature based on the attention weight to obtain a feature fusion vector; the relationship modeling layer uses a graph convolutional network to process the feature fusion vector, and obtains a process call feature vector based on the call association between processes;

[0022] Based on the process call feature vector, the call pattern similarity, resource occupancy similarity and timing behavior similarity between processes are calculated, the call pattern similarity, resource occupancy similarity and timing behavior similarity are weightedly combined using adaptive weights to obtain process similarity, and the process similarity is converted into a distance metric to construct a process affinity graph;

[0023] Running a minimum spanning tree algorithm based on the process affinity graph, introducing node importance constraints into the minimum spanning tree algorithm and dynamically adjusting the edge weight selection strategy based on the node importance constraints, maintaining the connectivity of process nodes through a union-find structure, and selecting a connected subgraph with a minimum edge weight combination as a process dependency graph;

[0024] In the process dependency graph, the node degree centrality of each process node is calculated to obtain a local centrality index, a global centrality index is calculated based on the shortest path between process nodes, and the local centrality index and the global centrality index are weighted and normalized to obtain a process criticality score;

[0025] The system load status and historical recognition accuracy are obtained from the criticality threshold calculation unit which includes two layers of neural networks, wherein the first layer of the neural network maps the system load status and the historical recognition accuracy into dynamic adjustment factors, and the second layer of the neural network modifies the preset criticality threshold based on the dynamic adjustment factor to obtain the current criticality threshold, and determines the target task process that is higher than the current criticality threshold as the core monitoring process.

[0026] In an optional embodiment,

[0027] Running a minimum spanning tree algorithm based on the process affinity graph, introducing node importance constraints in the minimum spanning tree algorithm and dynamically adjusting the edge weight selection strategy based on the node importance constraints, maintaining the connectivity of process nodes through a union-find structure, and selecting a connected subgraph with a minimum edge weight combination as a process dependency graph includes:

[0028] The process node degree centrality is obtained by counting the number of adjacent edges of each process node in the process affinity graph, the process node betweenness centrality is obtained by calculating the proportion of paths passing through each process node in all shortest paths, and the process node proximity centrality is obtained based on the shortest distance from each process node to other process nodes;

[0029] The process node degree centrality, process node betweenness centrality and process node closeness centrality are weighted and combined by an importance weight coefficient to obtain a node importance constraint index of each process node;

[0030] Extract the initial edge weight of each edge in the process affinity graph, obtain the node importance constraint index of the process nodes at both ends of each edge, select the smaller node importance constraint index of the process nodes at both ends as the importance constraint value of the edge, and calculate the edge weight adjustment factor according to the importance constraint value of the edge, wherein the edge weight adjustment factor is obtained by subtracting the product of the importance constraint value of the edge and the preset weight adjustment coefficient from the preset reference value;

[0031] Multiply the initial edge weight of each edge by the corresponding edge weight adjustment factor to obtain the adjusted edge weight, and sort all edges in ascending order according to the adjusted edge weights to obtain a sequence of edges to be selected;

[0032] Create and query a data structure, including a parent node array and a level array, wherein the parent node array records the connected component number to which each process node belongs, and the level array records the level value of each connected component;

[0033] Traversing the sequence of edges to be selected in sequence, for each edge, searching the connected component numbers of the process nodes at both ends of the edge through the parent node array in the union-find set structure, when the process nodes at both ends of the edge do not belong to the same connected component, calculating the sum of the node importance constraint indexes of the newly covered process nodes after the current edge is added, and performing a weighted combination with the sum of the adjusted edge weights of the currently selected edge set to obtain a connected subgraph importance score;

[0034] The importance score of the connected subgraph is compared with the current optimal score. When the importance score of the connected subgraph is better, the current edge is added to the final edge set, and the connectivity relationship is updated in the parent node array and level array of the union-find set, and the final edge set with the minimum edge weight combination is constructed as a process dependency graph.

[0035] In an optional embodiment,

[0036] Calculating the process resource fluctuation rate of the resource usage data, and substituting the process resource fluctuation rate into the adaptive feedback control algorithm to calculate the optimal sampling period includes:

[0037] Calculate the change rate of resource usage data at adjacent moments, obtain original fluctuation characteristic data by calculating the absolute value of the relative change rate, process the original fluctuation characteristic data by using an exponential smoothing model, attenuate the noise data based on weighted calculation of the current sampling value and the historical cumulative value, obtain smoothed resource fluctuation data, and store the smoothed resource fluctuation data in a fluctuation characteristic set;

[0038] Based on the fluctuation feature set, information entropy in three dimensions, namely, CPU usage, memory usage, and I / O operation count, is calculated, the information entropy is normalized to obtain weight coefficients of each dimension, and the smoothed resource fluctuation values ​​are weighted and integrated using the weight coefficients to obtain a process resource fluctuation rate that characterizes the overall fluctuation degree of the process;

[0039] Construct a state space model, take the process resource volatility and the current sampling period as state variables, take the sampling period adjustment amount as control input, establish the system state equation and output equation; design a performance indicator function, the performance indicator function includes a quadratic term of the process resource volatility deviation and a quadratic term of the sampling period adjustment cost;

[0040] An algebraic Riccati equation is constructed based on the system state equation and the performance indicator function, the optimal feedback gain matrix is ​​obtained by solving the algebraic Riccati equation, the process resource volatility is substituted into the state equation, and the adjustment amount of the sampling period is calculated according to the optimal feedback gain matrix;

[0041] An optimization objective function is constructed according to the adjustment amount. Under the sampling period constraint, the optimization objective function is solved by the gradient projection method. The candidate sampling periods that meet the constraints are obtained through iterative calculation. The convergence of the candidate sampling periods is judged. When the iterative difference meets the threshold requirement, the converged sampling period is determined as the optimal sampling period.

[0042] In an optional embodiment,

[0043] The principal component analysis method is used to reduce the dimension of the performance index to construct a feature vector, and a clustering algorithm is used based on the feature vector to divide the core monitoring process into different load types, including:

[0044] Organize the performance indicator data according to the time dimension, space dimension and resource type dimension, construct the performance indicator tensor of the core monitoring process, perform three-dimensional tensor decomposition on the performance indicator tensor, extract the characteristic subspace of the time dimension, space dimension and resource type dimension, and reconstruct the reduced dimension performance indicator based on the characteristic subspace;

[0045] Based on the dimensionality reduction performance index, a hierarchical progressive feature extraction network is constructed, local spatiotemporal features are extracted through the convolution layer of the hierarchical progressive feature extraction network, temporal dependencies are modeled through the recurrent layer, correlations between different resource indicators are captured through the attention layer, and the hidden layer output of the feature extraction network is constructed as a feature vector;

[0046] A dual clustering strategy is adopted for the feature vectors, initial clustering is performed based on local sensitive hashing to obtain initial clusters, and then a spectral clustering algorithm based on an adaptive kernel function is applied to the initial clusters for secondary partitioning to obtain clustering results;

[0047] Calculating hierarchical load characteristics based on the clustering results, wherein the hierarchical load characteristics include characteristic combinations of different time scales and different resource dimensions;

[0048] Using the hierarchical load characteristics to build a multi-task learning model, and simultaneously predict the resource demand trend, load level change, and performance risk level of the core monitoring process;

[0049] Combined with the online anomaly detection algorithm, the load characteristic deviation degree of the core monitoring process is monitored in real time. When the load characteristic deviation degree exceeds the adaptive deviation threshold, the load type of the core monitoring process is reclassified, and the core monitoring process is divided into different load types.

[0050] In an optional embodiment,

[0051] Inputting the feature vector and the load type into a deep reinforcement learning model, wherein the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data, including:

[0052] Generate a corresponding one-hot encoding according to the load type, combine the feature vector and the one-hot encoding to form a state vector, and normalize the state vector to obtain a standardized state representation;

[0053] Constructing an action space for process optimization, the action space includes CPU scheduling priority, memory page replacement strategy, IO buffer size, and process migration target node, and discretizing each optimization parameter in the action space to obtain a discrete action set;

[0054] Construct a deep reinforcement learning model with a dual network structure, where the policy network adopts a multi-layer fully connected structure, the input layer receives the standardized state representation, and outputs the action probability distribution after processing by a modified linear activation function, and the value network adopts a multi-layer fully connected structure to output the state value estimation;

[0055] An adaptive reward function is constructed based on the load type, the CPU efficiency weight is increased for computationally intensive loads, the memory efficiency weight is increased for memory intensive loads, and the IO efficiency weight is increased for IO intensive loads, and the weighted sum of each efficiency index and the corresponding weight is used as the reward value of historical optimization experience;

[0056] The optimization action output by the execution strategy network interacts with the environment, records the historical optimization experience data consisting of the standardized state representation, the execution action and the reward value, and stores the historical optimization experience data in the experience replay pool;

[0057] Sampling historical optimization experience data from the experience replay pool, calculating the value estimate of each state in the sampled data using the value network, and calculating the advantage function based on the value estimate and the reward value;

[0058] The proximal policy optimization algorithm is used to update the policy network parameters, the advantage function is used to construct the clipping objective function for gradient update, and the mean square error between the value estimate and the actual return is used to update the value network parameters, so as to obtain a trained deep reinforcement learning model;

[0059] The feature vector and load type of the process to be optimized are input into the trained deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data.

[0060] According to a second aspect of the embodiments of the present invention,

[0061] Provide a Python monitoring task resource usage system, including:

[0062] The first unit is used to obtain the runtime status data of the monitoring server through the Python system call interface, extract the process identification code, process startup timestamp and process call stack information from the runtime status data, perform hash matching on the process identification code and the pre-built target task feature library to obtain the target task process, extract call features from the process call stack information using a multi-layer convolutional neural network to obtain process dependency data, construct a process dependency graph based on the process dependency data using a minimum spanning tree algorithm, calculate the node degree centrality of the nodes in the process dependency graph to obtain a process criticality score, determine the target task process whose process criticality score is greater than a preset criticality threshold as a core monitoring process, and write the process identification code of the core monitoring process into a monitoring collection object pool;

[0063] The second unit is used to create a resource monitoring coroutine for each core monitoring process in the monitoring collection object pool, the resource monitoring coroutine obtains the resource usage data of the core monitoring process through the system call interface, calculates the process resource volatility of the resource usage data, substitutes the process resource volatility into the adaptive feedback control algorithm to calculate the optimal sampling period, writes the resource usage data into a time series database according to the optimal sampling period, uses a sliding time window algorithm to perform aggregation analysis on the data in the time series database to obtain performance indicators, builds an anomaly detection model according to the performance indicators, and uses the principal component analysis method to reduce the dimension of the performance indicators to construct a feature vector, and uses a clustering algorithm based on the feature vector to divide the core monitoring process into different load types;

[0064] The third unit is used to input the feature vector and load type into the deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data. When the anomaly detection model detects performance fluctuations, the differentiated process optimization strategy is executed, the optimized resource usage data is obtained, and the performance improvement index of the optimized resource usage data relative to the data before optimization is calculated. When the performance improvement index is greater than a preset optimization threshold, the parameters of the anomaly detection model are updated, an optimization diagnosis report is generated, and the optimization strategy, performance improvement index and load type are written into the training sample library of the deep reinforcement learning model for continuous optimization.

[0065] According to a third aspect of the embodiments of the present invention,

[0066] An electronic device is provided, comprising:

[0067] processor;

[0068] a memory for storing processor-executable instructions;

[0069] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0070] According to a fourth aspect of the embodiments of the present invention,

[0071] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0072] The beneficial effects of this application are as follows:

[0073] In this embodiment, by constructing a process dependency graph and calculating node centrality, the core monitoring process can be effectively identified, excessive monitoring of non-critical processes can be avoided, and the accuracy and efficiency of monitoring can be improved. At the same time, a multi-layer convolutional neural network is used to extract call features, which can more accurately capture the complex dependencies between processes. The introduction of an adaptive feedback control algorithm to dynamically adjust the sampling period and combine it with sliding time window aggregation analysis not only ensures the real-time nature of data collection, but also reduces the overhead of data storage and processing. The principal component analysis dimensionality reduction and clustering algorithm are used to classify processes, laying the foundation for subsequent differentiated optimization. Differentiated process optimization strategies are generated based on the deep reinforcement learning model, and the model parameters are optimized through continuous feedback to achieve self-evolution of the monitoring system. This intelligent monitoring and optimization method can flexibly adjust strategies according to different load types, effectively improving system resource utilization and overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 This is a flowchart of a method for using Python monitoring task resources according to an embodiment of the present invention;

[0075] Figure 2 It is a structural diagram of the Python monitoring task resource usage system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0078] Figure 1 The following is a flow chart of a method for using Python monitoring task resources according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0079] S101. Obtain the runtime status data of the monitoring server through the Python system call interface, extract the process identification code, process startup timestamp and process call stack information from the runtime status data, perform hash matching on the process identification code and the pre-built target task feature library to obtain the target task process, extract call features from the process call stack information using a multi-layer convolutional neural network to obtain process dependency data, construct a process dependency graph based on the process dependency data using a minimum spanning tree algorithm, calculate the node degree centrality of the nodes in the process dependency graph to obtain a process criticality score, determine the target task process whose process criticality score is greater than a preset criticality threshold as a core monitoring process, and write the process identification code of the core monitoring process into the monitoring collection object pool;

[0080] S102. Create a resource monitoring coroutine for each core monitoring process in the monitoring collection object pool, the resource monitoring coroutine obtains the resource usage data of the core monitoring process through the system call interface, calculates the process resource volatility of the resource usage data, substitutes the process resource volatility into the adaptive feedback control algorithm to calculate the optimal sampling period, writes the resource usage data into the time series database according to the optimal sampling period, uses the sliding time window algorithm to aggregate and analyze the data in the time series database to obtain performance indicators, builds an anomaly detection model according to the performance indicators, and uses the principal component analysis method to reduce the dimension of the performance indicators to construct a feature vector, and uses a clustering algorithm based on the feature vector to divide the core monitoring process into different load types;

[0081] S103. Input the feature vector and load type into the deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on the historical optimization experience data. When the anomaly detection model detects performance fluctuations, the differentiated process optimization strategy is executed, the optimized resource usage data is obtained, and the performance improvement index of the optimized resource usage data relative to the data before optimization is calculated. When the performance improvement index is greater than the preset optimization threshold, the parameters of the anomaly detection model are updated, an optimization diagnosis report is generated, and the optimization strategy, performance improvement index and load type are written into the training sample library of the deep reinforcement learning model for continuous optimization.

[0082] Exemplarily, the runtime status data of the monitoring server is obtained through the Python system call interface, and the data is used as input to extract the process identification code, process start timestamp and process call stack information. The process identification code is a unique identifier assigned by the operating system to each running process, which is used to distinguish different processes. The process start timestamp indicates the time when the process is activated in the operating system and is used to track the life cycle of the process. The process call stack information records the function call path of the current process and reflects the running logic and context of the process. The extracted process identification code is hash matched with the pre-built target task feature library to quickly identify the target task process. Then, the extracted process call stack information is feature extracted using a multi-layer convolutional neural network to obtain dependency data between processes. These dependency data are integrated into a process dependency graph through a minimum spanning tree algorithm. In the graph, nodes represent processes and edges represent dependencies between processes. By calculating the node degree centrality of the node, the importance of each process in the dependency is measured to determine the key process. For the target task process whose criticality score is greater than the preset threshold, it is determined as a core monitoring process, and its identification code is written into the monitoring collection object pool.

[0083] A resource monitoring coroutine is created for each core monitoring process. The coroutine periodically obtains the resource usage data of the core monitoring process through the Python system call interface, including indicators such as CPU occupancy, memory usage, and I / O throughput. These data are used to calculate the process resource volatility, which describes the degree of resource usage change over time. The resource volatility is substituted into the adaptive feedback control algorithm, and the sampling period is dynamically adjusted according to the volatility characteristics to reduce resource overhead while ensuring data accuracy. The collected data is written into the time series database to record its historical trajectory over time. Through the sliding time window algorithm, the data in the time series database is aggregated and analyzed to generate process performance indicators, such as average resource usage and peak utilization. An anomaly detection model is built based on these performance indicators to identify abnormal behaviors of the process. At the same time, the performance indicators are subjected to principal component analysis to extract key features, reduce redundant information, and generate feature vectors suitable for clustering analysis. Using the clustering algorithm, the core monitoring processes are divided into different types according to their load characteristics, such as high computing load type and IO intensive type.

[0084] The feature vector and load type of the process are input into the deep reinforcement learning model, which generates differentiated optimization strategies for different load types based on historical optimization experience data. When the anomaly detection model identifies performance fluctuations, the corresponding optimization strategy is triggered to adjust the resource usage of the core monitoring process, such as dynamically adjusting resource allocation and modifying priorities. After optimization, the performance before optimization is compared with the resource usage data to calculate the performance improvement effect. If the improvement index exceeds the preset threshold, the parameters of the anomaly detection model are updated to improve the accuracy of subsequent anomaly detection. An optimization diagnostic report is generated to record the optimization strategy, performance improvement index and load type. This information is stored in the training sample library of the deep reinforcement learning model for continuous optimization of model performance.

[0085] In an optional implementation, performing hash matching between the process identification code and a pre-built target task feature library to obtain the target task process includes:

[0086] A sliding time window is used to extract features from the process identification code, the sliding time window is divided into multiple time segments, and a weighted combination of feature vectors in each time segment is calculated based on an exponential decay function to obtain an initial time series feature matrix;

[0087] Based on the data distribution characteristics of the initial time series feature matrix, multi-scale overlapping sharding is performed, the optimal sharding parameters and the size of the overlapping area are determined by calculating the feature density, and multiple groups of local sensitive hash function families are used to map the feature shards to generate a multi-dimensional hash mapping sequence, wherein the parameters of the hash function family are determined by feature similarity constraints;

[0088] The multidimensional hash mapping sequence is used to construct a spatiotemporal feature association network, the local similarity and temporal migration features between adjacent shards are calculated, the local similarity and temporal migration features are combined to form edge weights, the path features in the network are extracted through deep walking, and the hierarchical feature fingerprint is generated by combining the single point features and the path features;

[0089] A multi-level cache feature library is constructed according to the access frequency and timeliness of the hierarchical feature fingerprints, and a feature index forest is established in each level of cache. The nodes of the feature index forest store feature fingerprints, associated network structures, and adaptive matching thresholds, and search priorities are assigned to nodes based on the accuracy of historical matching results and feature stability;

[0090] According to the search priority, a parallel search path is planned in the feature index forest, and a multi-path search is performed on the feature fingerprint to be matched. The comprehensive similarity between the node and the feature to be matched is calculated on the search path. The comprehensive similarity includes the feature vector distance, the network structure similarity and the timing pattern matching degree. When the comprehensive similarity of the node exceeds the adaptive matching threshold, the process corresponding to the current node is determined as the target task process.

[0091] Exemplarily, first, feature extraction is performed on the process identification code to generate a time series feature matrix. Specifically, a sliding time window is selected and the window is divided into multiple time segments. For example, a 10-second window can be divided into 10 1-second time segments. Feature extraction is performed on the process identification code in each time segment. For example, features such as CPU occupancy, memory usage, and network traffic can be extracted to form a feature vector. Then, the feature vectors of each time segment are weighted and combined based on the exponential decay function. For example, a time segment closer to the current time point can be given a greater weight to highlight the importance of recent features. Finally, an initial time series feature matrix is ​​obtained, for example, a matrix with 10 rows (representing 10 time segments) and N columns (representing N features). Assuming that the CPU occupancy of a time segment is 20%, the memory usage is 30%, and the network traffic is 10Mbps, the feature vector of the time segment is [20, 30, 10].

[0092] Next, the initial time series feature matrix is ​​sliced ​​in multi-scale overlapping manner. According to the data distribution characteristics, the optimal slice parameters and the size of the overlapping area are determined. For example, the size of the slice and the overlapping area can be determined based on the variance or information entropy of the eigenvalue. Assume that the above 10-row N-column matrix is ​​divided into 3 slices, each slice contains 4 rows of data, and 1 row of data overlaps between adjacent slices. Then, multiple groups of local sensitive hash function families are used to map the feature slices to generate a multidimensional hash mapping sequence. The parameters of the hash function family are determined by feature similarity constraints. For example, the parameters can be determined by minimizing the probability of hash conflicts. Assuming that two groups of hash functions are used, each slice will be mapped to a two-dimensional hash space.

[0093] Subsequently, a spatiotemporal feature association network is constructed using a multidimensional hash map sequence. The local similarity between adjacent shards is calculated. For example, cosine similarity or Euclidean distance can be used to measure the similarity between shards. At the same time, the temporal migration features between shards are calculated. For example, the average value of the feature vector difference between adjacent shards can be calculated. The local similarity and temporal migration features are combined to form edge weights. Assuming that the similarity of two adjacent shards is 0.8 and the temporal migration feature is 0.2, the edge weight between them can be set to 0.8×0.2=0.16. Then, the path features in the network are extracted by deep walking. For example, the frequency of occurrence of paths of different lengths can be counted. Finally, a hierarchical feature fingerprint is generated by combining single-point features (for example, the average feature vector of each shard) and path features.

[0094] Finally, a multi-level cache feature library is constructed based on the access frequency and timeliness of hierarchical feature fingerprints. A feature index forest is established in each level of cache, and nodes store feature fingerprints, associated network structures, and adaptive matching thresholds. For example, the cache can be divided into three levels according to the access frequency of feature fingerprints: the first-level cache stores the feature fingerprints with the highest access frequency, the second-level cache stores the feature fingerprints with the second highest access frequency, and the third-level cache stores the feature fingerprints with the lowest access frequency. Search priorities are assigned to nodes based on the accuracy and feature stability of historical matching results. For example, the search priority of nodes can be adjusted according to the matching accuracy and feature stability of nodes, and nodes with high accuracy and stable features have higher search priorities. According to the search priority, parallel search paths are planned in the feature index forest, and multi-path retrieval is performed on the feature fingerprints to be matched. The comprehensive similarity between the node and the feature to be matched is calculated on the retrieval path. The comprehensive similarity includes feature vector distance, network structure similarity, and temporal pattern matching. When the comprehensive similarity of the node exceeds the adaptive matching threshold, the process corresponding to the current node is determined as the target task process.

[0095] In this embodiment, through multi-scale overlapping sharding and spatiotemporal feature association network, the feature information of the process can be captured more comprehensively, thereby improving the matching accuracy. The design of multi-level cache and feature index forest can effectively reduce the computational complexity of retrieval and improve matching efficiency. The adaptive matching threshold and multi-path retrieval mechanism can enhance the robustness of the method, enabling it to adapt to process identification tasks in different scenarios.

[0096] In an optional implementation, extracting call features from the process call stack information using a multi-layer convolutional neural network to obtain process dependency data, constructing a process dependency graph based on the process dependency data using a minimum spanning tree algorithm, calculating the node degree centrality of nodes in the process dependency graph to obtain a process criticality score, and determining the target task process whose process criticality score is greater than a preset criticality threshold as a core monitoring process includes:

[0097] The process call stack information is segmented using a sliding time window, the process call stack information includes the caller process ID, the callee process ID, the call timestamp and the call depth, the process call frequency, the call level and the resource occupancy features are extracted from each time window, and the extracted features are weighted and combined based on the time series correlation to obtain a time series feature matrix;

[0098] The time series feature matrix is ​​input into a multi-layer convolutional neural network, which includes a feature extraction layer, a pattern fusion layer and a relationship modeling layer, wherein the feature extraction layer includes a plurality of convolution kernels of different sizes, each convolution kernel performs a convolution operation on the time series feature matrix to obtain a feature map of a corresponding scale, and each feature map is combined to form a multi-scale feature; the pattern fusion layer performs a weighted calculation on each feature component in the multi-scale feature based on the attention weight to obtain a feature fusion vector; the relationship modeling layer uses a graph convolutional network to process the feature fusion vector, and obtains a process call feature vector based on the call association between processes;

[0099] Based on the process call feature vector, the call pattern similarity, resource occupancy similarity and timing behavior similarity between processes are calculated, the call pattern similarity, resource occupancy similarity and timing behavior similarity are weightedly combined using adaptive weights to obtain process similarity, and the process similarity is converted into a distance metric to construct a process affinity graph;

[0100] Running a minimum spanning tree algorithm based on the process affinity graph, introducing node importance constraints into the minimum spanning tree algorithm and dynamically adjusting the edge weight selection strategy based on the node importance constraints, maintaining the connectivity of process nodes through a union-find structure, and selecting a connected subgraph with a minimum edge weight combination as a process dependency graph;

[0101] In the process dependency graph, the node degree centrality of each process node is calculated to obtain a local centrality index, a global centrality index is calculated based on the shortest path between process nodes, and the local centrality index and the global centrality index are weighted and normalized to obtain a process criticality score;

[0102] The system load status and historical recognition accuracy are obtained from the criticality threshold calculation unit which includes two layers of neural networks, wherein the first layer of the neural network maps the system load status and the historical recognition accuracy into dynamic adjustment factors, and the second layer of the neural network modifies the preset criticality threshold based on the dynamic adjustment factor to obtain the current criticality threshold, and determines the target task process that is higher than the current criticality threshold as the core monitoring process.

[0103] Exemplarily, first, the process call stack information is preprocessed. The process call stack information includes the caller process ID, the callee process ID, the call timestamp and the call depth. A sliding time window mechanism is used, for example, the window size is set to 1 minute, the sliding step is 10 seconds, and the process call stack information is segmented. In each time window, the call frequency (for example, the number of times process A calls process B), the call level (for example, the depth of process A calling process B) and the resource occupancy (for example, CPU usage, memory usage) of each process are counted. Then, the extracted features are weighted and combined according to the temporal correlation. For example, the features of the current time window are given a higher weight, while the features of the earlier time window are given a lower weight. Finally, the weighted combined features are constructed into a temporal feature matrix. For example, suppose there are three processes A, B and C. In a time window, process A calls process B 5 times, the call depth is 2, the CPU usage is 20%, and the memory usage is 10%; process B calls process C 3 times, the call depth is 1, the CPU usage is 10%, and the memory usage is 5%. This information will form part of the time series feature matrix for that time window.

[0104] Next, the constructed time series feature matrix is ​​input into a multi-layer convolutional neural network. The network contains a feature extraction layer, a pattern fusion layer, and a relationship modeling layer. The feature extraction layer uses multiple convolution kernels of different sizes (for example, 1x1, 3x3, 5x5) to perform convolution operations on the time series feature matrix to extract feature maps of different scales. Then, these feature maps are combined to form multi-scale features. The pattern fusion layer uses the attention mechanism to perform weighted calculations on each feature component in the multi-scale features to obtain a feature fusion vector. The relationship modeling layer uses a graph convolutional network to process the feature fusion vector, considers the call association between processes, and finally obtains the process call feature vector.

[0105] Then, the similarity between processes is calculated based on the process call feature vectors. This includes call pattern similarity (e.g., do two processes frequently call each other), resource usage similarity (e.g., are the CPU usage and memory usage of the two processes similar), and timing behavior similarity (e.g., are the activity patterns of the two processes synchronized in time). These similarities are then weighted and combined using adaptive weights to obtain the final process similarity. The process similarity is converted into a distance metric and a process affinity graph is constructed. For example, if the similarity between process A and process B is 0.8, their distance in the affinity graph is 0.2.

[0106] Based on the process affinity graph, the minimum spanning tree algorithm is run to build the process dependency graph. Node importance constraints are introduced into the algorithm. For example, processes with high resource usage have higher importance. The edge weight selection strategy is dynamically adjusted according to the node importance, and edges connecting important nodes are given priority. The connectivity of process nodes is maintained through the union-find structure, and the connected subgraph with the minimum edge weight combination is selected as the process dependency graph.

[0107] In the constructed process dependency graph, the centrality score of each process node is calculated. First, the node degree centrality of each node is calculated as the local centrality index. Then, the global centrality index is calculated based on the shortest path between process nodes. Finally, the local centrality index and the global centrality index are weighted and normalized to obtain the process criticality score.

[0108] Finally, the core monitoring process is determined dynamically. The system load status and historical recognition accuracy are input into a criticality threshold calculation unit containing two layers of neural networks. The first layer of the neural network maps the system load status and historical recognition accuracy into a dynamic adjustment factor. The second layer of the neural network modifies the preset criticality threshold based on the dynamic adjustment factor to obtain the current criticality threshold. The target task process with a criticality score higher than the current criticality threshold is determined as the core monitoring process. For example, assuming that the preset criticality threshold is 0.8 and the dynamic adjustment factor is 0.1, the current criticality threshold is 0.9. If the criticality score of process A is 0.95, process A will be determined as the core monitoring process.

[0109] In this embodiment, the core monitoring process can be accurately identified to avoid monitoring all processes with the same intensity, thereby improving monitoring efficiency and reducing resource consumption. By focusing on monitoring the core monitoring process, potential system risks can be discovered in a timely manner and corresponding measures can be taken to enhance the stability of the system. The criticality threshold can be dynamically adjusted according to the system load status and historical recognition accuracy to adapt to different system environments and operating states.

[0110] In an optional implementation, a minimum spanning tree algorithm is run based on the process affinity graph, node importance constraints are introduced into the minimum spanning tree algorithm, and a selection strategy of edge weights is dynamically adjusted based on the node importance constraints, connectivity of process nodes is maintained through a union-find structure, and a connected subgraph with a minimum edge weight combination is selected as a process dependency graph, including:

[0111] The process node degree centrality is obtained by counting the number of adjacent edges of each process node in the process affinity graph, the process node betweenness centrality is obtained by calculating the proportion of paths passing through each process node in all shortest paths, and the process node proximity centrality is obtained based on the shortest distance from each process node to other process nodes;

[0112] The process node degree centrality, process node betweenness centrality and process node closeness centrality are weighted and combined by an importance weight coefficient to obtain a node importance constraint index of each process node;

[0113] Extract the initial edge weight of each edge in the process affinity graph, obtain the node importance constraint index of the process nodes at both ends of each edge, select the smaller node importance constraint index of the process nodes at both ends as the importance constraint value of the edge, and calculate the edge weight adjustment factor according to the importance constraint value of the edge, wherein the edge weight adjustment factor is obtained by subtracting the product of the importance constraint value of the edge and the preset weight adjustment coefficient from the preset reference value;

[0114] Multiply the initial edge weight of each edge by the corresponding edge weight adjustment factor to obtain the adjusted edge weight, and sort all edges in ascending order according to the adjusted edge weights to obtain a sequence of edges to be selected;

[0115] Create and query a data structure, including a parent node array and a level array, wherein the parent node array records the connected component number to which each process node belongs, and the level array records the level value of each connected component;

[0116] Traversing the sequence of edges to be selected in sequence, for each edge, searching the connected component numbers of the process nodes at both ends of the edge through the parent node array in the union-find set structure, when the process nodes at both ends of the edge do not belong to the same connected component, calculating the sum of the node importance constraint indexes of the newly covered process nodes after the current edge is added, and performing a weighted combination with the sum of the adjusted edge weights of the currently selected edge set to obtain a connected subgraph importance score;

[0117] The importance score of the connected subgraph is compared with the current optimal score. When the importance score of the connected subgraph is better, the current edge is added to the final edge set, and the connectivity relationship is updated in the parent node array and level array of the union-find set, and the final edge set with the minimum edge weight combination is constructed as a process dependency graph.

[0118] For example, first, it is necessary to count the connection information of each process node in the process affinity graph to evaluate the importance of the node. Specifically, the degree centrality is obtained by counting the number of adjacent edges of each process node, the betweenness centrality is obtained by calculating the proportion of paths passing through each process node in all shortest paths, and the closeness centrality is obtained based on the shortest distance from each process node to other process nodes.

[0119] Then, these centrality indicators are weighted and combined by the importance weight coefficient to obtain the node importance constraint indicator of each process node. For example, the weight coefficients of degree centrality, betweenness centrality, and closeness centrality can be set to 0.3, 0.5, and 0.2 respectively, and then the three centrality indicators are multiplied by the corresponding weight coefficients and summed to obtain the node importance constraint indicator of each process node. Assuming that the degree centrality of process A is 5, the betweenness centrality is 0.8, and the closeness centrality is 0.2, the node importance constraint indicator of process A is 0.3×5+0.5×0.8+0.2×0.2=2.04.

[0120] Next, extract the initial edge weight of each edge in the process affinity graph, and dynamically adjust the edge weight according to the node importance constraint index. For each edge, select the smaller node importance constraint index of the process nodes at both ends as the edge importance constraint value. Then, calculate the edge weight adjustment factor based on the edge importance constraint value. The edge weight adjustment factor is obtained by subtracting the product of the edge importance constraint value and the preset weight adjustment coefficient from the preset baseline value. Assuming the preset baseline value is 1, the weight adjustment coefficient is 0.1, and the importance constraint value of an edge is 0.5, the edge weight adjustment factor of the edge is 1-0.1×0.5=0.95.

[0121] Multiply the initial edge weight of each edge by the corresponding edge weight adjustment factor to obtain the adjusted edge weight, and sort all edges in ascending order according to the adjusted edge weights to obtain the edge sequence to be selected. For example, if the initial edge weight of an edge is 0.8 and the edge weight adjustment factor is 0.95, the adjusted edge weight is 0.8×0.95=0.76.

[0122] Create a union-find data structure to maintain the connectivity of process nodes. The union-find data structure includes a parent node array and a level array. The parent node array records the number of the connected component to which each process node belongs, and the level array records the level value of each connected component.

[0123] Traverse the sequence of edges to be selected in sequence, and for each edge, find the connected component number to which the process nodes at both ends belong through the parent node array in the union-find set structure. When the process nodes at both ends of the edge do not belong to the same connected component, calculate the sum of the node importance constraint index of the newly covered process node after the current edge is added, and perform a weighted combination with the sum of the adjusted edge weights of the currently selected edge set to obtain the importance score of the connected subgraph.

[0124] Compare the importance score of the connected subgraph with the current optimal score. When the importance score of the connected subgraph is better, add the current edge to the final edge set, and update the connectivity relationship in the parent node array and level array of the union-find set. Finally, the final edge set with the minimum edge weight combination is constructed as the process dependency graph.

[0125] Assume that there are three process nodes A, B, and C, and the initial edge weights between them are: AB: 0.8, AC: 0.5, BC: 0.6. The node importance constraint indicators are: A: 2.04, B: 1.5, C: 1.2. The preset baseline value is 1, and the weight adjustment coefficient is 0.1. Then the adjusted edge weights are: AB: 0.8×(1-0.1×1.5)=0.68, AC: 0.5×(1-0.1×1.2)=0.44, BC: 0.6×(1-0.1×1.2)=0.528.

[0126] In this embodiment, by introducing node importance constraints, key process nodes and their dependencies can be more accurately identified, avoiding misjudgment that may be caused by relying solely on edge weight information. The node importance constraint indicator provides a quantitative assessment of the importance of process nodes, making the graph easier to understand and analyze. By dynamically adjusting edge weights and using a union-find structure, the computational complexity can be effectively reduced and the efficiency of graph construction can be improved.

[0127] In an optional implementation, calculating the process resource fluctuation rate of the resource usage data, and substituting the process resource fluctuation rate into the adaptive feedback control algorithm to calculate the optimal sampling period includes:

[0128] Calculate the change rate of resource usage data at adjacent moments, obtain original fluctuation characteristic data by calculating the absolute value of the relative change rate, process the original fluctuation characteristic data by using an exponential smoothing model, attenuate the noise data based on weighted calculation of the current sampling value and the historical cumulative value, obtain smoothed resource fluctuation data, and store the smoothed resource fluctuation data in a fluctuation characteristic set;

[0129] Based on the fluctuation feature set, information entropy in three dimensions, namely, CPU usage, memory usage, and I / O operation count, is calculated, the information entropy is normalized to obtain weight coefficients of each dimension, and the smoothed resource fluctuation values ​​are weighted and integrated using the weight coefficients to obtain a process resource fluctuation rate that characterizes the overall fluctuation degree of the process;

[0130] Construct a state space model, take the process resource volatility and the current sampling period as state variables, take the sampling period adjustment amount as control input, establish the system state equation and output equation; design a performance indicator function, the performance indicator function includes a quadratic term of the process resource volatility deviation and a quadratic term of the sampling period adjustment cost;

[0131] An algebraic Riccati equation is constructed based on the system state equation and the performance indicator function, the optimal feedback gain matrix is ​​obtained by solving the algebraic Riccati equation, the process resource volatility is substituted into the state equation, and the adjustment amount of the sampling period is calculated according to the optimal feedback gain matrix;

[0132] An optimization objective function is constructed according to the adjustment amount. Under the sampling period constraint, the optimization objective function is solved by the gradient projection method. The candidate sampling periods that meet the constraints are obtained through iterative calculation. The convergence of the candidate sampling periods is judged. When the iterative difference meets the threshold requirement, the converged sampling period is determined as the optimal sampling period.

[0133] For example, first, the resource usage data of the process is collected. For example, every initial sampling period (e.g., 50 milliseconds), data such as CPU usage, memory usage, and I / O operation counts are collected. Assume that the data collected at a certain moment are respectively CPU usage 70%, memory usage 2GB, and I / O operation count 1000 times.

[0134] Next, calculate the change rate of resource usage data at adjacent moments. For example, if the CPU usage was 60% at the previous moment, the change rate of CPU usage at the current moment is (70%-60%) / 60%=16.7%. Calculate this change rate for each resource dimension and take its absolute value to obtain the original fluctuation characteristic data. In this example, assume that the absolute value of the memory usage change rate is 5% and the absolute value of the I / O operation count change rate is 2%.

[0135] Then, the original fluctuation feature data is processed using an exponential smoothing model to reduce the impact of noise data. For example, with a smoothing factor of 0.8, the current CPU usage change rate of 16.7% and the smoothed CPU usage change rate of the previous moment (assuming it is 15%) are weighted averaged to obtain the smoothed CPU usage change rate: 0.8×16.7%+(1-0.8)×15%=16.36%. The same smoothing process is performed on memory usage and I / O operation counts. The smoothed resource fluctuation data is stored in the fluctuation feature set.

[0136] Based on the set of fluctuation characteristics, the information entropy of each resource dimension is calculated. Information entropy is used to measure the uncertainty of data. The greater the fluctuation, the greater the information entropy. There are many ways to calculate information entropy, for example, the Shannon entropy formula can be used. After calculating the information entropy of the three dimensions of CPU usage, memory usage, and I / O operation count, they are normalized to obtain their respective weight coefficients. For example, assuming that the information entropy of CPU usage is 0.5, the information entropy of memory usage is 0.3, and the information entropy of I / O operation count is 0.2, then the normalized weight coefficients are 0.5 / (0.5+0.3+0.2)=0.5, 0.3 / (0.5+0.3+0.2)=0.3, and 0.2 / (0.5+0.3+0.2)=0.2.

[0137] The smoothed resource fluctuation values ​​are weighted and merged using the weight coefficients to obtain the process resource fluctuation rate that represents the overall fluctuation degree of the process. For example, the smoothed CPU usage change rate of 16.36%, the memory usage change rate, and the I / O operation count change rate are multiplied by the corresponding weight coefficients of 0.5, 0.3, and 0.2, respectively, and then added to obtain the process resource fluctuation rate.

[0138] Construct a state space model, take the process resource volatility and the current sampling period as state variables, and take the sampling period adjustment as control input. Establish the system state equation and output equation to describe the law of system state change over time. Design a performance indicator function that contains the quadratic term of the process resource volatility deviation and the quadratic term of the sampling period adjustment cost. The goal is to minimize the performance indicator function.

[0139] Based on the system state equation and performance index function, the algebraic Riccati equation is constructed, and the optimal feedback gain matrix is ​​obtained by solving the equation. The process resource volatility is substituted into the state equation, and the adjustment amount of the sampling period is calculated according to the optimal feedback gain matrix.

[0140] The optimization objective function is constructed based on the calculated adjustment amount. Under the sampling period constraint (for example, the sampling period must be between 10 milliseconds and 1000 milliseconds), the optimization objective function is solved using the gradient projection method. The candidate sampling period that meets the constraint is obtained through iterative calculation. The convergence of the candidate sampling period is judged. When the iterative difference meets the threshold requirement (for example, less than 0.1 milliseconds), the converged sampling period is determined as the optimal sampling period.

[0141] In this embodiment, by adaptively adjusting the sampling period, the sampling frequency can be reduced when resource fluctuations are small, thereby reducing system overhead and improving resource utilization efficiency. The sampling frequency can be increased in a timely manner to ensure monitoring accuracy and avoid missing important resource fluctuation information. By dynamically adjusting the sampling period, the system can respond to resource fluctuations more promptly, thereby improving the stability and reliability of the system.

[0142] In an optional implementation, the principal component analysis method is used to reduce the dimension of the performance index to construct a feature vector, and a clustering algorithm is used based on the feature vector to divide the core monitoring process into different load types, including:

[0143] Organize the performance indicator data according to the time dimension, space dimension and resource type dimension, construct the performance indicator tensor of the core monitoring process, perform three-dimensional tensor decomposition on the performance indicator tensor, extract the characteristic subspace of the time dimension, space dimension and resource type dimension, and reconstruct the reduced dimension performance indicator based on the characteristic subspace;

[0144] Based on the dimensionality reduction performance index, a hierarchical progressive feature extraction network is constructed, local spatiotemporal features are extracted through the convolution layer of the hierarchical progressive feature extraction network, temporal dependencies are modeled through the recurrent layer, correlations between different resource indicators are captured through the attention layer, and the hidden layer output of the feature extraction network is constructed as a feature vector;

[0145] A dual clustering strategy is adopted for the feature vectors, initial clustering is performed based on local sensitive hashing to obtain initial clusters, and then a spectral clustering algorithm based on an adaptive kernel function is applied to the initial clusters for secondary partitioning to obtain clustering results;

[0146] Calculating hierarchical load characteristics based on the clustering results, wherein the hierarchical load characteristics include characteristic combinations of different time scales and different resource dimensions;

[0147] Using the hierarchical load characteristics to build a multi-task learning model, and simultaneously predict the resource demand trend, load level change, and performance risk level of the core monitoring process;

[0148] Combined with the online anomaly detection algorithm, the load characteristic deviation degree of the core monitoring process is monitored in real time. When the load characteristic deviation degree exceeds the adaptive deviation threshold, the load type of the core monitoring process is reclassified, and the core monitoring process is divided into different load types.

[0149] Exemplarily, first, the performance indicator data of the core monitoring process is collected. These performance indicators include CPU usage, memory occupancy, disk IO rate, network traffic, etc. The data is organized according to the time dimension (e.g., per second or per minute), space dimension (e.g., different servers or virtual machines), and resource type dimension (e.g., CPU, memory, disk, network) to form a three-dimensional performance indicator tensor. For example, the CPU usage, memory occupancy, disk IO rate, and network traffic data of 10 core monitoring processes within 5 minutes and across 3 servers can be collected to form a 10x5x3x4 three-dimensional tensor.

[0150] Then, the performance indicator tensor is decomposed into three dimensions. This is a dimensionality reduction technique that can decompose high-dimensional data into low-dimensional feature subspaces. By extracting the feature subspaces of the time dimension, space dimension, and resource type dimension respectively, the main features in different dimensions can be captured. For example, the feature subspace in the time dimension may reflect the periodic changes in the process load, the feature subspace in the space dimension may reflect the load differences on different servers, and the feature subspace in the resource type dimension may reflect the demand pattern of the process for different resources. The extracted feature subspace is used to reconstruct the performance indicator tensor to obtain the performance indicator data after dimensionality reduction.

[0151] Next, a hierarchical progressive feature extraction network is constructed. The input of the network is the performance indicator data after dimensionality reduction. The network contains convolutional layers, recurrent layers, and attention layers. The convolutional layer is used to extract local spatiotemporal features, such as the combined features of CPU usage and memory occupancy on a specific server within a certain period of time. The recurrent layer is used to model temporal dependencies, such as the changing trend of process load over a period of time. The attention layer is used to capture the correlation between different resource indicators, such as the correlation between CPU usage and disk IO rate. The hidden layer output of the feature extraction network is used as the feature vector. For example, assuming that the hidden layer output of the feature extraction network is a 128-dimensional vector, each core monitoring process corresponds to a 128-dimensional feature vector.

[0152] Perform dual clustering on the feature vectors. First, use the local sensitive hashing algorithm for initial clustering to divide the feature vectors into several initial clusters. The local sensitive hashing algorithm is a fast approximate nearest neighbor search algorithm that can efficiently divide similar feature vectors into the same cluster. For example, the feature vectors of 10 core monitoring processes can be divided into 3 initial clusters. Then, the spectral clustering algorithm based on the adaptive kernel function is applied to the initial clusters for secondary partitioning to obtain the final clustering results. The spectral clustering algorithm is a clustering algorithm based on graph theory that can effectively process non-convex data. The adaptive kernel function can automatically adjust parameters according to the distribution of the data to improve the clustering effect. For example, the initial 3 clusters are further divided into 5 final clusters.

[0153] Calculate hierarchical load features based on clustering results. Hierarchical load features contain feature combinations of different time scales and different resource dimensions. For example, you can calculate the average CPU usage, memory usage, and disk IO rate of each cluster in the past 1 minute, 5 minutes, and 1 hour, as well as the variance, peak, and other statistical features of these indicators. These features can more comprehensively describe the load characteristics of each cluster.

[0154] A multi-task learning model is constructed using hierarchical load features. The model can simultaneously predict the resource demand trend, load level changes, and performance risk level of the core monitoring process. For example, a multi-layer neural network can be used as a multi-task learning model. The input of the model is the hierarchical load features, and the output is the CPU usage prediction value, load level prediction value, and performance risk prediction value for the next 1 minute, 5 minutes, and 1 hour.

[0155] Finally, the load characteristic deviation of the core monitoring process is monitored in real time in combination with the online anomaly detection algorithm. When the load characteristic deviation exceeds the adaptive deviation threshold, the load type of the core monitoring process is reclassified. For example, a sliding window can be used to calculate the mean and variance of the load characteristics of the core monitoring process, and the deviation threshold can be dynamically adjusted based on historical data. When the load characteristic deviation of a process exceeds the threshold, it is reclassified into a more appropriate load type.

[0156] In this embodiment, through three-dimensional tensor decomposition, hierarchical progressive feature extraction and dual clustering strategy, the load characteristics of the core monitoring process can be more effectively extracted and divided into more appropriate load types. The multi-task learning model can simultaneously predict the resource demand trend, load level changes and performance risk level of the core monitoring process, providing more refined guidance for resource management and performance optimization. The online anomaly detection and load type reclassification mechanism can enable the system to automatically adjust according to load changes, improving the system's adaptability and robustness.

[0157] In an optional implementation, the feature vector and the load type are input into a deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data, including:

[0158] Generate a corresponding one-hot encoding according to the load type, combine the feature vector and the one-hot encoding to form a state vector, and normalize the state vector to obtain a standardized state representation;

[0159] Constructing an action space for process optimization, the action space includes CPU scheduling priority, memory page replacement strategy, IO buffer size, and process migration target node, and discretizing each optimization parameter in the action space to obtain a discrete action set;

[0160] Construct a deep reinforcement learning model with a dual network structure, where the policy network adopts a multi-layer fully connected structure, the input layer receives the standardized state representation, and outputs the action probability distribution after processing by a modified linear activation function, and the value network adopts a multi-layer fully connected structure to output the state value estimation;

[0161] An adaptive reward function is constructed based on the load type, the CPU efficiency weight is increased for computationally intensive loads, the memory efficiency weight is increased for memory intensive loads, and the IO efficiency weight is increased for IO intensive loads, and the weighted sum of each efficiency index and the corresponding weight is used as the reward value of historical optimization experience;

[0162] The optimization action output by the execution strategy network interacts with the environment, records the historical optimization experience data consisting of the standardized state representation, the execution action and the reward value, and stores the historical optimization experience data in the experience replay pool;

[0163] Sampling historical optimization experience data from the experience replay pool, calculating the value estimate of each state in the sampled data using the value network, and calculating the advantage function based on the value estimate and the reward value;

[0164] The proximal policy optimization algorithm is used to update the policy network parameters, the advantage function is used to construct the clipping objective function for gradient update, and the mean square error between the value estimate and the actual return is used to update the value network parameters, so as to obtain a trained deep reinforcement learning model;

[0165] The feature vector and load type of the process to be optimized are input into the trained deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data.

[0166] Exemplarily, first, obtain the feature vector and load type of the process to be optimized. The feature vector may include indicators such as CPU usage, memory occupancy, and IO request times, and the load type may be divided into three types: compute-intensive, memory-intensive, and IO-intensive. For example, the feature vector of a process is [0.8, 0.5, 0.2], and the load type is compute-intensive.

[0167] Then, the load type is one-hot encoded. Compute-intensive is encoded as [1, 0, 0], memory-intensive is encoded as [0, 1, 0], and IO-intensive is encoded as [0, 0, 1]. In this example, the one-hot encoding of the compute-intensive load is [1, 0, 0].

[0168] Next, the feature vector and the one-hot encoding are combined to form the state vector. In this example, the state vector is [0.8, 0.5, 0.2, 1, 0, 0].

[0169] After that, the state vector is normalized to obtain a standardized state representation. For example, using the min-max normalization method, each eigenvalue is scaled to between 0 and 1. Assume that the normalized state vector is [0.9, 0.6, 0.3, 1, 0, 0].

[0170] Construct an action space for process optimization. The action space includes CPU scheduling priority (high, medium, low), memory page replacement strategy (FIFO, LRU), IO buffer size (small, medium, large), and process migration target node (node ​​1, node 2, node 3). Discretize these optimization parameters to obtain a discrete action set. For example, high CPU scheduling priority is encoded as 0, medium is encoded as 1, and low is encoded as 2; memory page replacement strategy FIFO is encoded as 0, LRU is encoded as 1; small IO buffer size is encoded as 0, medium is encoded as 1, and large is encoded as 2; process migration target node 1 is encoded as 0, node 2 is encoded as 1, and node 3 is encoded as 2.

[0171] Construct a deep reinforcement learning model with a dual network structure. The policy network adopts a multi-layer fully connected structure. The input layer receives the standardized state representation and outputs the action probability distribution after processing by the modified linear unit activation function. The value network adopts a multi-layer fully connected structure and outputs the state value estimation. For example, both the policy network and the value network contain two hidden layers, each containing 64 neurons.

[0172] Build an adaptive reward function based on the load type. For compute-intensive loads, increase the weight of CPU efficiency; for memory-intensive loads, increase the weight of memory efficiency; for IO-intensive loads, increase the weight of IO efficiency. The weighted sum of each efficiency indicator and the corresponding weight is used as the reward value of historical optimization experience. For example, the reward function for compute-intensive loads is: Reward = 0.8×CPU efficiency + 0.1×memory efficiency + 0.1×IO efficiency. Assuming that the CPU efficiency of a compute-intensive process is 0.9, the memory efficiency is 0.8, and the IO efficiency is 0.7, the reward value is 0.8×0.9+0.1×0.8+0.1×0.7=0.87.

[0173] The optimization actions output by the execution strategy network interact with the environment, record the historical optimization experience data consisting of standardized state representation, execution actions and reward values, and store these data in the experience replay pool. For example, the size of the experience replay pool is 10,000.

[0174] Sample historical optimization experience data from the experience replay pool, use the value network to calculate the value estimate of each state in the sampled data, and calculate the advantage function based on the value estimate and reward value.

[0175] The proximal policy optimization algorithm is used to update the policy network parameters, and the advantage function is used to construct the clipping objective function for gradient update. At the same time, the mean square error between the value estimation and the actual return is used to update the value network parameters to obtain a trained deep reinforcement learning model.

[0176] The feature vector and load type of the process to be optimized are input into the trained deep reinforcement learning model, and the model generates a differentiated process optimization strategy for the load type based on historical optimization experience data. For example, if the action probability distribution output by the model is [0.2, 0.5, 0.3, 0.1, 0.9, 0.0, 0.6, 0.2, 0.2, 0.1, 0.2, 0.7], the selected actions are CPU scheduling priority medium, memory page replacement strategy LRU, IO buffer size medium, and process migration target node 3.

[0177] In this embodiment, by learning from historical optimization experience, the model can formulate differentiated optimization strategies for different load types, so as to more effectively utilize resources such as CPU, memory and IO, avoid resource waste, and improve overall resource utilization. Differentiated optimization strategies can better meet the needs of different load types, assign higher CPU priority to compute-intensive processes, and assign larger IO buffers to IO-intensive processes, thereby improving the execution efficiency of processes and enhancing the performance of the entire system. The optimization strategy is automatically generated through the deep reinforcement learning model without manual intervention, which reduces the workload of manual tuning, simplifies the process optimization process, and improves operation and maintenance efficiency.

[0178] Figure 2 FIG. 1 is a schematic diagram of a Python monitoring task resource usage system according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0179] The first unit is used to obtain the runtime status data of the monitoring server through the Python system call interface, extract the process identification code, process startup timestamp and process call stack information from the runtime status data, perform hash matching on the process identification code and the pre-built target task feature library to obtain the target task process, extract call features from the process call stack information using a multi-layer convolutional neural network to obtain process dependency data, construct a process dependency graph based on the process dependency data using a minimum spanning tree algorithm, calculate the node degree centrality of the nodes in the process dependency graph to obtain a process criticality score, determine the target task process whose process criticality score is greater than a preset criticality threshold as a core monitoring process, and write the process identification code of the core monitoring process into a monitoring collection object pool;

[0180] The second unit is used to create a resource monitoring coroutine for each core monitoring process in the monitoring collection object pool, the resource monitoring coroutine obtains the resource usage data of the core monitoring process through the system call interface, calculates the process resource volatility of the resource usage data, substitutes the process resource volatility into the adaptive feedback control algorithm to calculate the optimal sampling period, writes the resource usage data into a time series database according to the optimal sampling period, uses a sliding time window algorithm to perform aggregation analysis on the data in the time series database to obtain performance indicators, builds an anomaly detection model according to the performance indicators, and uses the principal component analysis method to reduce the dimension of the performance indicators to construct a feature vector, and uses a clustering algorithm based on the feature vector to divide the core monitoring process into different load types;

[0181] The third unit is used to input the feature vector and load type into the deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data. When the anomaly detection model detects performance fluctuations, the differentiated process optimization strategy is executed, the optimized resource usage data is obtained, and the performance improvement index of the optimized resource usage data relative to the data before optimization is calculated. When the performance improvement index is greater than a preset optimization threshold, the parameters of the anomaly detection model are updated, an optimization diagnosis report is generated, and the optimization strategy, performance improvement index and load type are written into the training sample library of the deep reinforcement learning model for continuous optimization.

[0182] According to a third aspect of the embodiments of the present invention,

[0183] An electronic device is provided, comprising:

[0184] processor;

[0185] a memory for storing processor-executable instructions;

[0186] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0187] A fourth aspect of the embodiments of the present invention is:

[0188] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0189] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Python monitoring task resource usage method, characterized in that: include: The runtime status data of the monitoring server is obtained through the Python system call interface, and the process identification code, process startup timestamp and process call stack information are extracted from the runtime status data. The process identification code is hash matched with the pre-built target task feature library to obtain the target task process, and the call features of the process call stack information are extracted using a multi-layer convolutional neural network to obtain process dependency data. Based on the process dependency data, a process dependency graph is constructed using a minimum spanning tree algorithm, and the node degree centrality of the nodes in the process dependency graph is calculated to obtain a process criticality score. The target task process whose process criticality score is greater than a preset criticality threshold is determined as a core monitoring process, and the process identification code of the core monitoring process is written into the monitoring collection object pool; A resource monitoring coroutine is created for each core monitoring process in the monitoring collection object pool. The resource monitoring coroutine obtains resource usage data of the core monitoring process through a system call interface, calculates the process resource volatility of the resource usage data, substitutes the process resource volatility into an adaptive feedback control algorithm to calculate an optimal sampling period, writes the resource usage data into a time series database according to the optimal sampling period, uses a sliding time window algorithm to perform aggregation analysis on the data in the time series database to obtain performance indicators, builds an anomaly detection model based on the performance indicators, uses principal component analysis to reduce the dimension of the performance indicators to construct a feature vector, and uses a clustering algorithm based on the feature vector to divide the core monitoring process into different load types; The feature vector and load type are input into a deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data. When the anomaly detection model detects performance fluctuations, the differentiated process optimization strategy is executed, the optimized resource usage data is obtained, and the performance improvement index of the optimized resource usage data relative to the data before optimization is calculated. When the performance improvement index is greater than a preset optimization threshold, the parameters of the anomaly detection model are updated, an optimization diagnosis report is generated, and the optimization strategy, performance improvement index and load type are written into the training sample library of the deep reinforcement learning model for continuous optimization.

2. The method according to claim 1, characterized in that The target task process is obtained by performing hash matching on the process identification code and the pre-built target task feature library, including: A sliding time window is used to extract features from the process identification code, the sliding time window is divided into multiple time segments, and a weighted combination of feature vectors in each time segment is calculated based on an exponential decay function to obtain an initial time series feature matrix; Based on the data distribution characteristics of the initial time series feature matrix, multi-scale overlapping sharding is performed, the optimal sharding parameters and the size of the overlapping area are determined by calculating the feature density, and multiple groups of local sensitive hash function families are used to map the feature shards to generate a multi-dimensional hash mapping sequence, wherein the parameters of the hash function family are determined by feature similarity constraints; The multidimensional hash mapping sequence is used to construct a spatiotemporal feature association network, the local similarity and temporal migration features between adjacent shards are calculated, the local similarity and temporal migration features are combined to form edge weights, the path features in the network are extracted through deep walking, and the hierarchical feature fingerprint is generated by combining the single point features and the path features; A multi-level cache feature library is constructed according to the access frequency and timeliness of the hierarchical feature fingerprints, and a feature index forest is established in each level of cache. The nodes of the feature index forest store feature fingerprints, associated network structures, and adaptive matching thresholds, and search priorities are assigned to nodes based on the accuracy of historical matching results and feature stability; According to the search priority, a parallel search path is planned in the feature index forest, and a multi-path search is performed on the feature fingerprint to be matched. The comprehensive similarity between the node and the feature to be matched is calculated on the search path. The comprehensive similarity includes the feature vector distance, the network structure similarity and the timing pattern matching degree. When the comprehensive similarity of the node exceeds the adaptive matching threshold, the process corresponding to the current node is determined as the target task process.

3. The method according to claim 1, characterized in that The process call stack information is extracted using a multi-layer convolutional neural network to obtain call features to obtain process dependency data, a process dependency graph is constructed based on the process dependency data using a minimum spanning tree algorithm, the node degree centrality of the nodes in the process dependency graph is calculated to obtain a process criticality score, and the target task process whose process criticality score is greater than a preset criticality threshold is determined as a core monitoring process, including: The process call stack information is segmented using a sliding time window, the process call stack information includes the caller process ID, the callee process ID, the call timestamp and the call depth, the process call frequency, the call level and the resource occupancy features are extracted from each time window, and the extracted features are weighted and combined based on the time series correlation to obtain a time series feature matrix; The time series feature matrix is ​​input into a multi-layer convolutional neural network, which includes a feature extraction layer, a pattern fusion layer and a relationship modeling layer, wherein the feature extraction layer includes a plurality of convolution kernels of different sizes, each convolution kernel performs a convolution operation on the time series feature matrix to obtain a feature map of a corresponding scale, and each feature map is combined to form a multi-scale feature; the pattern fusion layer performs a weighted calculation on each feature component in the multi-scale feature based on the attention weight to obtain a feature fusion vector; the relationship modeling layer uses a graph convolutional network to process the feature fusion vector, and obtains a process call feature vector based on the call association between processes; Based on the process call feature vector, the call pattern similarity, resource occupancy similarity and timing behavior similarity between processes are calculated, the call pattern similarity, resource occupancy similarity and timing behavior similarity are weightedly combined using adaptive weights to obtain process similarity, and the process similarity is converted into a distance metric to construct a process affinity graph; Running a minimum spanning tree algorithm based on the process affinity graph, introducing node importance constraints into the minimum spanning tree algorithm and dynamically adjusting the edge weight selection strategy based on the node importance constraints, maintaining the connectivity of process nodes through a union-find structure, and selecting a connected subgraph with a minimum edge weight combination as a process dependency graph; In the process dependency graph, the node degree centrality of each process node is calculated to obtain a local centrality index, a global centrality index is calculated based on the shortest path between process nodes, and the local centrality index and the global centrality index are weighted and normalized to obtain a process criticality score; The system load status and historical recognition accuracy are obtained from the criticality threshold calculation unit which includes two layers of neural networks, wherein the first layer of the neural network maps the system load status and the historical recognition accuracy into dynamic adjustment factors, and the second layer of the neural network modifies the preset criticality threshold based on the dynamic adjustment factor to obtain the current criticality threshold, and determines the target task process that is higher than the current criticality threshold as the core monitoring process.

4. The method according to claim 3, characterized in that Running a minimum spanning tree algorithm based on the process affinity graph, introducing node importance constraints in the minimum spanning tree algorithm and dynamically adjusting the edge weight selection strategy based on the node importance constraints, maintaining the connectivity of process nodes through a union-find structure, and selecting a connected subgraph with a minimum edge weight combination as a process dependency graph includes: The process node degree centrality is obtained by counting the number of adjacent edges of each process node in the process affinity graph, the process node betweenness centrality is obtained by calculating the proportion of paths passing through each process node in all shortest paths, and the process node proximity centrality is obtained based on the shortest distance from each process node to other process nodes; The process node degree centrality, process node betweenness centrality and process node closeness centrality are weighted and combined by an importance weight coefficient to obtain a node importance constraint index of each process node; Extract the initial edge weight of each edge in the process affinity graph, obtain the node importance constraint index of the process nodes at both ends of each edge, select the smaller node importance constraint index of the process nodes at both ends as the importance constraint value of the edge, and calculate the edge weight adjustment factor according to the importance constraint value of the edge, wherein the edge weight adjustment factor is obtained by subtracting the product of the importance constraint value of the edge and the preset weight adjustment coefficient from the preset reference value; Multiply the initial edge weight of each edge by the corresponding edge weight adjustment factor to obtain the adjusted edge weight, and sort all edges in ascending order according to the adjusted edge weights to obtain a sequence of edges to be selected; Create and query a data structure, including a parent node array and a level array, wherein the parent node array records the connected component number to which each process node belongs, and the level array records the level value of each connected component; Traversing the sequence of edges to be selected in sequence, for each edge, searching the connected component numbers of the process nodes at both ends of the edge through the parent node array in the union-find set structure, when the process nodes at both ends of the edge do not belong to the same connected component, calculating the sum of the node importance constraint indexes of the newly covered process nodes after the current edge is added, and performing a weighted combination with the sum of the adjusted edge weights of the currently selected edge set to obtain a connected subgraph importance score; The importance score of the connected subgraph is compared with the current optimal score. When the importance score of the connected subgraph is better, the current edge is added to the final edge set, and the connectivity relationship is updated in the parent node array and level array of the union-find set, and the final edge set with the minimum edge weight combination is constructed as a process dependency graph.

5. The method according to claim 1, characterized in that Calculating the process resource fluctuation rate of the resource usage data, and substituting the process resource fluctuation rate into the adaptive feedback control algorithm to calculate the optimal sampling period includes: Calculate the change rate of resource usage data at adjacent moments, obtain original fluctuation characteristic data by calculating the absolute value of the relative change rate, process the original fluctuation characteristic data by using an exponential smoothing model, attenuate the noise data based on weighted calculation of the current sampling value and the historical cumulative value, obtain smoothed resource fluctuation data, and store the smoothed resource fluctuation data in a fluctuation characteristic set; Based on the fluctuation feature set, information entropy in three dimensions, namely, CPU usage, memory usage, and I / O operation count, is calculated, the information entropy is normalized to obtain weight coefficients of each dimension, and the smoothed resource fluctuation values ​​are weighted and integrated using the weight coefficients to obtain a process resource fluctuation rate that characterizes the overall fluctuation degree of the process; Construct a state space model, take the process resource volatility and the current sampling period as state variables, take the sampling period adjustment amount as control input, establish the system state equation and output equation; design a performance indicator function, the performance indicator function includes a quadratic term of the process resource volatility deviation and a quadratic term of the sampling period adjustment cost; An algebraic Riccati equation is constructed based on the system state equation and the performance indicator function, the optimal feedback gain matrix is ​​obtained by solving the algebraic Riccati equation, the process resource volatility is substituted into the state equation, and the adjustment amount of the sampling period is calculated according to the optimal feedback gain matrix; An optimization objective function is constructed according to the adjustment amount. Under the sampling period constraint, the optimization objective function is solved by the gradient projection method. The candidate sampling periods that meet the constraints are obtained through iterative calculation. The convergence of the candidate sampling periods is judged. When the iterative difference meets the threshold requirement, the converged sampling period is determined as the optimal sampling period.

6. The method according to claim 1, characterized in that The principal component analysis method is used to reduce the dimension of the performance index to construct a feature vector, and a clustering algorithm is used based on the feature vector to divide the core monitoring process into different load types, including: Organize the performance indicator data according to the time dimension, space dimension and resource type dimension, construct the performance indicator tensor of the core monitoring process, perform three-dimensional tensor decomposition on the performance indicator tensor, extract the characteristic subspace of the time dimension, space dimension and resource type dimension, and reconstruct the reduced dimension performance indicator based on the characteristic subspace; Based on the dimensionality reduction performance index, a hierarchical progressive feature extraction network is constructed, local spatiotemporal features are extracted through the convolution layer of the hierarchical progressive feature extraction network, temporal dependencies are modeled through the recurrent layer, correlations between different resource indicators are captured through the attention layer, and the hidden layer output of the feature extraction network is constructed as a feature vector; A dual clustering strategy is adopted for the feature vectors, initial clustering is performed based on local sensitive hashing to obtain initial clusters, and then a spectral clustering algorithm based on an adaptive kernel function is applied to the initial clusters for secondary partitioning to obtain clustering results; Calculating hierarchical load characteristics based on the clustering results, wherein the hierarchical load characteristics include characteristic combinations of different time scales and different resource dimensions; Using the hierarchical load characteristics to build a multi-task learning model, and simultaneously predict the resource demand trend, load level change, and performance risk level of the core monitoring process; Combined with the online anomaly detection algorithm, the load characteristic deviation degree of the core monitoring process is monitored in real time. When the load characteristic deviation degree exceeds the adaptive deviation threshold, the load type of the core monitoring process is reclassified, and the core monitoring process is divided into different load types.

7. The method according to claim 1, characterized in that Inputting the feature vector and the load type into a deep reinforcement learning model, wherein the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data, including: Generate a corresponding one-hot encoding according to the load type, combine the feature vector and the one-hot encoding to form a state vector, and normalize the state vector to obtain a standardized state representation; Constructing an action space for process optimization, the action space includes CPU scheduling priority, memory page replacement strategy, IO buffer size, and process migration target node, and discretizing each optimization parameter in the action space to obtain a discrete action set; Construct a deep reinforcement learning model with a dual network structure, where the policy network adopts a multi-layer fully connected structure, the input layer receives the standardized state representation, and outputs the action probability distribution after processing by a modified linear activation function, and the value network adopts a multi-layer fully connected structure to output the state value estimation; An adaptive reward function is constructed based on the load type, the CPU efficiency weight is increased for computationally intensive loads, the memory efficiency weight is increased for memory intensive loads, and the IO efficiency weight is increased for IO intensive loads, and the weighted sum of each efficiency index and the corresponding weight is used as the reward value of historical optimization experience; The optimization action output by the execution strategy network interacts with the environment, records the historical optimization experience data consisting of the standardized state representation, the execution action and the reward value, and stores the historical optimization experience data in the experience replay pool; Sampling historical optimization experience data from the experience replay pool, calculating the value estimate of each state in the sampled data using the value network, and calculating the advantage function based on the value estimate and the reward value; The proximal policy optimization algorithm is used to update the policy network parameters, the advantage function is used to construct the clipping objective function for gradient update, and the mean square error between the value estimate and the actual return is used to update the value network parameters, so as to obtain a trained deep reinforcement learning model; The feature vector and load type of the process to be optimized are input into the trained deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data.

8. A Python monitoring task resource usage system, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain the runtime status data of the monitoring server through the Python system call interface, extract the process identification code, process startup timestamp and process call stack information from the runtime status data, perform hash matching on the process identification code and the pre-built target task feature library to obtain the target task process, extract call features from the process call stack information using a multi-layer convolutional neural network to obtain process dependency data, construct a process dependency graph based on the process dependency data using a minimum spanning tree algorithm, calculate the node degree centrality of the nodes in the process dependency graph to obtain a process criticality score, determine the target task process whose process criticality score is greater than a preset criticality threshold as a core monitoring process, and write the process identification code of the core monitoring process into a monitoring collection object pool; The second unit is used to create a resource monitoring coroutine for each core monitoring process in the monitoring collection object pool, the resource monitoring coroutine obtains the resource usage data of the core monitoring process through the system call interface, calculates the process resource volatility of the resource usage data, substitutes the process resource volatility into the adaptive feedback control algorithm to calculate the optimal sampling period, writes the resource usage data into a time series database according to the optimal sampling period, uses a sliding time window algorithm to perform aggregation analysis on the data in the time series database to obtain performance indicators, builds an anomaly detection model according to the performance indicators, and uses the principal component analysis method to reduce the dimension of the performance indicators to construct a feature vector, and uses a clustering algorithm based on the feature vector to divide the core monitoring process into different load types; The third unit is used to input the feature vector and load type into the deep reinforcement learning model, and the deep reinforcement learning model generates a differentiated process optimization strategy for the load type based on historical optimization experience data. When the anomaly detection model detects performance fluctuations, the differentiated process optimization strategy is executed, the optimized resource usage data is obtained, and the performance improvement index of the optimized resource usage data relative to the data before optimization is calculated. When the performance improvement index is greater than a preset optimization threshold, the parameters of the anomaly detection model are updated, an optimization diagnosis report is generated, and the optimization strategy, performance improvement index and load type are written into the training sample library of the deep reinforcement learning model for continuous optimization.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images