A computing power demand prediction method and system based on big data processing

Abstract

The application discloses a kind of based on big data processing's computing power demand prediction method and system, comprising: step one: the task running data of computing power platform, resource monitoring data and scheduling data are collected, and computing power request fragment is constructed;Step two: resource state feature is extracted and computing power demand fingerprint vector is constructed;Step three: computing power demand fingerprint trajectory chain is constructed;Step four: calculate fingerprint drift feature and determine computing power demand drift node;Step five: retrieve target historical fingerprint fragment;Step six: construct computing power evolution potential sequence and potential diffusion matrix;Step seven: potential diffusion matrix is input into improved DLinear model to perform drift modulation and linear prediction, and obtain potential diffusion matrix prediction sequence;Step eight: determine the computing power demand prediction result corresponding to future time window.The application realizes the high-precision prediction of computing power demand.

Description

Technical Field

[0001] This invention relates to the field of cloud computing and computing resource management technology, and in particular to a method and system for predicting computing demand based on big data processing. Background Technology

[0002] With the rapid development of cloud computing, artificial intelligence training, large-scale model inference, and high-performance computing services, the scale of tasks carried by computing platforms continues to grow, and the types of resources have gradually expanded from a single central processing unit to various heterogeneous resources such as graphics processors, video memory, RAM, and containerized resources. Computing demand exhibits characteristics of frequent dynamic changes, complex resource relationships, and significant load fluctuations. To improve resource utilization efficiency and ensure stable business operation, computing demand forecasting has become an important research direction in the field of cloud computing and computing resource management. Existing technologies typically construct time-series forecasting models based on historical resource usage data, task execution data, or monitoring indicators, and use statistical analysis methods or machine learning models to predict future resource demands. However, most existing technologies use fixed time windows to collect and model resource data, making it difficult to accurately reflect the dynamic changes in computing power demand at different operational stages. Furthermore, existing methods typically use single indicators such as resource utilization and resource occupancy as prediction targets, lacking the ability to jointly represent task load structure, resource supply and demand relationships, and resource evolution states, resulting in insufficient feature representation capabilities in complex computing scenarios. For scenarios where resource demand undergoes phased changes or abrupt changes, existing prediction models often rely on continuous time series for prediction, lacking mechanisms for identifying and utilizing resource demand drift behavior, making it difficult to effectively uncover similar evolutionary patterns in historical processes. In addition, existing time series prediction models typically model resource indicator sequences directly, lacking the ability to structurally represent the relationships between multidimensional resource states and the resource evolution process, making it difficult for prediction results to accurately reflect the collaborative change characteristics between heterogeneous resources.

[0003] Therefore, how to provide a method and system for predicting computing power demand based on big data processing is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a method and system for predicting computing power demand based on big data processing. This invention achieves a structured representation of the evolution process of computing power demand by constructing a computing power demand fingerprint, fingerprint trajectory chain, and drift nodes; it enhances the ability to model heterogeneous resource relationships by combining a historical fingerprint fragment retrieval mechanism, computing power evolution potential energy sequence, and potential energy diffusion matrix; and it introduces an improved DLinear model with drift-aware potential energy decomposition, drift modulation, and drift feedback mechanisms to achieve accurate prediction of sudden changes in resource demand and load fluctuations, thereby improving the accuracy of computing power demand prediction for future time windows.

[0005] A method for predicting computing power demand based on big data processing according to an embodiment of the present invention includes the following steps: Step 1: Collect task execution data, resource monitoring data, and scheduling data from the computing power platform, dynamically divide time windows based on changes in resource status, and construct computing power request segments; Step 2: Extract resource status features based on the computing power request fragment, and construct a computing power demand fingerprint vector based on the resource status features; Step 3: Connect the multiple computing power demand fingerprint vectors in chronological order to construct a computing power demand fingerprint trajectory chain; Step 4: Calculate the fingerprint drift features based on the computing power requirement fingerprint trajectory chain, and determine the computing power requirement drift nodes according to the fingerprint drift features; Step 5: Based on the computing power demand fingerprint trajectory chain and computing power demand drift nodes, retrieve the target historical fingerprint fragment in the historical fingerprint sample database; Step 6: Construct a computing power evolution potential energy sequence based on the target historical fingerprint fragment and computing power demand fingerprint trajectory chain, and construct a potential energy diffusion matrix based on the computing power evolution potential energy sequence; Step 7: Input the potential energy diffusion matrix into the improved DLinear model, perform drift modulation processing on the potential energy diffusion matrix based on the computing power demand drift node, and perform linear prediction on the modulated potential energy diffusion matrix to obtain the potential energy diffusion matrix prediction sequence. Step 8: Determine the computing power demand prediction result corresponding to the future time window based on the predicted sequence of the potential energy diffusion matrix.

[0006] Optionally, step one includes: The system collects task submission records, task start records, task completion records, node resource usage records, container start records, and scheduling event records from the computing power platform during continuous operation. It also extracts the CPU resource request amount, GPU resource request amount, video memory request amount, memory request amount, task queue length, and number of task scheduling failures for each task. The collected data is divided into time slices according to time sequence. Within each time slice, the total number of task submissions, total CPU resource requests, total GPU resource requests, total video memory requests, total memory requests, and total number of scheduling failures are counted. The difference between the total number of task submissions in the current time slice and the previous time slice is calculated to form the task submission change rate. The difference between the total number of resource requests in the current time slice and the previous time slice is calculated to form the resource request change rate. The difference between the total number of scheduling failures in the current time slice and the previous time slice is calculated to form the scheduling status change rate. Based on the task submission change rate, resource request change rate, and scheduling status change rate, a change rate sequence is constructed. Time nodes in the change rate sequence whose change amplitude exceeds the average change amplitude of adjacent time slices are identified. The identified time nodes are used as the boundaries of the dynamic time window to form a continuous and non-overlapping dynamic time window. The task submission records, node resource usage records, and scheduling event records are aggregated according to the dynamic time window. The total number of task submissions, total number of CPU resource requests, total number of GPU resource requests, total number of video memory requests, total number of memory requests, total task queue length, and total number of scheduling failures within each dynamic time window are counted to construct the computing power request segment for the corresponding dynamic time window.

[0007] Optionally, step two includes: Read the total number of tasks submitted, the total number of CPU resource requests, the total number of GPU resource requests, the total number of video memory requests, the total number of memory requests, the total length of the task queue, and the total number of scheduling failures from each of the computing power request segments; Divide the total length of the task queue by the total number of task submissions to calculate the task congestion characteristics. Divide the total number of graphics processor resource requests by the total number of central processing unit resource requests to calculate resource skew characteristics. Divide the total amount of video memory requested by the total amount of graphics processor resources requested to calculate the video memory coupling characteristics. Divide the total number of scheduling failures by the total number of task submissions to calculate the scheduling blocking characteristics; The proportions of total CPU resource requests, total GPU resource requests, total video memory requests, and total RAM requests to the total resource requests are calculated separately, and the resource imbalance characteristics are calculated based on the differences between the proportions of each resource request. Calculate the difference in total task submissions between the current computing power request segment and the adjacent preceding computing power request segment, and calculate the task fluctuation characteristics based on the difference in total task submissions; The characteristics of task congestion, resource skew, memory coupling, scheduling blockage, resource imbalance, and task fluctuation are normalized and combined in a unified order to construct the computing power demand fingerprint vector of the corresponding computing power request segment.

[0008] Optionally, step three includes: According to the time sequence corresponding to the dynamic time window, multiple computing power demand fingerprint vectors are sorted to form a fingerprint vector sequence; Taking the current computing power demand fingerprint vector and the adjacent preceding computing power demand fingerprint vector as the analysis objects, the differences between the corresponding feature values ​​of task congestion feature, resource skew feature, memory coupling feature, scheduling blocking feature, resource imbalance feature and task fluctuation feature are calculated respectively to form the feature change quantity corresponding to each dimension feature. Based on the feature change amount corresponding to each dimension, a state transition record is constructed between the corresponding fingerprint vectors. Each state transition record includes the starting fingerprint vector identifier, the target fingerprint vector identifier, and the feature change amount corresponding to each dimension. The state transition records are connected sequentially in chronological order, and the state transition relationships between the corresponding fingerprint vectors are written into the fingerprint vector sequence to form the fingerprint evolution path. The cumulative feature changes and consistency of change direction corresponding to multiple consecutive state transition records in the fingerprint evolution path are statistically analyzed to construct the evolution state information corresponding to each fingerprint vector. The fingerprint vector sequence, state transition record, fingerprint evolution path, and evolution state information are associated and stored, and a corresponding trajectory chain index identifier is established to construct a computing power requirement fingerprint trajectory chain.

[0009] Optionally, step four includes: Read the state transition records, evolution state information, and trajectory chain index identifiers from the computing power demand fingerprint trajectory chain; For each state transition record, the feature change amounts corresponding to task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features are obtained respectively. The absolute values ​​of the feature change amounts of each dimension are accumulated and normalized according to the number of feature dimensions involved in the calculation to form the state change intensity. The number of times the same feature change direction remains consistent in continuous state transition records is counted, and the proportion of the number of consistent changes to the total number of corresponding continuous state transition records is calculated to form the consistency of change direction. Extract the changes in resource imbalance characteristics from the evolutionary state information, and construct the degree of resource imbalance based on the changes in resource imbalance characteristics corresponding to the continuous state transition records; Fingerprint drift features are constructed based on the intensity of state changes, the consistency of change direction, and the degree of resource imbalance. Based on the time sequence corresponding to the trajectory chain index, the drift features of multiple consecutive fingerprints are analyzed to identify the corresponding positions where the intensity of state change and the degree of resource imbalance increase synchronously and the consistency of the change direction is consistently higher than the average consistency of the corresponding consecutive state transition records. The fingerprint vector of the corresponding position is then identified as the computing power demand drift node.

[0010] Optionally, step five includes: Read the fingerprint evolution path, evolution status information, and computing power demand drift nodes in the computing power demand fingerprint trajectory chain; Starting from the computing power demand drift node, extract the continuous state transition records associated with the computing power demand drift node to construct the current evolution segment; Extract historical evolution fragments corresponding to historical fingerprint trajectory chains from the historical fingerprint sample database, and read the state transition records and evolution state information corresponding to each historical evolution fragment; Calculate the differences in the intensity of state changes, the consistency of change direction, and the degree of resource imbalance between the current evolutionary segment and each historical evolutionary segment; The differences in the intensity of state changes, the consistency of change directions, and the degree of resource imbalance are normalized, and then combined according to the proportions of each difference to form an evolutionary matching degree. The historical evolution segments are sorted according to their evolution matching degree. The historical fingerprint segment with the highest evolution matching degree and whose corresponding drift node evolution direction is consistent with the current computing power requirement drift node evolution direction is selected as the target historical fingerprint segment.

[0011] Optionally, step six includes: Read the state transition records, evolution state information, and evolution information after the drift node corresponding to the target historical fingerprint fragment, and read the state transition records and evolution state information corresponding to the computing power requirement fingerprint trajectory chain; The state change intensity sequence is constructed by arranging the state change intensity corresponding to the target historical fingerprint segment and the state change intensity corresponding to the computing power demand fingerprint trajectory chain in chronological order. The resource imbalance degree corresponding to the target historical fingerprint segment and the resource imbalance degree corresponding to the computing power demand fingerprint trajectory chain are arranged in chronological order to construct a resource imbalance evolution sequence. The state change intensity sequence and the resource imbalance evolution sequence are weighted and merged to construct the computing power evolution potential energy sequence. Each potential energy value in the computing power evolution potential energy sequence corresponds to the combination result of the state change intensity and the degree of resource imbalance at the same time position. Based on the computing power evolution potential energy sequence, the feature proportions corresponding to task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features are mapped to the potential energy values ​​at each time position, forming the potential energy components corresponding to each feature. Using the potential energy components corresponding to each feature as matrix elements, matrix row and column indices are established according to the evolutionary relationship between task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features. A potential energy diffusion matrix is ​​constructed, and the values ​​of the matrix elements correspond to the potential energy intensity transferred from the matrix row index features to the matrix column index features.

[0012] Optionally, step seven includes: Input the potential energy diffusion matrix into the improved DLinear model; The improved DLinear model includes a drift-sensing potential energy decomposition module, a drift modulation module, a two-way path prediction module, a drift feedback gate module, and a potential energy fusion module. The drift sensing potential energy decomposition module takes the potential energy diffusion matrix and the computing power demand drift node as input, and reads the matrix elements at each time position in the potential energy diffusion matrix and the state change intensity, change direction consistency and resource imbalance degree of the drift node corresponding to the state change intensity, change direction consistency and resource imbalance degree of the drift node. The potential energy evolution interval is divided according to the time position corresponding to the drift node; Perform a moving average process on each interval matrix element to construct a stable potential energy component, and calculate the residual between the potential energy diffusion matrix and the stable component to form a potential energy drift component. Output the stable potential energy component and the potential energy drift component. The drift modulation module takes potential energy drift components and drift nodes as inputs, and includes: The normalization results are used to determine the intensity and direction of state changes, as well as the degree of resource imbalance, and to construct drift modulation coefficients based on the normalization results. The drift enhancement component is generated and output by weighting the potential energy drift component with the drift modulation coefficient; The dual-path prediction module takes the stable potential energy component and the drift enhancement component as inputs to generate stable potential energy prediction results and drift potential energy prediction results, respectively. The drift feedback gate module takes the stable potential energy prediction result and the drift potential energy prediction result as input, calculates the change amplitude of the drift potential energy prediction result, and generates a drift feedback weight in combination with the drift modulation coefficient to adjust the stable potential energy prediction result and output the feedback stable potential energy prediction result. The potential energy fusion module takes the feedback stable potential energy prediction results and the drift potential energy prediction results as inputs, and generates a potential energy diffusion matrix prediction sequence by fusing them according to time position.

[0013] Optionally, step eight includes: Read the potential energy diffusion matrix corresponding to each time position in the predicted sequence; Based on the matrix row and column indices corresponding to task congestion characteristics, resource skew characteristics, memory coupling characteristics, scheduling blocking characteristics, resource imbalance characteristics, and task fluctuation characteristics, the potential energy transfer intensity corresponding to each characteristic is extracted. Accumulate the potential energy transfer intensity corresponding to the same feature to construct a sequence of feature potential energy values; The contribution values ​​of graphics processor (GPU) resource demand and central processing unit (CPU) resource demand are calculated based on the characteristic potential energy value sequence corresponding to the resource skew characteristics. Combined with the total number of GPU resource requests and CPU resource requests corresponding to the current time window, the predicted values ​​of GPU resource demand and CPU resource demand are determined. The contribution value of video memory resource demand is calculated based on the characteristic potential energy value sequence corresponding to the video memory coupling characteristics, and the predicted value of video memory resource demand is determined by combining the total amount of video memory application corresponding to the current time window. The task load contribution value is calculated based on the characteristic potential value sequence corresponding to the task congestion characteristics, scheduling blocking characteristics, and task fluctuation characteristics. Combined with the total task queue length and the number of container instances corresponding to the current time window, the predicted values ​​of task queue length and container instance number are determined. The predicted values ​​for graphics processor resource requirements, central processing unit resource requirements, video memory resource requirements, task queue length, and container instance quantity are correlated and output to form the computing power demand prediction results for the future time window.

[0014] A computing power demand prediction system based on big data processing according to an embodiment of the present invention includes: The computing power request fragment construction module is used to collect task operation data, resource monitoring data and scheduling data from the computing power platform, dynamically divide time windows according to changes in resource status, and construct computing power request fragments. The computing power demand fingerprint construction module is used to extract resource status features based on the computing power request fragment, and construct a computing power demand fingerprint vector based on the resource status features. The fingerprint trajectory chain construction module is used to connect multiple computing power demand fingerprint vectors in chronological order to construct a computing power demand fingerprint trajectory chain. A drift node identification module is used to calculate fingerprint drift features based on the computing power requirement fingerprint trajectory chain, and determine computing power requirement drift nodes according to the fingerprint drift features; The historical fingerprint retrieval module is used to retrieve target historical fingerprint fragments from the historical fingerprint sample library based on the computing power demand fingerprint trajectory chain and computing power demand drift nodes. The potential energy modeling module is used to construct a computing power evolution potential energy sequence based on the target historical fingerprint fragment and computing power demand fingerprint trajectory chain, and to construct a potential energy diffusion matrix based on the computing power evolution potential energy sequence. The potential energy prediction module is used to input the potential energy diffusion matrix into the improved DLinear model, perform drift modulation processing on the potential energy diffusion matrix based on the computing power demand drift node, and perform linear prediction on the modulated potential energy diffusion matrix to obtain the potential energy diffusion matrix prediction sequence. The computing power demand output module is used to determine the computing power demand prediction result corresponding to the future time window based on the potential energy diffusion matrix prediction sequence.

[0015] The beneficial effects of this invention are: This invention addresses the shortcomings of existing computing power demand prediction methods, such as insufficient resource state representation, difficulty in identifying demand drift behavior, and difficulty in modeling heterogeneous resource relationships, by constructing a computing power demand fragment generation mechanism based on dynamic time windows and combining the collaborative design of computing power demand fingerprint vectors, computing power demand fingerprint trajectory chains, and computing power demand drift nodes. It proposes a computing power demand representation strategy based on resource state feature extraction, fingerprint evolution modeling, and drift feature analysis, significantly improving the ability to characterize the changing process of computing power demand and uncover its evolutionary patterns. In the historical information utilization stage, a historical fingerprint fragment retrieval mechanism is introduced, achieving correlation alignment between the historical evolution process and the current evolution process through evolutionary state matching and drift direction consistency constraints. In the potential energy modeling stage, a computing power evolution potential energy sequence and potential energy diffusion matrix are constructed, utilizing… A structured potential energy representation is established by considering the intensity of state changes, the degree of resource imbalance, and the evolutionary relationship between multidimensional resource state characteristics, effectively enhancing the expressive power of the heterogeneous resource co-evolution process. In the prediction stage, an improved DLinear model is introduced. Through the synergistic effect of the drift-aware potential energy decomposition module, drift modulation module, and drift feedback gate module, the model achieves separate modeling and dynamic correction of the stable and drift components of potential energy, improving its adaptability to sudden changes in resource demand, resource structure migration, and complex load fluctuation scenarios. Finally, based on the potential energy diffusion matrix prediction sequence, predicted values ​​for graphics processor resource demand, central processing unit resource demand, video memory resource demand, task queue length, and container instance quantity are generated, achieving high-precision prediction of computing power demand in future time windows. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of a computing power demand prediction method based on big data processing proposed in this invention; Figure 2 This is a schematic diagram of the structure of a computing power demand prediction system based on big data processing proposed in this invention. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0018] refer to Figure 1 A method for predicting computing power demand based on big data processing includes the following steps: Step 1: Collect task execution data, resource monitoring data, and scheduling data from the computing power platform, dynamically divide time windows based on changes in resource status, and construct computing power request segments; Step 2: Extract resource status features based on computing power request fragments, and construct a computing power demand fingerprint vector based on the resource status features; Step 3: Connect multiple computing power demand fingerprint vectors in chronological order to construct a computing power demand fingerprint trajectory chain; Step 4: Calculate fingerprint drift features based on the fingerprint trajectory chain of computing power requirements, and determine the computing power requirement drift nodes based on the fingerprint drift features; Step 5: Based on the computing power demand fingerprint trajectory chain and computing power demand drift nodes, retrieve the target historical fingerprint fragment in the historical fingerprint sample database; Step 6: Construct a computing power evolution potential energy sequence based on the target historical fingerprint fragments and computing power demand fingerprint trajectory chain, and construct a potential energy diffusion matrix based on the computing power evolution potential energy sequence; Step 7: Input the potential energy diffusion matrix into the improved DLinear model, perform drift modulation processing on the potential energy diffusion matrix based on the computing power requirement drift node, and perform linear prediction on the modulated potential energy diffusion matrix to obtain the potential energy diffusion matrix prediction sequence. Step 8: Determine the computing power demand prediction results corresponding to the future time window based on the potential energy diffusion matrix prediction sequence.

[0019] In this embodiment, step one includes: The system collects task submission records, task start records, task completion records, node resource usage records, container start records, and scheduling event records from the computing power platform during continuous operation. It also extracts the CPU resource request amount, GPU resource request amount, video memory request amount, memory request amount, task queue length, and number of task scheduling failures for each task. The collected data is divided into time slices according to time sequence. Within each time slice, the total number of task submissions, total CPU resource requests, total GPU resource requests, total video memory requests, total memory requests, and total number of scheduling failures are counted. The difference between the total number of task submissions in the current time slice and the previous time slice is calculated to form the task submission change rate. The difference between the total number of resource requests in the current time slice and the previous time slice is calculated to form the resource request change rate. The difference between the total number of scheduling failures in the current time slice and the previous time slice is calculated to form the scheduling status change rate. A change rate sequence is constructed based on the task submission change rate, resource request change rate, and scheduling status change rate. Time nodes in the change rate sequence whose change magnitude exceeds the average change magnitude of adjacent time slices are identified. The identified time nodes are used as the boundaries of the dynamic time window to form a continuous and non-overlapping dynamic time window. The task submission records, node resource usage records, and scheduling event records are aggregated according to the dynamic time window. The total number of task submissions, total number of CPU resource requests, total number of GPU resource requests, total number of video memory requests, total number of memory requests, total task queue length, and total number of scheduling failures within each dynamic time window are counted to construct the computing power request segment for the corresponding dynamic time window.

[0020] In this implementation, the task submission change rate, resource request change rate, and scheduling status change rate are obtained by differential calculation of the cumulative data of consecutive time slices, and the time window boundary is dynamically determined according to the change rate sequence, thereby realizing the adaptive division of computing power request segments. The dynamic time window can automatically adjust its length according to different task types, node resource distribution, and scheduling status to ensure that each computing power request segment can fully reflect the changes in task load and resource consumption fluctuations. When constructing computing power request segments, the resource request amount, task queue length, and scheduling failure number of each task in the same time window are accumulated to form a complete data set for the subsequent generation of computing power demand fingerprint vectors and fingerprint trajectory chains. The method can be executed in real time on the computing power platform through computing nodes or scheduling management modules, supporting continuous data acquisition, change rate calculation, and adaptive window division, and can dynamically update computing power request segments to adapt to the real-time distribution of computing power resources and changes in task structure.

[0021] In this embodiment, step two includes: Read the total number of tasks submitted, the total number of CPU resource requests, the total number of GPU resource requests, the total number of video memory requests, the total number of memory requests, the total length of the task queue, and the total number of scheduling failures from each computing power request segment; Divide the total length of the task queue by the total number of task submissions to calculate the task congestion characteristics. Divide the total number of graphics processor resource requests by the total number of central processing unit resource requests to calculate resource skew characteristics. Divide the total amount of video memory requested by the total amount of graphics processor resources requested to calculate the video memory coupling characteristics. Divide the total number of scheduling failures by the total number of task submissions to calculate the scheduling blocking characteristics; The proportions of total CPU resource requests, total GPU resource requests, total video memory requests, and total RAM requests to the total resource requests are calculated separately, and the resource imbalance characteristics are calculated based on the differences between the proportions of each resource request. Calculate the difference in total task submissions between the current computing power request segment and the adjacent preceding computing power request segment, and calculate the task fluctuation characteristics based on the difference in total task submissions; The characteristics of task congestion, resource skew, memory coupling, scheduling blockage, resource imbalance, and task fluctuation are normalized and combined in a unified order to construct the computing power demand fingerprint vector of the corresponding computing power request segment.

[0022] In this implementation, the computing power demand fingerprint vector is constructed using a multi-dimensional resource state joint representation method. Each resource state feature corresponds to the resource supply and demand relationship in different dimensions. Among them, the task congestion feature reflects the correlation between task queuing status and task arrival status; the resource skew feature reflects the demand distribution status among heterogeneous computing power resources; the memory coupling feature reflects the collaborative occupation status between graphics processor resources and memory resources; the scheduling blocking feature reflects the matching status between resource application and scheduling execution; the resource imbalance feature reflects the occupation balance status among multiple types of resources; and the task fluctuation feature reflects the temporal change status of task load. The normalization process uses a unified numerical interval conversion method to eliminate the differences in the dimensions of different features and writes them into the corresponding positions of the feature vector in a fixed dimensional order, so that continuous computing power request segments form a computing power demand fingerprint vector with a consistent structure, providing a unified data foundation for subsequent construction of computing power demand fingerprint trajectory chains and fingerprint drift analysis.

[0023] In this embodiment, step three includes: According to the time sequence corresponding to the dynamic time window, multiple computing power demand fingerprint vectors are sorted to form a fingerprint vector sequence; Taking the current computing power demand fingerprint vector and the adjacent preceding computing power demand fingerprint vector as the analysis objects, the differences between the corresponding feature values ​​of task congestion feature, resource skew feature, memory coupling feature, scheduling blocking feature, resource imbalance feature and task fluctuation feature are calculated respectively to form the feature change quantity corresponding to each dimension feature. Based on the feature change amount corresponding to each dimension, a state transition record is constructed between the corresponding fingerprint vectors. Each state transition record includes the starting fingerprint vector identifier, the target fingerprint vector identifier, and the feature change amount corresponding to each dimension. The state transition records are connected sequentially in chronological order, and the state transition relationships between the corresponding fingerprint vectors are written into the fingerprint vector sequence to form the fingerprint evolution path. The cumulative feature changes and consistency of change direction corresponding to multiple consecutive state transition records in the fingerprint evolution path are statistically analyzed to construct the evolution state information corresponding to each fingerprint vector. The fingerprint vector sequence, state transition record, fingerprint evolution path and evolution state information are associated and stored, and a corresponding trajectory chain index is established to construct a computing power requirement fingerprint trajectory chain.

[0024] In this implementation, state transition records are used to characterize the resource state evolution process between adjacent computing power demand fingerprint vectors. The feature changes corresponding to each dimension are stored according to the feature arrangement order in the computing power demand fingerprint vector. The consistency of change direction is characterized by statistically analyzing the number of times the change direction of the same dimension feature remains consistent in consecutive state transition records. The change direction includes both increasing and decreasing directions. The evolution state information is composed of the cumulative feature changes and the consistency of change direction, used to describe the change intensity and trend of the corresponding fingerprint vector in the continuous evolution process. The trajectory chain index identifier establishes a correspondence between the fingerprint vector sequence, state transition records, fingerprint evolution path, and evolution state information using a time-series association method. This ensures that the computing power demand fingerprint trajectory chain maintains a continuous and traceable data organization structure, providing a unified data index foundation for subsequent fingerprint drift feature calculation, drift node identification, and historical fingerprint fragment retrieval.

[0025] In this embodiment, step four includes: Read the state transition records, evolution state information, and trajectory chain index identifiers from the computing power demand fingerprint trajectory chain; For each state transition record, the feature change amounts corresponding to task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features are obtained respectively. The absolute values ​​of the feature change amounts of each dimension are accumulated and normalized according to the number of feature dimensions involved in the calculation to form the state change intensity. The number of times the same feature change direction remains consistent in continuous state transition records is counted, and the proportion of the number of consistent changes to the total number of corresponding continuous state transition records is calculated to form the consistency of change direction. Extract the changes in resource imbalance characteristics from the evolutionary state information, and construct the degree of resource imbalance based on the changes in resource imbalance characteristics corresponding to the continuous state transition records; Fingerprint drift features are constructed based on the intensity of state changes, the consistency of change direction, and the degree of resource imbalance. Based on the time sequence corresponding to the trajectory chain index, the drift features of multiple consecutive fingerprints are analyzed to identify the corresponding positions where the intensity of state change and the degree of resource imbalance increase synchronously and the consistency of the change direction is consistently higher than the average consistency of the corresponding consecutive state transition records. The fingerprint vector of the corresponding position is then identified as the computing power demand drift node.

[0026] In this implementation, fingerprint drift features are used to characterize the degree of shift in computing power demand fingerprints during continuous evolution. The intensity of state change reflects the overall change magnitude of resource state features in each dimension, the consistency of change direction reflects the stability of resource state change trends during continuous evolution, and the degree of resource imbalance reflects the degree of change in heterogeneous resource demand structure. In continuous state transition records, when multi-dimensional resource state features change continuously at the same time, the intensity of state change and the degree of resource imbalance increase synchronously, and the consistency of change direction remains at a high level, corresponding to a phased migration of the task load structure and resource demand structure represented by the fingerprint vector. The trajectory chain index is used to establish the temporal correspondence between fingerprint drift features and computing power demand fingerprint trajectory chains, enabling each fingerprint drift feature to be correlated and analyzed according to the evolution order, thereby forming a set of computing power demand drift nodes with continuous evolution characteristics, providing a drift position basis for subsequent historical fingerprint fragment retrieval and computing power evolution potential energy sequence construction.

[0027] In this embodiment, step five includes: Read the fingerprint evolution path, evolution status information, and computing power demand drift nodes in the computing power demand fingerprint trajectory chain; Starting from the computing power demand drift node, extract the continuous state transition records associated with the computing power demand drift node to construct the current evolution segment; Extract historical evolution fragments corresponding to historical fingerprint trajectory chains from the historical fingerprint sample database, and read the state transition records and evolution state information corresponding to each historical evolution fragment; Calculate the differences in the intensity of state changes, the consistency of change direction, and the degree of resource imbalance between the current evolutionary segment and each historical evolutionary segment; The differences in the intensity of state changes, the consistency of change directions, and the degree of resource imbalance are normalized, and then combined according to the proportions of each difference to form an evolutionary matching degree. The historical evolution segments are sorted according to their evolution matching degree. The historical fingerprint segment with the highest evolution matching degree and whose corresponding drift node evolution direction is consistent with the current computing power requirement drift node evolution direction is selected as the target historical fingerprint segment.

[0028] In this implementation, the historical fingerprint sample library classifies and stores historical fingerprint trajectory chains according to trajectory chain index identifiers, and establishes the association between state transition records, evolutionary state information, and drift nodes. The matching process between the current evolutionary segment and the historical evolutionary segment adopts a unified-dimensional evolutionary state description structure, enabling the consistency of state change intensity, change direction, and resource imbalance degree to be compared within the same feature space. The evolution direction is used to characterize the directionality of the change trend during the resource state evolution process, including the growth direction and the decline direction. The evolution matching degree is used to reflect the overall evolutionary similarity between the current evolutionary segment and the historical evolutionary segment. When the evolution direction of the drift node corresponding to the historical evolutionary segment is consistent with the evolution direction of the current computing power demand drift node, it indicates that the two have similar resource evolution patterns. The target historical fingerprint segment retains the evolutionary information after the corresponding drift node for subsequent construction of fingerprint evolution association sequences.

[0029] In this embodiment, step six includes: Read the state transition records, evolution state information, and evolution information after the drift node corresponding to the target historical fingerprint fragment, and read the state transition records and evolution state information corresponding to the computing power requirement fingerprint trajectory chain; The state change intensity sequence is constructed by arranging the state change intensity corresponding to the target historical fingerprint segment and the state change intensity corresponding to the computing power demand fingerprint trajectory chain in chronological order. The resource imbalance degree corresponding to the target historical fingerprint segment and the resource imbalance degree corresponding to the computing power demand fingerprint trajectory chain are arranged in chronological order to construct a resource imbalance evolution sequence. The state change intensity sequence and the resource imbalance evolution sequence are weighted and merged to construct the computing power evolution potential energy sequence. Each potential energy value in the computing power evolution potential energy sequence corresponds to the combination result of the state change intensity and the resource imbalance degree at the same time position. Based on the computing power evolution potential energy sequence, the feature proportions corresponding to task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features are mapped to the potential energy values ​​at each time position, forming the potential energy components corresponding to each feature. Using the potential energy components corresponding to each feature as matrix elements, matrix row and column indices are established according to the evolutionary relationship between task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features. A potential energy diffusion matrix is ​​constructed, and the values ​​of the matrix elements correspond to the potential energy intensity transferred from the matrix row index features to the matrix column index features.

[0030] In this implementation, the computing power evolution potential energy sequence is used to characterize the cumulative change state during the resource demand evolution process. The intensity of state change reflects the magnitude of resource state change, and the degree of resource imbalance reflects the degree of change in heterogeneous resource demand structure. Both participate in the calculation of potential energy value. The potential energy values ​​corresponding to each time position are arranged continuously in chronological order to form the computing power evolution potential energy sequence, so that the historical evolution information and the current evolution information maintain a unified temporal structure. The potential energy components are used to describe the degree of evolutionary contribution of different resource state features at the corresponding time position. Each potential energy component together constitutes the resource state evolution distribution at the same time position. The potential energy diffusion matrix uses a row and column mapping method to describe the evolutionary correlation between different resource state features. The matrix row index corresponds to the potential energy output feature, and the matrix column index corresponds to the potential energy input feature. The matrix element values ​​are dynamically updated with the resource state change, thereby forming a structured potential energy representation that can reflect the multi-dimensional resource state co-evolution process.

[0031] In this embodiment, step seven includes: Input the potential energy diffusion matrix into the improved DLinear model; The improved DLinear model includes a drift-sensing potential energy decomposition module, a drift modulation module, a dual-path prediction module, a drift feedback gate module, and a potential energy fusion module. The drift-aware potential energy decomposition module takes the potential energy diffusion matrix and the computing power demand drift node as input, and reads the matrix elements at each time position in the potential energy diffusion matrix and the intensity of state change, consistency of change direction, and degree of resource imbalance of the drift node. The potential energy evolution interval is divided according to the time position corresponding to the drift node; Perform a moving average process on each interval matrix element to construct a stable potential energy component, and calculate the residual between the potential energy diffusion matrix and the stable component to form a potential energy drift component. Output the stable potential energy component and the potential energy drift component. The drift modulation module takes the potential energy drift component and the drift node as input, and includes: The normalization results are used to determine the intensity and direction of state changes, as well as the degree of resource imbalance, and to construct drift modulation coefficients based on the normalization results. The drift enhancement component is generated and output by weighting the potential energy drift component with the drift modulation coefficient; The dual-path prediction module takes the stable potential energy component and the drift enhancement component as inputs to generate stable potential energy prediction results and drift potential energy prediction results, respectively. The drift feedback gate module takes the stable potential energy prediction result and the drift potential energy prediction result as input, calculates the change amplitude of the drift potential energy prediction result, and generates the drift feedback weight by combining the drift modulation coefficient, adjusts the stable potential energy prediction result, and outputs the feedback stable potential energy prediction result. The potential energy fusion module takes the feedback stable potential energy prediction results and the drift potential energy prediction results as inputs, and fuses them according to time position to generate a potential energy diffusion matrix prediction sequence.

[0032] In this implementation, both the improved DLinear model and the DLinear model adopt a processing method that combines sequence decomposition and linear prediction. Both perform linear prediction after decomposing the input sequence into components and then fuse the prediction results for output, thereby maintaining the structural characteristics of the DLinear model, such as small number of parameters, high training efficiency and suitability for long sequence prediction. The improved DLinear model uses the potential energy diffusion matrix as input and introduces a drift-aware potential energy decomposition module. It divides the potential energy evolution interval using the drift nodes based on computing power requirements and constructs stable and drift components within each interval. Simultaneously, a drift modulation module is set up to construct drift modulation coefficients based on the intensity of state changes, consistency of change direction, and degree of resource imbalance, thereby modulating the potential energy drift components. Furthermore, a drift feedback gate module is set up to use the drift potential energy prediction results to correct the stable potential energy prediction results. After adopting the above improvements, the model decomposition boundary can be dynamically adjusted as computing power demand evolves, making the stable and drift components of potential energy more consistent with the changing patterns of computing power resources; the drift modulation mechanism can enhance the ability to represent resource changes during the drift phase; and the drift feedback gate mechanism can establish a correlation between stable and drift potential energy, thereby improving the ability of the potential energy diffusion matrix prediction sequence to characterize sudden changes in resource demand and resource structure migration processes.

[0033] In this embodiment, step eight includes: Read the potential energy diffusion matrix corresponding to each time position in the predicted sequence; Based on the matrix row and column indices corresponding to task congestion characteristics, resource skew characteristics, memory coupling characteristics, scheduling blocking characteristics, resource imbalance characteristics, and task fluctuation characteristics, the potential energy transfer intensity corresponding to each characteristic is extracted. Accumulate the potential energy transfer intensity corresponding to the same feature to construct a sequence of feature potential energy values; The contribution values ​​of graphics processor (GPU) resource demand and central processing unit (CPU) resource demand are calculated based on the characteristic potential energy value sequence corresponding to the resource skew characteristics. Combined with the total number of GPU resource requests and CPU resource requests corresponding to the current time window, the predicted values ​​of GPU resource demand and CPU resource demand are determined. The contribution value of video memory resource demand is calculated based on the characteristic potential energy value sequence corresponding to the video memory coupling characteristics, and the predicted value of video memory resource demand is determined by combining the total amount of video memory application corresponding to the current time window. The task load contribution value is calculated based on the characteristic potential value sequence corresponding to the task congestion characteristics, scheduling blocking characteristics, and task fluctuation characteristics. Combined with the total task queue length and the number of container instances corresponding to the current time window, the predicted values ​​of task queue length and container instance number are determined. The predicted values ​​for graphics processor resource requirements, central processing unit resource requirements, video memory resource requirements, task queue length, and container instance quantity are correlated and output to form the computing power demand prediction results for the future time window.

[0034] refer to Figure 2 A computing power demand prediction system based on big data processing includes: The computing power request fragment construction module is used to collect task operation data, resource monitoring data and scheduling data from the computing power platform, dynamically divide time windows according to changes in resource status, and construct computing power request fragments. The computing power demand fingerprint construction module is used to extract resource status features based on computing power request fragments and construct computing power demand fingerprint vectors based on resource status features. The fingerprint trajectory chain construction module is used to connect multiple computing power demand fingerprint vectors in chronological order to construct a computing power demand fingerprint trajectory chain. The drift node identification module is used to calculate fingerprint drift features based on the fingerprint trajectory chain of computing power requirements, and to determine the drift nodes of computing power requirements based on the fingerprint drift features. The historical fingerprint retrieval module is used to retrieve target historical fingerprint fragments from the historical fingerprint sample library based on the computing power demand fingerprint trajectory chain and computing power demand drift nodes. The potential energy modeling module is used to construct a computing power evolution potential energy sequence based on the target's historical fingerprint fragments and computing power demand fingerprint trajectory chain, and to construct a potential energy diffusion matrix based on the computing power evolution potential energy sequence. The potential energy prediction module is used to input the potential energy diffusion matrix into the improved DLinear model, perform drift modulation processing on the potential energy diffusion matrix based on the computing power requirement drift node, and perform linear prediction on the modulated potential energy diffusion matrix to obtain the potential energy diffusion matrix prediction sequence. The computing power demand output module is used to determine the computing power demand prediction results corresponding to future time windows based on the potential energy diffusion matrix prediction sequence.

[0035] Example 1: To verify the feasibility of this invention in practice, it was applied to a large-scale cloud computing power scheduling platform. The platform is deployed in a multi-datacenter environment, managing 1200 CPU server nodes and 320 GPU server nodes, serving tasks including large model training, inference, data analysis, and containerized microservices. With the continuous growth of business scale, the platform experiences significant fluctuations in resource demand at different times of the day, especially during periods of concentrated large model training task submissions, sudden increases in inference business, and the initiation of batch data processing tasks. During these periods, GPU resources, video memory resources, and the number of container instances all change rapidly. Traditional prediction methods typically use fixed time windows to model resource utilization sequences. When the business load structure changes, it is difficult to identify the drift behavior during the evolution of resource demand, leading to a large deviation between the predicted resource demand and the actual demand, further causing problems such as insufficient resource reservation or idle resources.

[0036] In practical applications, the system first continuously collects task submission records, task start records, task completion records, node resource monitoring records, and scheduling event records. Based on the task submission change rate, resource request change rate, and scheduling status change rate, it dynamically generates computing power request fragments. Then, it extracts task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features to construct a computing power demand fingerprint vector. Further, it establishes a computing power demand fingerprint trajectory chain in chronological order and calculates the intensity of state changes, consistency of change direction, and degree of resource imbalance to identify computing power demand drift nodes. Using these drift nodes, it retrieves historical fingerprint fragments from the historical fingerprint sample database that match the current evolution process, and models the correlation between historical and current evolution information to construct a computing power evolution potential energy sequence and a potential energy diffusion matrix. Finally, it inputs the potential energy diffusion matrix into the improved DLinear model, decomposes the potential energy diffusion matrix using the drift-aware potential energy decomposition module, and completes the prediction calculation using the drift modulation module and drift feedback gate module, outputting the computing power demand prediction results for the future time window.

[0037] To verify the predictive performance of this invention, it was compared with traditional LSTM prediction models, Transformer prediction models, and the original DLinear model. The test period was 30 consecutive days, and the prediction window was the resource demand for the next 60 minutes. Using the actual resource demand as a benchmark, the errors of different models in predicting GPU resource demand, CPU resource demand, VRAM resource demand, and the number of container instances were statistically analyzed. The experimental results are shown in Table 1.

[0038] Table 1 Comparison of Computing Power Demand Prediction Results

[0039] As shown in Table 1, compared to the original DLinear model, this invention reduces the error in predicting graphics processor resource requirements by 54.9%, central processing unit resource requirements by 55.2%, video memory resource requirements by 57.3%, and container instance quantity by 59.8%; the overall prediction accuracy improves from 91.7% to 97.3%. This demonstrates that this invention, by utilizing computing power demand fingerprinting, drift node identification mechanisms, historical evolution matching mechanisms, and the improved DLinear prediction model, effectively enhances the adaptability to scenarios involving sudden changes in resource demand, resource structure migration, and complex load fluctuations. It can more accurately predict computing power demands in future time windows, providing reliable data support for computing power platform resource scheduling, resource reservation, and resource scaling.

[0040] Further analysis revealed that during periods of stable business load, various prediction models achieved good prediction results. However, when the platform experienced scenarios such as the concentrated launch of large model training tasks, a surge in inference requests, and resource structure migration, traditional models, unable to identify resource demand shifts, exhibited significantly increased prediction errors. This invention, by constructing a computing power demand fingerprint trajectory chain and computing power demand drift nodes, can identify resource demand structure changes in advance. Simultaneously, it utilizes a historical fingerprint fragment retrieval mechanism to obtain similar evolution patterns and constructs a computing power evolution potential energy sequence and potential energy diffusion matrix to model the correlation and changes between heterogeneous resources. Therefore, it can maintain high prediction accuracy even in scenarios with sudden changes in resource demand.

[0041] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for predicting computing power demand based on big data processing, characterized in that, Includes the following steps: Step 1: Collect task execution data, resource monitoring data, and scheduling data from the computing power platform, dynamically divide time windows based on changes in resource status, and construct computing power request segments; Step 2: Extract resource status features based on the computing power request fragment, and construct a computing power demand fingerprint vector based on the resource status features; Step 3: Connect the multiple computing power demand fingerprint vectors in chronological order to construct a computing power demand fingerprint trajectory chain; Step 4: Calculate the fingerprint drift features based on the fingerprint trajectory chain of the computing power requirement, and determine the computing power requirement drift node according to the fingerprint drift features; Step 5: Based on the computing power demand fingerprint trajectory chain and computing power demand drift nodes, retrieve the target historical fingerprint fragment in the historical fingerprint sample database; Step 6: Construct a computing power evolution potential energy sequence based on the target historical fingerprint fragment and computing power demand fingerprint trajectory chain, and construct a potential energy diffusion matrix based on the computing power evolution potential energy sequence; Step 7: Input the potential energy diffusion matrix into the improved DLinear model, perform drift modulation processing on the potential energy diffusion matrix based on the computing power demand drift node, and perform linear prediction on the modulated potential energy diffusion matrix to obtain the potential energy diffusion matrix prediction sequence. Step 8: Determine the computing power demand prediction result corresponding to the future time window based on the predicted sequence of the potential energy diffusion matrix.

2. The computing power demand prediction method based on big data processing according to claim 1, characterized in that, Step one includes: The system collects task submission records, task start records, task completion records, node resource usage records, container start records, and scheduling event records from the computing power platform during continuous operation. It also extracts the CPU resource request amount, GPU resource request amount, video memory request amount, memory request amount, task queue length, and number of task scheduling failures for each task. The collected data is divided into time slices according to time sequence. Within each time slice, the total number of task submissions, total CPU resource requests, total GPU resource requests, total video memory requests, total memory requests, and total number of scheduling failures are counted. The difference between the total number of task submissions in the current time slice and the previous time slice is calculated to form the task submission change rate. The difference between the total number of resource requests in the current time slice and the previous time slice is calculated to form the resource request change rate. The difference between the total number of scheduling failures in the current time slice and the previous time slice is calculated to form the scheduling status change rate. Based on the task submission change rate, resource request change rate, and scheduling status change rate, a change rate sequence is constructed. Time nodes in the change rate sequence whose change amplitude exceeds the average change amplitude of adjacent time slices are identified. The identified time nodes are used as the boundaries of the dynamic time window to form a continuous and non-overlapping dynamic time window. The task submission records, node resource usage records, and scheduling event records are aggregated according to the dynamic time window. The total number of task submissions, total number of CPU resource requests, total number of GPU resource requests, total number of video memory requests, total number of memory requests, total task queue length, and total number of scheduling failures within each dynamic time window are counted to construct the computing power request segment for the corresponding dynamic time window.

3. The computing power demand prediction method based on big data processing according to claim 1, characterized in that, Step two includes: Read the total number of tasks submitted, the total number of CPU resource requests, the total number of GPU resource requests, the total number of video memory requests, the total number of memory requests, the total length of the task queue, and the total number of scheduling failures from each of the computing power request segments; Divide the total length of the task queue by the total number of task submissions to calculate the task congestion characteristics. Divide the total number of graphics processor resource requests by the total number of central processing unit resource requests to calculate resource skew characteristics. Divide the total amount of video memory requested by the total amount of graphics processor resources requested to calculate the video memory coupling characteristics. Divide the total number of scheduling failures by the total number of task submissions to calculate the scheduling blocking characteristics; The proportions of total CPU resource requests, total GPU resource requests, total video memory requests, and total RAM requests to the total resource requests are calculated separately, and the resource imbalance characteristics are calculated based on the differences between the proportions of each resource request. Calculate the difference in total task submissions between the current computing power request segment and the adjacent preceding computing power request segment, and calculate the task fluctuation characteristics based on the difference in total task submissions; The characteristics of task congestion, resource skew, memory coupling, scheduling blockage, resource imbalance, and task fluctuation are normalized and combined in a unified order to construct the computing power demand fingerprint vector of the corresponding computing power request segment.

4. The computing power demand prediction method based on big data processing according to claim 1, characterized in that, Step three includes: According to the time sequence corresponding to the dynamic time window, multiple computing power demand fingerprint vectors are sorted to form a fingerprint vector sequence; Taking the current computing power demand fingerprint vector and the adjacent preceding computing power demand fingerprint vector as the analysis objects, the differences between the corresponding feature values ​​of task congestion feature, resource skew feature, memory coupling feature, scheduling blocking feature, resource imbalance feature and task fluctuation feature are calculated respectively to form the feature change quantity corresponding to each dimension feature. Based on the feature change amount corresponding to each dimension, a state transition record is constructed between the corresponding fingerprint vectors. Each state transition record includes the starting fingerprint vector identifier, the target fingerprint vector identifier, and the feature change amount corresponding to each dimension. The state transition records are connected sequentially in chronological order, and the state transition relationships between the corresponding fingerprint vectors are written into the fingerprint vector sequence to form the fingerprint evolution path. The cumulative feature changes and consistency of change direction corresponding to multiple consecutive state transition records in the fingerprint evolution path are statistically analyzed to construct the evolution state information corresponding to each fingerprint vector. The fingerprint vector sequence, state transition record, fingerprint evolution path and evolution state information are associated and stored, and a corresponding trajectory chain index identifier is established to construct a computing power requirement fingerprint trajectory chain.

5. The method for predicting computing power demand based on big data processing according to claim 1, characterized in that, Step four includes: Read the state transition records, evolution state information, and trajectory chain index identifiers from the computing power demand fingerprint trajectory chain; For each state transition record, the feature change amounts corresponding to task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features are obtained respectively. The absolute values ​​of the feature change amounts of each dimension are accumulated and normalized according to the number of feature dimensions involved in the calculation to form the state change intensity. The number of times the same feature change direction remains consistent in continuous state transition records is counted, and the proportion of the number of consistent changes to the total number of corresponding continuous state transition records is calculated to form the consistency of change direction. Extract the changes in resource imbalance characteristics from the evolutionary state information, and construct the degree of resource imbalance based on the changes in resource imbalance characteristics corresponding to the continuous state transition records; Fingerprint drift features are constructed based on the intensity of state changes, the consistency of change direction, and the degree of resource imbalance. Based on the time sequence corresponding to the trajectory chain index, the drift features of multiple consecutive fingerprints are analyzed to identify the corresponding positions where the intensity of state change and the degree of resource imbalance increase synchronously and the consistency of the change direction is consistently higher than the average consistency of the corresponding consecutive state transition records. The fingerprint vector of the corresponding position is then identified as the computing power demand drift node.

6. The method for predicting computing power demand based on big data processing according to claim 1, characterized in that, Step five includes: Read the fingerprint evolution path, evolution status information, and computing power demand drift nodes in the computing power demand fingerprint trajectory chain; Starting from the computing power demand drift node, extract the continuous state transition records associated with the computing power demand drift node to construct the current evolution segment; Extract historical evolution fragments corresponding to historical fingerprint trajectory chains from the historical fingerprint sample database, and read the state transition records and evolution state information corresponding to each historical evolution fragment; Calculate the differences in the intensity of state changes, the consistency of change direction, and the degree of resource imbalance between the current evolutionary segment and each historical evolutionary segment; The differences in the intensity of state changes, the consistency of change directions, and the degree of resource imbalance are normalized, and then combined according to the proportions of each difference to form an evolutionary matching degree. The historical evolution segments are sorted according to their evolution matching degree. The historical fingerprint segment with the highest evolution matching degree and whose corresponding drift node evolution direction is consistent with the current computing power requirement drift node evolution direction is selected as the target historical fingerprint segment.

7. The method for predicting computing power demand based on big data processing according to claim 1, characterized in that, Step six includes: Read the state transition records, evolution state information, and evolution information after the drift node corresponding to the target historical fingerprint fragment, and read the state transition records and evolution state information corresponding to the computing power requirement fingerprint trajectory chain; The state change intensity sequence is constructed by arranging the state change intensity corresponding to the target historical fingerprint segment and the state change intensity corresponding to the computing power demand fingerprint trajectory chain in chronological order. The resource imbalance degree corresponding to the target historical fingerprint segment and the resource imbalance degree corresponding to the computing power demand fingerprint trajectory chain are arranged in chronological order to construct a resource imbalance evolution sequence. The state change intensity sequence and the resource imbalance evolution sequence are weighted and merged to construct the computing power evolution potential energy sequence. Each potential energy value in the computing power evolution potential energy sequence corresponds to the combination result of the state change intensity and the degree of resource imbalance at the same time position. Based on the computing power evolution potential energy sequence, the feature proportions corresponding to task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features are mapped to the potential energy values ​​at each time position, forming the potential energy components corresponding to each feature. Using the potential energy components corresponding to each feature as matrix elements, matrix row and column indices are established according to the evolutionary relationship between task congestion features, resource skew features, memory coupling features, scheduling blocking features, resource imbalance features, and task fluctuation features. A potential energy diffusion matrix is ​​constructed, and the values ​​of the matrix elements correspond to the potential energy intensity transferred from the matrix row index features to the matrix column index features.

8. The method for predicting computing power demand based on big data processing according to claim 1, characterized in that, Step seven includes: Input the potential energy diffusion matrix into the improved DLinear model; The improved DLinear model includes a drift-sensing potential energy decomposition module, a drift modulation module, a two-way path prediction module, a drift feedback gate module, and a potential energy fusion module. The drift sensing potential energy decomposition module takes the potential energy diffusion matrix and the computing power demand drift node as input, and reads the matrix elements at each time position in the potential energy diffusion matrix and the state change intensity, change direction consistency and resource imbalance degree of the drift node corresponding to the state change intensity, change direction consistency and resource imbalance degree of the drift node. The potential energy evolution interval is divided according to the time position corresponding to the drift node; Perform a moving average process on each interval matrix element to construct a stable potential energy component, and calculate the residual between the potential energy diffusion matrix and the stable component to form a potential energy drift component. Output the stable potential energy component and the potential energy drift component. The drift modulation module takes potential energy drift components and drift nodes as inputs, and includes: The normalization results are used to determine the intensity and direction of state changes, as well as the degree of resource imbalance, and to construct drift modulation coefficients based on the normalization results. The drift enhancement component is generated and output by weighting the potential energy drift component with the drift modulation coefficient; The dual-path prediction module takes the stable potential energy component and the drift enhancement component as inputs to generate stable potential energy prediction results and drift potential energy prediction results, respectively. The drift feedback gate module takes the stable potential energy prediction result and the drift potential energy prediction result as input, calculates the change amplitude of the drift potential energy prediction result, and generates a drift feedback weight in combination with the drift modulation coefficient to adjust the stable potential energy prediction result and output the feedback stable potential energy prediction result. The potential energy fusion module takes the feedback stable potential energy prediction results and the drift potential energy prediction results as inputs, and generates a potential energy diffusion matrix prediction sequence by fusing them according to time position.

9. The method for predicting computing power demand based on big data processing according to claim 1, characterized in that, Step eight includes: Read the potential energy diffusion matrix corresponding to each time position in the predicted sequence; Based on the matrix row and column indices corresponding to task congestion characteristics, resource skew characteristics, memory coupling characteristics, scheduling blocking characteristics, resource imbalance characteristics, and task fluctuation characteristics, the potential energy transfer intensity corresponding to each characteristic is extracted. Accumulate the potential energy transfer intensity corresponding to the same feature to construct a sequence of feature potential energy values; The contribution values ​​of graphics processor (GPU) resource demand and central processing unit (CPU) resource demand are calculated based on the characteristic potential energy value sequence corresponding to the resource skew characteristics. Combined with the total number of GPU resource requests and CPU resource requests corresponding to the current time window, the predicted values ​​of GPU resource demand and CPU resource demand are determined. The contribution value of video memory resource demand is calculated based on the sequence of characteristic potential energy values ​​corresponding to the video memory coupling characteristics, and the predicted value of video memory resource demand is determined by combining the total amount of video memory requests corresponding to the current time window. The task load contribution value is calculated based on the characteristic potential value sequence corresponding to the task congestion characteristics, scheduling blocking characteristics, and task fluctuation characteristics. Combined with the total task queue length and the number of container instances corresponding to the current time window, the predicted values ​​of task queue length and container instance number are determined. The predicted values ​​for graphics processor resource requirements, central processing unit resource requirements, video memory resource requirements, task queue length, and container instance quantity are correlated and output to form the computing power demand prediction results for the future time window.

10. A computing power demand prediction system based on big data processing, executing the computing power demand prediction method based on big data processing as described in any one of claims 1 to 9, characterized in that, include: The computing power request fragment construction module is used to collect task operation data, resource monitoring data and scheduling data from the computing power platform, dynamically divide time windows according to changes in resource status, and construct computing power request fragments. The computing power demand fingerprint construction module is used to extract resource status features based on the computing power request fragment, and construct a computing power demand fingerprint vector based on the resource status features. The fingerprint trajectory chain construction module is used to connect multiple computing power demand fingerprint vectors in chronological order to construct a computing power demand fingerprint trajectory chain. A drift node identification module is used to calculate fingerprint drift features based on the computing power requirement fingerprint trajectory chain, and determine computing power requirement drift nodes according to the fingerprint drift features; The historical fingerprint retrieval module is used to retrieve target historical fingerprint fragments from the historical fingerprint sample library based on the computing power demand fingerprint trajectory chain and computing power demand drift nodes. The potential energy modeling module is used to construct a computing power evolution potential energy sequence based on the target historical fingerprint fragment and computing power demand fingerprint trajectory chain, and to construct a potential energy diffusion matrix based on the computing power evolution potential energy sequence. The potential energy prediction module is used to input the potential energy diffusion matrix into the improved DLinear model, perform drift modulation processing on the potential energy diffusion matrix based on the computing power demand drift node, and perform linear prediction on the modulated potential energy diffusion matrix to obtain the potential energy diffusion matrix prediction sequence. The computing power demand output module is used to determine the computing power demand prediction result corresponding to the future time window based on the potential energy diffusion matrix prediction sequence.

Images