A cold data computing resource occupation prediction method based on machine learning

Abstract

The application discloses a cold data computing resource occupation prediction method based on machine learning, which comprises the following steps: generating task resource state data; generating resource prediction infrastructure; establishing a resource activation risk set to obtain executable resource event intensity; establishing a task fragment partial order structure to determine single event resource occupation intensity; generating a resource occupation prediction trajectory by using a Tweedie regression model; constraining the resource occupation prediction trajectory by using a Skorokhod reflection mapping according to a node executable resource trajectory to obtain an unfulfilled resource occupation trajectory; performing directional conversion on the unfulfilled resource occupation quantity, and iteratively executing the Tweedie regression and the Skorokhod reflection mapping based on the directional conversion result to output a computing resource occupation prediction result. The application can improve the accuracy of cross-border cold data task computing resource occupation prediction, and provide a reliable basis for the collaborative scheduling between border park computing nodes and domestic computing nodes.

Description

Technical Field

[0001] This invention relates to the field of cross-border data processing technology, and in particular to a method for predicting the utilization of computing resources for cold data based on machine learning. Background Technology

[0002] With the development of cross-border data asset development and computing infrastructure construction, border digital parks are gradually undertaking tasks such as accessing overseas data, optimizing data processing, coordinating domestic computing power, and sending back processing results. Cross-border cold data is typically characterized by large data volume, low access frequency, long processing cycles, and significant fluctuations in resource consumption. During processing, it needs to pass through computing nodes in border parks, domestic computing nodes, and data transmission links. Existing computing resource management methods typically allocate tasks based on the current node load, historical average resource consumption, or preset resource thresholds, and adjust computing resources according to the task queue length. Some methods utilize regression models to predict the usage trends of central processing units, graphics processing units, memory, and storage resources, providing a reference for computing node configuration.

[0003] However, cross-border cold data tasks involve stages including overseas data access, preprocessing in border parks, collaborative processing with domestic computing power, and result feedback. These stages have clear dependencies on each other, and the same task may repeatedly trigger resource activation events due to waiting, pauses, resource congestion, and computing node migration. Existing prediction methods typically use natural time to statistically analyze resource usage data, making it difficult to eliminate unexecutable time and maintain continuity of resource activation events and processing progress before and after task migration, leading to distorted predictions of resource event frequency. Furthermore, the branching and merging structures in the task processing chain can easily cause duplicate counting of preceding processing volumes, making it difficult to accurately determine the actual resource usage level corresponding to a single resource activation event.

[0004] Therefore, how to provide a machine learning-based method for predicting cold data computing resource usage is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a machine learning-based method for predicting the occupancy of cold data computing resources. This invention can combine the execution status of cross-border cold data tasks, resource activation frequency, and the executable resource capabilities of nodes to jointly predict zero occupancy and positive continuous occupancy states. The prediction results are corrected by feedback of unfulfilled resource occupancy, thereby improving the accuracy, stability, and node capacity adaptability of computing resource occupancy prediction, and providing a reliable basis for the collaborative scheduling between computing nodes in border parks and domestic computing nodes.

[0006] A cold data computing resource occupancy prediction method based on machine learning according to an embodiment of the present invention includes the following steps: Acquire cross-border cold data computing power operation data, divide it into task segments according to the cross-border task processing chain, and generate task resource status data; Based on task resource status data, a resource prediction infrastructure is generated according to resource activation relationships and prerequisite dependencies. Based on the resource prediction infrastructure, a set of resource activation risks is established. By combining the execution status and processing progress of task segments, the intensity of duplicate resource events is estimated, and the intensity of executable resource events is obtained. Based on the resource prediction infrastructure, a partial order structure of task segments is established. The contribution of the preceding sequence is deducted according to the preceding dependency to obtain the marginal processing amount of the task segment and pass it along the cross-border task processing chain to determine the resource occupancy intensity of a single event. The executable resource event intensity and single event resource occupancy intensity are converted into Tweedie composite Poisson parameters and composite gamma parameters, respectively, and the resource occupancy prediction trajectory is generated by using the Tweedie regression model. Extract the executable resource trajectory of nodes from the task resource status data, and use Skorok reflection mapping to constrain the resource occupation prediction trajectory according to the executable resource trajectory of nodes to obtain the unfulfilled resource occupation trajectory. Based on the execution status of the task segment, the unrealized resource usage is transformed into a specific category. Based on the transformation results, Tweedie regression and Skorokhod reflection mapping are iteratively executed to output the predicted computing power resource usage results.

[0007] Optionally, obtaining the task resource status data specifically includes: Collect cross-border cold data operation records and establish cross-border cold data computing power operation data according to task spatiotemporal association rules; Based on the cross-border cold data computing power operation data, the stage connection relationship of the cross-border cold data task is determined. Based on the stage connection relationship, the cross-border task processing chain is established. The cross-border task processing chain is divided into task segments according to the task segment division conditions, and a task segment identifier is configured for each task segment. Time alignment is performed based on the task status and node status corresponding to the task segment at the time of acquisition to generate task resource status data.

[0008] Optionally, obtaining the resource prediction infrastructure specifically includes: Based on task resource status data, the resource occupancy status of task segments is compared sequentially according to task segment identifier and collection time. The resource activation event is confirmed by combining the task segment execution status and computing resource type. The resource activation event is associated with task segment identifier, computing resource type and collection time to generate resource activation relationship. Identify the corresponding preceding task segments according to the data transfer direction and execution order between task segments, establish the preceding dependency relationship based on the completion status of the preceding task segment and the start condition of the current task segment, and update the preceding dependency relationship according to the collection time. Task fragments are identified as structural nodes, resource activation relationships are written into the corresponding structural nodes, prerequisite dependencies are converted into directed dependency connections between structural nodes, and task resource status data is associated with the corresponding structural nodes to form the basic structure for resource prediction.

[0009] Optionally, obtaining the strength of the executable resource event specifically includes: Resource activation relationships, prerequisite dependencies, and task resource status data are extracted from the resource prediction infrastructure to determine the executable exposure intervals corresponding to task segments. The durations of task segments within the executable exposure intervals are sequentially connected to form the executable time axis of the task segments. A resource activation risk set is then established based on the executable time axis of the task segments. The resource activation events in the resource activation relationship are mapped to the executable time axis of the task segment according to the corresponding collection time. The resource activation events that occur repeatedly in the same task segment are accumulated according to the task segment identifier. When the computing power node of the task segment is migrated, the executable exposure interval corresponding to the original computing power node is terminated. When the target computing power node meets the resource execution conditions, a new executable exposure interval is established. The accumulated results of the resource activation events formed by the task segment and the processing progress of the task segment are transferred to the target computing power node to generate a continuous event sequence across computing power nodes. The executable timeline of task segments is used as the counting time benchmark for the Andersen-Gill counting process, replacing the natural acquisition time. Event counting intervals are divided according to the executable exposure interval. The event counting intervals corresponding to the original computing power node and the target computing power node for the same task segment are continuously connected. The cumulative number of resource activation events and the processing progress of task segments are transferred to the target computing power node. The task segments participating in intensity estimation are determined according to the resource activation risk set. Based on the occurrence density of resource activation events within the executable exposure interval and the difference in processing progress of task segments, the baseline resource event intensity and state influence relationship are updated. The intensity of repeated resource events in a unit executable time is determined. The intensity of repeated resource events is aggregated according to computing power resource type and computing power node, and time mapping is performed in combination with the length of the executable exposure interval within the prediction period to obtain the executable resource event intensity.

[0010] Optionally, obtaining the single-event resource occupancy intensity specifically includes: Extract the preceding dependencies and task resource status data from the resource prediction infrastructure, use task fragments as partial order elements, determine the partial order direction according to the preceding dependencies, arrange the task fragments according to the execution order in the cross-border task processing chain, establish the task fragment partial order structure, and extract the cumulative processing amount of the task fragments at the time of collection from the task resource status data. Along the partial order structure of task segments, the source of the cumulative processing volume is traced from the starting task segment to the ending task segment. The source task segment identifier and the transmission path identifier are configured for the processing volume from the previous task segment to the current task segment. At the branch position, the previous contribution volume leading to different downstream task segments is retained. At the merging position, the same source previous contribution volume arriving through different transmission paths is merged according to the source task segment identifier to generate a previous contribution record. Task segments are processed sequentially according to the partial order hierarchy in the partial order structure of the task segments. The previous contribution amount corresponding to the previous contribution record is deducted from the cumulative processing amount of the current task segment using partial order Möbius inversion. The same source previous contribution amount that is repeatedly written through different transmission paths is deducted once. The repeated deduction amount caused by the overlap of partial order hierarchy is replenished according to the partial order hierarchy. The processing amount after deduction and replenishment is determined as the marginal processing amount of the task segment. According to the data transmission direction in the cross-border task processing chain, the marginal processing amount of the task segment is written into the transmission connection corresponding to the adjacent downstream task segment. The marginal processing amount of the task segment in the transmission connection is adjusted according to the data conversion relationship. At the branch position, the marginal processing amount of the task segment is allocated according to the actual receiving ratio of the downstream task segment. At the merging position, the duplicate incoming amount is removed according to the source task segment identifier and the remaining incoming amount is collected to generate the marginal processing amount transmission result. Based on the marginal processing volume transfer results, determine the actual marginal processing volume undertaken by the task segment in a single resource activation event. Extract the positive resource occupancy corresponding to the resource activation event from the task resource status data. Establish a resource occupancy conversion relationship between marginal processing volume and positive resource occupancy according to the task segment identifier, computing power resource type, and computing power node. Use the resource occupancy conversion relationship to convert the actual marginal processing volume undertaken by the task segment in a single resource activation event into single-event resource occupancy intensity.

[0011] Optionally, obtaining the resource occupancy prediction trajectory specifically includes: Based on the computing power node, computing power resource type, and collection time, the intensity of executable resource events, the intensity of single event resource occupancy, and the task resource status data are correlated. The actual resource occupancy corresponding to the collection time is used as the supervision label to form Tweedie regression training samples. The intensity of executable resource events is accumulated over time according to the collection time. The accumulated time result is converted into Tweedie composite Poisson parameters. Training samples with supervision labels greater than zero are selected from the Tweedie regression training samples to form positive resource occupancy samples. Tweedie composite gamma parameters are generated based on the intensity of single event resource occupancy and the dispersion of positive resource occupancy samples. The Tweedie composite Poisson parameters and Tweedie composite gamma parameters are written into the corresponding Tweedie regression training samples. Based on the Tweedie composite Poisson parameter and the Tweedie composite gamma parameter, a dual-intensity resource prediction feature is established. The dual-intensity resource prediction feature and the task resource status data are input into the Tweedie regression model to obtain the resource occupancy prediction value. At the same time, a supervised training method is used to fit the mapping relationship between the resource prediction feature and the supervision label. The model parameters are updated based on the prediction deviation between the predicted resource usage and the supervision label. A dual-intensity consistency constraint is established based on the deviation between the number of resource activation events corresponding to the predicted resource usage and the intensity of executable resource events, and the deviation between the positive continuous resource usage and the intensity of single event resource usage. The model parameter update is repeated until the prediction deviation and the dual-intensity consistency deviation meet the training convergence condition, and the trained Tweedie regression model is obtained. Based on the computing power node, computing power resource type, and collection time within the prediction period, generate dual-intensity resource prediction features corresponding to the prediction stage. Input the dual-intensity resource prediction features and task resource status data into the trained Tweedie regression model to obtain the corresponding zero occupancy probability, positive continuous resource occupancy level, and resource occupancy prediction value. Connect the resource occupancy prediction values ​​according to the time sequence of the collection time to generate the resource occupancy prediction trajectory.

[0012] Optionally, obtaining the unfulfilled resource usage trajectory specifically includes: Extract node resource status from task resource status data, perform node executable resource quantity generation processing, and obtain node executable resource trajectory; Based on the executable resource trajectory and predicted resource occupancy trajectory of the node, resource supply and demand alignment processing is performed to obtain the resource supply and demand alignment sequence; Skorokhod reflection mapping is performed on the resource supply and demand alignment sequence, and non-negative boundary constraints and unfulfilled resource occupancy recursion processing are performed to obtain the unfulfilled resource occupancy trajectory.

[0013] Optionally, obtaining the computing resource occupancy prediction result specifically includes: Based on the execution status of the task segment, the unfulfilled resource usage in the unfulfilled resource usage trajectory is allocated to the corresponding task segment and status-based processing is performed to obtain the status-based resource usage. Perform intensity conversion processing on the resource occupancy of each state and obtain the conversion result; Input the direction conversion results and task resource status data into the trained Tweedie regression model to generate an updated resource occupancy prediction trajectory. Input the updated resource occupancy prediction trajectory and the node executable resource trajectory into Skorokhod reflection mapping to generate an updated unfulfilled resource occupancy trajectory. Repeat the state direction conversion process and intensity conversion process based on the updated unfulfilled resource occupancy trajectory. Compare the resource usage prediction trajectories formed by two adjacent iterations, normalize the differences in resource usage prediction values ​​corresponding to different computing power resource types, and stop iterating when the differences in the normalized resource usage prediction trajectories are not greater than the corresponding trajectory convergence threshold, and output the computing power resource usage prediction results.

[0014] The beneficial effects of this invention are: This invention addresses the execution characteristics of cross-border cold data tasks, including waiting, pauses, resource blockages, and computing node migrations. Instead of directly counting resource activation events based on natural acquisition time, it improves the Andersen-Gill counting process based on the executable timeline of task segments. This removes non-executable time from the resource event risk exposure time and continuously inherits the cumulative results of resource activation events and the task segment processing progress when computing node migration occurs within a task segment. The Andersen-Gill counting process can characterize recurring resource activation events within the same task segment, and the improvement based on the executable timeline of task segments avoids the miscounting of waiting time and migration waiting time as resource event exposure time. This improves the accuracy of estimating the intensity of executable resource events, making the resource activation frequency more consistent with the actual execution processes of computing nodes in border parks and domestic computing nodes.

[0015] This invention addresses the issues of branching, merging, and data transformation relationships in cross-border task processing chains. It employs partial-order Möbius inversion to separate the preceding contribution from the cumulative processing volume of task segments. By tracing the transmission source of the processing volume through the source task segment identifier and transmission path identifier, it deducts the preceding contribution from the same source that is repeatedly written through different transmission paths, and replenishes the duplicate deduction caused by overlapping partial-order levels. The partial-order Möbius inversion can extract the marginal processing volume of newly added task segments from the cumulative processing volume. Combined with improvements in branch allocation, merging deduplication, and data transformation relationships, it avoids excessive resource occupancy intensity in single events due to repeated accumulation of preceding processing volume, making the positive continuous resource occupancy level corresponding to a single resource activation event more accurate.

[0016] This invention addresses the coexistence of zero-occupancy and positive continuous occupancy states in cold data computing resource usage. It employs a Tweedie regression model for joint prediction of resource usage. The intensity of executable resource events is converted into Tweedie composite Poisson parameters to characterize the number of resource activation events and the probability of zero-occupancy states. The intensity of single-event resource usage is converted into Tweedie composite gamma parameters to characterize the positive continuous resource usage level and fluctuation degree formed by a single resource activation event. A dual-intensity consistency constraint restricts the two types of intensities from acting independently on their respective distribution components. The Tweedie regression model can uniformly handle a large number of zero-value and positive continuous samples, and the dual-intensity consistency constraint prevents the influence of executable resource event intensities and single-event resource usage intensities from mixing during model training, thereby improving the stability and interpretability of the resource usage prediction trajectory.

[0017] This invention employs Skorok reflection mapping to constrain and match the predicted resource occupancy trajectory with the executable resource trajectory of nodes. Through non-negative boundary constraints and recursive processing of unfulfilled resource occupancy, it identifies the resource occupancy in the predicted resource demand that cannot be promptly handled by computing nodes. Based on the task segment execution status, the unfulfilled resource occupancy is converted into the increment of executable resource event intensity, the increment of single-event resource occupancy intensity, and the intensity increment corresponding to the target computing node. Then, Tweedie regression and Skorok reflection mapping are iteratively executed. The use of Skorok reflection mapping can limit the statistical prediction results to the actual carrying capacity of computing nodes. The directional conversion and feedback iteration can form differentiated corrections for tasks that have not been started, are in progress but not completed, and are being migrated. This ensures that the final computing resource occupancy prediction result simultaneously reflects changes in resource demand and node carrying constraints, providing more accurate, stable, and execution-based prediction results for cross-border cold data collaborative scheduling between computing nodes in border parks and domestic computing nodes. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a cold data computing resource occupancy prediction method based on machine learning proposed in this invention; Figure 2 This diagram illustrates the iterative optimization of computing resource occupancy prediction based on Tweedie regression and Skorokhod reflection mapping, which is a machine learning-based cold data computing resource occupancy prediction method proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-2 A machine learning-based method for predicting cold data computing resource usage includes the following steps: Acquire cross-border cold data computing power operation data, divide it into task segments according to the cross-border task processing chain, and generate task resource status data; Based on task resource status data, a resource prediction infrastructure is generated according to resource activation relationships and prerequisite dependencies. Based on the resource prediction infrastructure, a set of resource activation risks is established. By combining the execution status and processing progress of task segments, the intensity of duplicate resource events is estimated, and the intensity of executable resource events is obtained. Based on the resource prediction infrastructure, a partial order structure of task segments is established. The contribution of the preceding sequence is deducted according to the preceding dependency to obtain the marginal processing amount of the task segment and pass it along the cross-border task processing chain to determine the resource occupancy intensity of a single event. The executable resource event intensity and single event resource occupancy intensity are converted into Tweedie composite Poisson parameters and composite gamma parameters, respectively, and the resource occupancy prediction trajectory is generated by using the Tweedie regression model. Extract the executable resource trajectory of nodes from the task resource status data, and use Skorok reflection mapping to constrain the resource occupation prediction trajectory according to the executable resource trajectory of nodes to obtain the unfulfilled resource occupation trajectory. Based on the execution status of the task segment, the unrealized resource usage is transformed into a specific category. Based on the transformation results, Tweedie regression and Skorokhod reflection mapping are iteratively executed to output the predicted computing power resource usage results.

[0021] In this embodiment, obtaining the task resource status data specifically includes: Cross-border cold data operation records are collected, and cross-border cold data computing power operation data is established according to task spatiotemporal association rules. The cross-border cold data operation records include task execution records, task queuing records, task processing records, resource allocation records, task migration records, and computing power node operation records. The task spatiotemporal association rules are as follows: cross-border cold data operation records corresponding to the same cross-border cold data task are associated using task identifiers; computing power node identifiers are used to determine the computing power nodes corresponding to the cross-border cold data tasks; and the time base of the cross-border cold data operation records is unified using the collection time. The cross-border cold data computing power operation data is a data set reflecting the task execution process, resource occupation process, and computing power node operation process of the cross-border cold data task from the access of overseas data to the return of processing results. The task identifier is used to associate the data formed by the same cross-border cold data task at different processing stages, and the computing power node identifier is used to associate the execution position of the cross-border cold data task on the computing power nodes in the border park and the domestic computing power nodes. Based on the cross-border cold data computing power operation data, the stage connection relationship of the cross-border cold data task is determined. A cross-border task processing chain is established based on the stage connection relationship. The cross-border task processing chain is divided into task segments according to the task segment division conditions, and a task segment identifier is configured for each task segment. The stage connection relationship refers to the data transmission relationship, execution sequence relationship, and computing power node affiliation relationship formed by the cross-border cold data task in the process of overseas data access, border park processing, domestic computing power collaboration, and processing result feedback. The cross-border task processing chain is a directed processing structure formed by connecting each processing stage of the cross-border cold data task. The task segment division conditions are the transformation of processing stage, transformation of computing power node affiliation, and transformation of resource occupancy status. The task segment is a processing unit in the cross-border task processing chain that has a continuous execution process, determines the computing power node affiliation, and has a continuous resource occupancy status. Based on the task status and node status corresponding to the task fragment at the acquisition time, time alignment is performed to generate task resource status data. The task status includes task fragment execution status, task fragment processing progress, task fragment resource occupancy status, and task fragment migration status. The node status includes task fragment computing power node affiliation status and computing power node executable resource status. The task fragment computing power node affiliation status is used to characterize the computing power node corresponding to the task fragment at the acquisition time. The computing power node executable resource status is used to characterize the amount of computing power resources that the computing power node can allocate to the task fragment at the acquisition time. The task resource status data is task status and node status association data organized according to task fragment identifier and acquisition time. The number of cross-border cold data tasks is set to 240, the total amount of cross-border cold data is set to 118 TB, the data size of a single cross-border cold data task is set to 80 GB to 2.5 TB, the number of computing power nodes in the border park is set to 8, the number of domestic computing power nodes is set to 16, and the time interval between acquisition times is set to 10 s. The 240 cross-border cold data tasks are divided into 1680 task fragments.

[0022] In this embodiment, obtaining the resource prediction infrastructure specifically includes: Based on task resource status data, the resource occupancy status of task segments is compared sequentially according to task segment identifier and acquisition time. Resource activation events are confirmed by combining task segment execution status and computing resource type. These resource activation events are then associated with the task segment identifier, computing resource type, and acquisition time to generate resource activation relationships. The sequential comparison is used to identify state changes in resource occupancy from zero to positive between adjacent acquisition times. A resource activation event is a state transition event formed when a task segment did not occupy computing resources in the previous acquisition time but begins to occupy computing resources in the current acquisition time. The task segment execution status is used to determine whether the task segment meets the resource execution conditions in the current acquisition time. The resource activation relationship records the task segment, computing resource type, activation time, and initial resource occupancy amount corresponding to the resource activation event. When the same task segment repeatedly enters the resource occupancy state in different acquisition periods, the corresponding resource activation events are recorded respectively, ensuring that the resource activation relationship retains the state characteristics of intermittent execution and repeated computing resource occupancy of cross-border cold data tasks. Identify the corresponding preceding task segments according to the data transmission direction and execution order between task segments. Establish prerequisite dependencies based on the completion status of the preceding task segments and the starting conditions of the current task segment. Update the prerequisite dependencies according to the collection time. The starting condition of the current task segment is that all preceding task segments with prerequisite dependencies have reached the completion status. The prerequisite dependencies are established according to the data transmission direction in the cross-border task processing chain, so that the branch processing process and the convergence processing process in the cross-border task processing chain maintain definite execution constraints. Task segments are identified as structural nodes, resource activation relationships are written into the corresponding structural nodes, preceding dependencies are converted into directed dependency connections between structural nodes, and task resource status data is associated with the corresponding structural nodes to form the basic structure for resource prediction. The directed dependency connections point from the structural node corresponding to the preceding task segment to the structural node corresponding to the current task segment. The resource activation relationships are written into the corresponding structural nodes according to the task segment identifier, and resource activation events are sorted according to the computing power resource type and collection time. The basic structure for resource prediction consists of structural nodes, resource activation relationships, and directed dependency connections.

[0023] In this embodiment, obtaining the executable resource event intensity specifically includes: Resource activation relationships, prerequisite dependencies, and task resource status data are extracted from the resource prediction infrastructure to determine the executable exposure intervals corresponding to task segments. The durations of task segments within the executable exposure intervals are sequentially connected to form the executable time axis of the task segments. A resource activation risk set is established based on the executable time axis of the task segments. The executable exposure interval is a continuous time interval in which the prerequisite dependencies of the task segments are satisfied, the task segments are eligible to start, and the computing power nodes corresponding to the task segments meet the resource execution conditions. The executable time axis of the task segments is a resource event risk exposure time benchmark formed after removing the waiting time, pause time, resource blocking time, and migration waiting time of the task segments. The resource activation risk set consists of task segments that are within the executable exposure interval and have the possibility of resource activation. The natural collection time corresponding to the task segments is set to 420 s, of which the waiting time is set to 70 s, the pause time is set to 20 s, the resource blocking time is set to 40 s, the migration waiting time is set to 20 s, and after deducting the non-executable time, an executable exposure time of 270 s is obtained. The prediction period is set to 60 min. Resource activation events in the resource activation relationship are mapped to the executable time axis of the task segment according to the corresponding collection time. Resource activation events that occur repeatedly in the same task segment are accumulated according to the task segment identifier. When the computing power node of the task segment is migrated, the executable exposure interval corresponding to the original computing power node is terminated. When the target computing power node meets the resource execution conditions, a new executable exposure interval is established. The accumulated results of resource activation events formed by the task segment and the processing progress of the task segment are transferred to the target computing power node to generate a cross-computing power node continuous event sequence. The cross-computing power node continuous event sequence is used to record the resource activation events that occur repeatedly in the same task segment before and after the computing power node migration, and to maintain the continuity of the resource activation event accumulation relationship and the task segment processing progress. The Andersen-Gill counting process uses the executable timeline of task segments instead of natural acquisition time as the counting time benchmark. Event counting intervals are divided according to the executable exposure interval. The event counting intervals corresponding to the original computing power node and the target computing power node for the same task segment are continuously connected. The cumulative number of resource activation events and the task segment processing progress are transferred to the target computing power node. Task segments participating in intensity estimation are determined based on the resource activation risk set. The benchmark resource event intensity and state influence relationship are updated based on the occurrence density of resource activation events within the executable exposure interval and the difference in task segment processing progress. The intensity of repeated resource events within a unit executable time is determined. Repeated resource event intensity is aggregated according to computing power resource type and computing power node, and time mapping is performed in conjunction with the length of the executable exposure interval within the prediction period to obtain the executable resource event intensity. The Andersen-Gill counting process is as follows: according to events... The counting process involves continuously accumulating resource activation events that occur repeatedly within the same task segment within a counting interval, and estimating the intensity of repeated resource events using a resource activation risk set that changes with the event counting interval and the task segment processing progress. The continuous connection refers to connecting the event counting interval corresponding to the original computing power node and the event counting interval corresponding to the target computing power node in the order of execution time when the same task segment undergoes computing power node migration, while maintaining the continuity of the accumulated number of resource activation events and the task segment processing progress. The state influence relationship refers to the influence of the difference in task segment processing progress on the level of resource activation events occurring within a unit executable time. The benchmark resource event intensity is a resource event occurrence benchmark determined based on the density of resource activation events occurring within the executable exposure interval. The time mapping is the conversion of the intensity of repeated resource events within a unit executable time into the level of the number of resource activation events within the prediction period based on the length of the executable exposure interval within the prediction period.

[0024] In this embodiment, obtaining the single-event resource occupancy intensity specifically includes: Extracting prerequisite dependencies and task resource status data from the resource prediction infrastructure, using task fragments as partial order elements, determining the partial order direction according to prerequisite dependencies, arranging task fragments according to the execution order in the cross-border task processing chain, establishing a task fragment partial order structure, and extracting the cumulative processing volume of task fragments at the acquisition time from the task resource status data. The task fragment partial order structure is a hierarchical structure formed by task fragments as structural nodes and prerequisite dependencies as directed connections. The cumulative processing volume is the processing volume of task fragments that has been received and processed up to the acquisition time. The cumulative processing volume is uniformly converted from the input data processing volume, computational workload, and intermediate result processing volume. Along the partial order structure of task segments, the source of the cumulative processing volume is traced from the starting task segment to the ending task segment. The source task segment identifier and the transmission path identifier are configured for the processing volume of the previous task segment into the current task segment. At the branch position, the previous contribution volume leading to different downstream task segments is retained. At the merging position, the same source previous contribution volume arriving through different transmission paths is merged according to the source task segment identifier to generate a previous contribution record. The previous contribution volume is the processing volume of the previous task segment that has been written into the cumulative processing volume of the current task segment. The same source previous contribution volume is the previous contribution volume with the same source task segment that enters the same current task segment through different transmission paths. The previous contribution record is used to save the source, transmission path, partial order level and merging position of the previous contribution volume. Task segments are processed sequentially according to the partial order hierarchy in the partial order structure of the task segments. The partial order Möbius inversion is used to deduct the previous contribution corresponding to the previous contribution record from the cumulative processing amount of the current task segment. The same source previous contribution that is repeatedly written through different transmission paths is deducted once. The repeated deduction caused by the overlap of the partial order hierarchy is replenished according to the partial order hierarchy. The processing amount after deduction and replenishment is determined as the marginal processing amount of the task segment. The partial order Möbius inversion is a process of separating the newly added processing contribution of the current task segment layer by layer according to the inclusion relationship in the partial order structure of the task segments. When the initial task segment does not have a previous contribution, the cumulative processing amount is determined as the marginal processing amount of the task segment. For non-initial task segments, the cumulative processing amount is deducted from the previous contribution already formed by the previous task segment to obtain the marginal processing amount of the task segment. According to the data transmission direction in the cross-border task processing chain, the marginal processing amount of the task segment is written into the transmission connection corresponding to the adjacent downstream task segment. The marginal processing amount of the task segment in the transmission connection is adjusted according to the data conversion relationship. At the branch position, the marginal processing amount of the task segment is allocated according to the actual receiving ratio of the downstream task segment. At the merging position, duplicate incoming amounts are removed according to the source task segment identifier and the remaining incoming amounts are collected to generate the marginal processing amount transmission result. The data conversion relationship is the transformation relationship between the input processing amount and the output processing amount formed when cross-border cold data is transmitted between task segments. The marginal processing amount transmission result records the transmission direction, transmission value and target task segment of the marginal processing amount of the task segment in the cross-border task processing chain. The marginal processing volume actually undertaken by a task segment in a single resource activation event is determined based on the marginal processing volume transfer result. The positive resource occupancy corresponding to the resource activation event is extracted from the task resource status data. A resource occupancy conversion relationship between marginal processing volume and positive resource occupancy is established according to the task segment identifier, computing power resource type, and computing power node. The marginal processing volume actually undertaken by a task segment in a single resource activation event is converted into single-event resource occupancy intensity using the resource occupancy conversion relationship. The resource occupancy conversion relationship is the positive resource occupancy level corresponding to the marginal processing volume of a unit task segment. The positive resource occupancy is the resource occupancy formed during the continuous use of computing power resources by the task segment after the occurrence of the resource activation event. The single-event resource occupancy intensity is the positive continuous resource occupancy level formed by the corresponding computing power resource type when a task segment experiences a single resource activation event.

[0025] This invention separates the preceding contribution and the newly added processing volume of task segments in the cross-border task processing chain through the partial order structure of task segments and the partial order Möbius inversion. It also combines the source task segment identifier, the transmission path identifier, the branch allocation, the deduplication of the convergence and the data transformation relationship to reduce the calculation deviation caused by the repeated inclusion of the same source processing volume and the overlap of levels. This improves the accuracy of determining the marginal processing volume of task segments and the resource occupancy intensity of single events, and provides a reliable input for predicting the computing power resource occupancy of cross-border cold data.

[0026] In this embodiment, obtaining the resource occupancy prediction trajectory specifically includes: Based on the computing power node, computing power resource type, and collection time, the intensity of executable resource events, the intensity of single event resource occupancy, and the task resource status data are associated. The actual resource occupancy corresponding to the collection time is used as the supervision label to form Tweedie regression training samples. The supervision label is the actual resource occupancy of the computing power node extracted from the task resource status data after the collection time ends. The intensity of executable resource events is accumulated over time according to the acquisition time. The accumulated time result is converted into Tweedie composite Poisson parameters. Training samples with supervision labels greater than zero are selected from the Tweedie regression training samples to form positive resource occupancy samples. Tweedie composite gamma parameters are generated based on the dispersion of single-event resource occupancy intensity and positive resource occupancy samples. The Tweedie composite Poisson parameters and Tweedie composite gamma parameters are written into the corresponding Tweedie regression training samples. The conversion process of Tweedie composite Poisson parameters is as follows: the intensity of executable resource events at each time point between adjacent acquisition times is multiplied by the corresponding time interval, and the products are accumulated. The accumulated result is determined as the expected resource occupancy between adjacent acquisition times. The number of resource activation events is determined by the Tweedie composite Poisson parameter, which controls the expected number of resource activation events and the probability of zero occupancy. Increasing the Tweedie composite Poisson parameter increases the expected number of resource activation events and decreases the probability of zero occupancy. The Tweedie composite gamma parameter includes a shape parameter and a scale parameter. The shape parameter is determined by dividing the square of the single-event resource occupancy intensity by the occupancy variance of the positive resource occupancy sample, and is used to characterize the concentration of positive continuous resource occupancy formed by a single resource activation event around the single-event resource occupancy intensity. The scale parameter is determined by dividing the occupancy variance of the positive resource occupancy sample by the single-event resource occupancy intensity, and is used to characterize the fluctuation scale of positive continuous resource occupancy formed by a single resource activation event. Dual-intensity resource prediction features are established based on Tweedie composite Poisson parameters and Tweedie composite gamma parameters. The dual-intensity resource prediction features and task resource status data are input into the Tweedie regression model to obtain the resource occupancy prediction value. At the same time, a supervised training method is used to fit the mapping relationship between the resource prediction features and the supervision labels. Each row of the dual-intensity resource prediction feature matrix corresponds to a computing power node forming a Tweedie regression training sample for a certain type of computing power resource at a collection time. The columns are written sequentially with the Tweedie composite Poisson parameters, Tweedie composite gamma parameters, and numerical features in the task resource status data that are associated with the corresponding computing power node, computing power resource type, and collection time. All Tweedie regression training samples are arranged into a fixed-length numerical matrix according to a uniform column order. Missing features are filled in according to a preset missing value filling rule. The model parameters are updated based on the prediction deviation between the predicted resource usage and the supervision label. A dual-strength consistency constraint is established based on the deviation between the number of resource activation events corresponding to the predicted resource usage and the intensity of executable resource events, and the deviation between the positive continuous resource usage component and the intensity of single-event resource usage. This model parameter update is repeated until the prediction deviation and the dual-strength consistency deviation satisfy the training convergence condition, resulting in a trained Tweedie regression model. The model parameters include feature weights, regression intercept, power exponent, discrete parameters, compound Poisson correction weights, and compound gamma correction weights. The model parameter update involves adjusting the feature weights and regression intercept using the prediction deviation, adjusting the compound Poisson correction weights using the deviation corresponding to the number of resource activation events, adjusting the compound gamma correction weights using the deviation corresponding to the positive continuous resource usage component, and adjusting the power exponent and discrete parameters using the resource usage distribution deviation. The dual-strength consistency constraint limits the executable resource event intensity to the number of resource activation events and limits the single-event resource usage intensity to the positive continuous resource usage component. The training convergence condition is that both the prediction deviation and the dual-strength consistency deviation are not greater than their corresponding thresholds. Based on the computing power node, computing power resource type and the collection time within the prediction period, generate dual-intensity resource prediction features corresponding to the prediction stage. Input the dual-intensity resource prediction features and task resource status data into the trained Tweedie regression model to obtain the corresponding zero occupancy probability, positive continuous resource occupancy level and resource occupancy prediction value. Connect the resource occupancy prediction values ​​according to the time sequence of the collection time to generate a resource occupancy prediction trajectory. The resource occupancy prediction trajectory is used to characterize the process of the computing power resource corresponding to the computing power node changing from a zero occupancy state to a positive continuous occupancy state, maintaining a positive continuous occupancy state and returning to a zero occupancy state within the prediction period. The Tweedie regression model comprises a feature input layer, a linear prediction layer, a logarithmic connection layer, and a Tweedie distribution output layer. The feature input layer receives a fixed-length numerical vector from the dual-intensity resource prediction feature matrix. The linear prediction layer applies trainable weights to each feature in the fixed-length numerical vector and performs a weighted combination. The logarithmic connection layer converts the weighted combination result into a positive mean resource occupancy value. The Tweedie distribution output layer determines the probability of zero occupancy and the level of resource activation events based on the Tweedie composite Poisson parameter, and determines the positive continuous resource occupancy level and volatility corresponding to resource activation events based on the Tweedie composite gamma parameter. It then combines the zero occupancy state and the positive continuous occupancy state to form a predicted resource occupancy value. Supervised training uses the Tweedie deviation between the predicted resource occupancy value and the supervision label as a benchmark. The training loss is formed by weighting the main loss, the zero-occupancy constraint loss by the deviation between the probability of the zero-occupancy state and the zero-occupancy state corresponding to the supervision label, and the positive occupancy constraint loss by the deviation between the positive continuous resource occupancy level and the positive resource occupancy amount corresponding to the supervision label. The training loss is formed by weighting the main loss, the zero-occupancy constraint loss and the positive occupancy constraint loss according to the training weights. The trainable weights of the linear prediction layer and the distribution parameters of the Tweedie distribution output layer are updated in reverse according to the training loss. The training control parameters include the initial learning rate, batch size, maximum number of training rounds, regularization coefficient and training weights. The initial learning rate is adjusted by decreasing according to the change of training loss in adjacent training rounds. The regularization coefficient is used to limit the change of trainable weights. Supervised training stops when the decrease of training loss in consecutive training rounds is not greater than the training convergence threshold. The Tweedie regression training sample size was set to 120,000 sets, the ratio of training samples, validation samples, and test samples was set to 8:1:1, the batch size was set to 128, the initial learning rate was set to 0.001, the maximum number of training epochs was set to 300, the regularization coefficient was set to 0.0001, the initial value of the power exponent was set to 1.5, and the power exponent ranged from 1.1 to 1.9. The Tweedie regression model was considered to have met the training convergence condition when the training loss decreased by no more than 0.0001 after 10 consecutive training epochs, and the double strength consistency deviation was no more than 0.01.

[0027] This invention uses the Tweedie regression model to uniformly characterize the zero-occupancy and positive continuous occupancy states of cross-border cold data computing resources. It also utilizes dual-strength consistency constraints to maintain the independent relationship between the number of resource activation events and the resource occupancy level of a single event. Combined with supervised training and parameter convergence control, it improves the accuracy, stability, and interpretability of resource occupancy prediction trajectories, providing a reliable basis for resource allocation and collaborative scheduling of computing nodes in border parks and domestic computing nodes.

[0028] In this embodiment, obtaining the unfulfilled resource usage trajectory specifically includes: Extract node resource status from task resource status data, perform node executable resource quantity generation processing to obtain node executable resource trajectory. The node executable resource quantity generation processing is as follows: according to computing power node, computing power resource type and collection time, deduct the unallocated resource quantity from the total node resource quantity, and determine the deduction result as the node executable resource quantity. For node executable resource quantities with a value less than zero, zero value correction is performed, and the node executable resource quantities formed by connecting the same computing power node for the same computing power resource type according to the collection time sequence are obtained. The unallocated resource quantity includes basic operation occupancy, queued reserved resource quantity and migration locked resource quantity. The node executable resource trajectory is used to characterize the resource quantity change process that can support cross-border cold data tasks during the prediction period. Based on the executable resource trajectory and resource occupancy prediction trajectory of a node, resource supply and demand alignment processing is performed to obtain a resource supply and demand alignment sequence. The resource supply and demand alignment processing is as follows: align the executable resource trajectory and resource occupancy prediction trajectory of a node according to the computing power node, computing power resource type and collection time. According to the time interval between adjacent collection times, the predicted resource occupancy value is converted into the predicted resource demand, the executable resource quantity of the node is converted into the executable resource supply quantity, and a correspondence between the predicted resource demand and the executable resource supply quantity is established according to the collection time sequence. The resource supply and demand alignment sequence records the predicted resource demand and executable resource supply quantity formed by the same computing power node for the same computing power resource type within the interval between adjacent collection times. A Skorokhod reflection mapping is performed on the resource supply and demand alignment sequence, and non-negative boundary constraints and unfulfilled resource occupancy recursion processing are applied to obtain the unfulfilled resource occupancy trajectory. The Skorokhod reflection mapping is a process of mapping the trajectory of the resource supply and demand difference formed by the predicted resource demand and the executable resource supply to a non-negative state space. The non-negative boundary constraint uses zero as the reflection boundary. When the result of subtracting the current executable resource supply from the current demand to be borne is lower than zero, the result is adjusted to zero using the minimum reflection adjustment amount that meets the non-negativity requirement. If the value is not lower than zero, the result is retained. The recursive processing of the unfulfilled resource occupancy is as follows: according to the collection time sequence, the unfulfilled resource occupancy at the end of the previous adjacent collection time interval is combined with the current predicted resource demand to form the current resource demand to be carried. The non-negative result obtained by subtracting the current executable resource supply from the current resource demand to be carried is determined as the current unfulfilled resource occupancy. The current unfulfilled resource occupancy is then passed to the next adjacent collection time interval. The unfulfilled resource occupancy trajectory is formed by connecting the unfulfilled resource occupancy corresponding to each adjacent collection time interval according to the collection time sequence.

[0029] This invention combines changes in executable resources of nodes, resource occupancy prediction results, and the actual carrying capacity of computing nodes to uniformly constrain predicted resource demand and executable resource supply. It also utilizes Skorokhod reflection mapping to continuously represent the amount of resource occupancy that cannot be realized in time, thereby avoiding prediction results from exceeding the node's carrying capacity. This improves the accuracy, stability, and dynamic adaptability of cross-border cold data computing resource occupancy prediction, and provides a reliable basis for resource allocation and collaborative scheduling between computing nodes in border parks and domestic computing nodes.

[0030] In this embodiment, obtaining the computing resource occupancy prediction result specifically includes: Based on the execution status of the task segment, the unfulfilled resource occupancy in the unfulfilled resource occupancy trajectory is allocated to the corresponding task segment and status-oriented processing is performed to obtain the status-oriented resource occupancy. The unfulfilled resource occupancy is allocated proportionally according to the predicted resource demand contribution formed by the executable resource event intensity and single event resource occupancy intensity corresponding to the task segment. The status-oriented resource occupancy includes unstarted resource occupancy, incomplete execution resource occupancy, and migration resource occupancy. The intensity conversion process is performed on the resource occupancy of the state to obtain the conversion result. The intensity conversion process is to convert the unstarted resource occupancy into the intensity increment of executable resource events, the incomplete resource occupancy during execution into the intensity increment of single event resource occupancy, and the migration resource occupancy into the intensity increment of executable resource events corresponding to the target computing power node. The intensity increments are then used to update the intensity of executable resource events and the intensity of single event resource occupancy. Input the direction conversion results and task resource status data into the trained Tweedie regression model to generate an updated resource occupancy prediction trajectory. Input the updated resource occupancy prediction trajectory and the node executable resource trajectory into Skorokhod reflection mapping to generate an updated unfulfilled resource occupancy trajectory. Repeat the state direction conversion process and intensity conversion process based on the updated unfulfilled resource occupancy trajectory. Compare the resource usage prediction trajectories formed by two adjacent iterations, and normalize the differences in resource usage prediction values ​​corresponding to different computing power resource types. Stop iterating when the differences in the normalized resource usage prediction trajectories are all no greater than the corresponding trajectory convergence threshold, and output the computing power resource usage prediction result. The trajectory convergence threshold is the upper limit of the difference between the resource usage prediction trajectories formed by two adjacent iterations to determine whether they have reached a stable state. The computing power resource usage prediction result is the resource usage prediction trajectory and the unrealized resource usage trajectory organized according to computing power nodes, computing power resource types, and collection time. It is used to respectively characterize the resource usage change process that can be borne by computing power nodes and the resource usage change process that cannot be borne by the executable resources of nodes in a timely manner during the prediction period.

[0031] Example 1: To verify the feasibility of the present invention in practice, the present invention was applied to a test environment for cold data processing and computing resource management in a cross-border digital park. The test environment was used to receive cold data from the overseas side, complete data verification, format conversion, feature extraction, compression optimization and result encapsulation, and call the computing power nodes of the border park and domestic computing power nodes according to the task processing requirements. After processing, the result data was sent back to the overseas side. The test data in this example is used to illustrate the specific execution process and technical effects of the present invention and does not constitute a limitation on the scope of protection.

[0032] The test environment was configured with 8 computing nodes in border parks and 16 domestic computing nodes. Computing resources included CPUs, GPU memory, RAM, and storage read / write resources. The nodes were configured with a total of 1536 CPU cores, 48 ​​GPUs, 6 TB of RAM, and 420 TB of storage. The cross-border data transfer bandwidth was set to 20 Gbit / s. The data collection interval for node and task running status was set to 10 seconds, and the prediction period was set to 60 minutes. Data was collected continuously for 30 running days. During the test, 240 cross-border cold data tasks were received, totaling 118 TB. Individual task data ranged from 80 GB to 2.5 TB. Task content included image archive data optimization, device operation log compression, historical business data cleaning, and offline model feature generation.

[0033] The cross-border cold data task begins with the access of data from overseas, and proceeds sequentially through border park verification and processing, border park preprocessing, domestic computing power collaborative processing, result packaging, and result feedback. Based on the processing stage transitions, computing power node affiliation transitions, and resource occupancy status transitions, the 240 cross-border cold data tasks are divided into 1680 task segments, and each task segment is assigned a task segment identifier. According to the task segment identifier and the collection time, the task execution status, task segment processing progress, resource occupancy status, task migration status, computing power node affiliation status, and computing power node executable resource status are correlated to form task resource status data.

[0034] During the test task execution, a total of 8460 resource activation events and 1934 prerequisite dependencies were generated. Some task segments lacked the conditions for resource execution while waiting for overseas data transmission to complete, for the output of preceding task segments, for the release of the graphics processor, and for the migration of computing power nodes. Conventional resource usage statistics methods still calculate the frequency of resource events based on natural acquisition time, which easily includes waiting time in the resource event exposure time. This invention establishes an executable timeline for task segments based on the executable exposure interval, removing waiting time, pause time, resource blocking time, and migration waiting time from the natural acquisition time.

[0035] One cross-border cold data task segment lasted 420 seconds from entering the task queue to completion, with a 70-second delay for prerequisites, a 40-second delay for resource blocking, a 20-second pause, a 20-second migration delay, and an actual executable exposure time of 270 seconds. The task segment generated four resource activation events at the border park computing nodes and migrated to domestic computing nodes when the task segment processing progress reached 62%. This invention transmits the cumulative results of the four resource activation events and the 62% task segment processing progress to the domestic computing nodes, and establishes a new executable exposure interval when the domestic computing nodes meet the resource execution conditions, thereby maintaining the continuity of the cumulative relationship of resource activation events before and after migration. After performing the Andersen-Gill counting process based on the executable time axis of the task segment, the intensity of repeated resource events per unit executable time is obtained, and the executable resource event intensity is formed based on the executable exposure time within a 60-minute prediction period.

[0036] In cross-border task processing chains, branching and merging processes occur. The same upstream data may enter the same task segment via different transmission paths. Directly using the cumulative processing volume would result in duplicate calculations of the same-source processing volume. This invention identifies a 60 GB prior contribution based on the source task segment identifier and transmission path identifier. A partial-order Möbius inversion is used to deduct this prior contribution, and the duplicate deductions caused by overlapping partial-order levels are replenished, resulting in a 420 GB marginal processing volume for the task segment. A resource utilization conversion relationship is established based on the marginal processing volume and the corresponding positive resource utilization, determining the positive resource utilization level corresponding to each unit of marginal processing volume and forming a single-event resource utilization intensity.

[0037] The Tweedie regression model uses 120,000 training samples, with a training, validation, and test sample ratio of 8:1:1. Each sample corresponds to one computing node, one type of computing resource, and one data collection time. The batch size is set to 128, the initial learning rate is set to 0.001, the maximum number of training epochs is set to 300, the regularization coefficient is set to 0.0001, the initial value of the power exponent is set to 1.5, and the power exponent range is set to 1.1 to 1.9. Training stops when the training loss decreases by no more than 0.0001 after 10 consecutive training epochs, and the double-strength consistency deviation is no more than 0.01.

[0038] The intensity of executable resource events, accumulated over time, forms the Tweedie composite Poisson parameter, used to characterize the expected number of resource activation events and the probability of a zero-occupancy state. The intensity of single-event resource occupancy, combined with the dispersion of positive resource occupancy samples, forms the Tweedie composite gamma parameter, used to characterize the positive continuous resource occupancy level and volatility corresponding to a single resource activation event. The dual-intensity consistency constraint restricts the intensity of executable resource events to affect the number of resource activation events and restricts the intensity of single-event resource occupancy to affect the positive continuous resource occupancy portion, preventing the model from using a single feature to simultaneously compensate for deviations in the number of resource events and deviations in resource occupancy magnitude.

[0039] In the node load constraint test, a domestic computing node is configured with 64 CPU cores, of which 8 CPU cores are used for basic operation, 12 CPU cores are reserved in the queue, 4 CPU cores are locked for migration, and 40 CPU cores are available for execution. Skorokhod reflection mapping determines the current resource demand to be carried as 490 CPU cores. After deducting the available resource supply of 400 CPU cores, 90 CPU cores are left as unfulfilled resource occupancy, and these 90 CPU cores are passed to the next collection interval.

[0040] This invention converts the unrealized resource occupancy corresponding to unstarted task segments into executable resource event intensity increments, converts the unrealized resource occupancy corresponding to incomplete task segments into single-event resource occupancy intensity increments, and converts the unrealized resource occupancy corresponding to migration task segments into executable resource event intensity increments for the target computing power node. It then reconstructs Tweedie composite Poisson parameters, Tweedie composite gamma parameters, and dual-intensity resource prediction features, and repeatedly executes Tweedie regression, resource supply and demand alignment processing, and Skorokhod reflection mapping. After six iterations, the difference in the maximum normalized resource occupancy prediction trajectory corresponding to each computing power resource type is no greater than 0.02, and the resource occupancy prediction trajectory reaches a stable state.

[0041] To verify the technical effectiveness of this invention, the historical average prediction method, the ordinary regression prediction method, and this invention were tested using the same task resource status data. The historical average prediction method uses the average resource usage of the same computing node during the historical data collection period as the prediction result for the corresponding resource usage during the prediction period. The ordinary regression prediction method uses the task resource status data as input and fits the relationship between resource status characteristics and actual resource usage through a conventional regression model. It does not distinguish between zero-occupancy and positive continuous-occupancy states, nor does it introduce node executable resource constraints. All three methods use the same training sample division, prediction period, computing node configuration, and resource scheduling rules. The comparison results are shown in Table 1. Table 1. Comparison of the overall performance of different computing resource utilization prediction methods

[0042] As shown in Table 1, this invention demonstrates high comprehensive performance in predicting the resource occupancy of cross-border cold data computing power. Compared with historical average prediction methods and ordinary regression prediction methods, the average absolute percentage error of resource occupancy is reduced from 18.7% and 12.4% to 6.8%, respectively; the root mean square error of CPU resource occupancy is reduced from 11.6 cores and 8.2 cores to 4.5 cores, respectively; the accuracy of zero occupancy status identification is improved to 96.2%; the average prediction error of positive continuous resource occupancy is reduced to 6.1%; the intensity error of repeated resource activation events is reduced to 7.2%; the identification rate of unfulfilled resource occupancy is improved to 94.1%; the over-allocation ratio of node resources is reduced to 3.8%; the continuity rate of prediction trajectory after computing power node migration is improved to 97.5%; and the average task queuing time is shortened to 18.6 min. Although the average calculation time for a single prediction period is increased to 5.6 s, it still meets the real-time processing requirements for cross-border cold data computing power resource prediction and collaborative scheduling.

[0043] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for predicting cold data computing resource occupancy based on machine learning, characterized in that, Includes the following steps: Acquire cross-border cold data computing power operation data, divide it into task segments according to the cross-border task processing chain, and generate task resource status data; Based on task resource status data, a resource prediction infrastructure is generated according to resource activation relationships and prerequisite dependencies. Based on the resource prediction infrastructure, a set of resource activation risks is established. By combining the execution status and processing progress of task segments, the intensity of duplicate resource events is estimated, and the intensity of executable resource events is obtained. Based on the resource prediction infrastructure, a partial order structure of task segments is established. The contribution of the preceding sequence is deducted according to the preceding dependency to obtain the marginal processing amount of the task segment and pass it along the cross-border task processing chain to determine the resource occupancy intensity of a single event. The executable resource event intensity and single event resource occupancy intensity are converted into Tweedie composite Poisson parameters and composite gamma parameters, respectively, and the resource occupancy prediction trajectory is generated by using the Tweedie regression model. Extract the executable resource trajectory of nodes from the task resource status data, and use Skorok reflection mapping to constrain the resource occupation prediction trajectory according to the executable resource trajectory of nodes to obtain the unfulfilled resource occupation trajectory. Based on the execution status of the task segment, the unrealized resource usage is transformed into a specific category. Based on the transformation results, Tweedie regression and Skorokhod reflection mapping are iteratively executed to output the predicted computing power resource usage results.

2. The method for predicting cold data computing resource occupancy based on machine learning according to claim 1, characterized in that, The acquisition of the task resource status data specifically includes: Collect cross-border cold data operation records and establish cross-border cold data computing power operation data according to task spatiotemporal association rules; Based on the cross-border cold data computing power operation data, determine the stage connection relationship of the cross-border cold data task, establish a cross-border task processing chain based on the stage connection relationship, divide the cross-border task processing chain into task segments according to the task segment division conditions, and configure a task segment identifier for each task segment. Time alignment is performed based on the task status and node status corresponding to the task segment at the time of acquisition to generate task resource status data.

3. The method for predicting cold data computing resource occupancy based on machine learning according to claim 1, characterized in that, The acquisition of the resource forecasting infrastructure specifically includes: Based on task resource status data, the resource occupancy status of task segments is compared sequentially according to task segment identifier and collection time. The resource activation event is confirmed by combining the task segment execution status and computing resource type. The resource activation event is associated with task segment identifier, computing resource type and collection time to generate resource activation relationship. Identify the corresponding preceding task segments according to the data transfer direction and execution order between task segments, establish the preceding dependency relationship based on the completion status of the preceding task segment and the start condition of the current task segment, and update the preceding dependency relationship according to the collection time. Task fragments are identified as structural nodes, resource activation relationships are written into the corresponding structural nodes, prerequisite dependencies are converted into directed dependency connections between structural nodes, and task resource status data is associated with the corresponding structural nodes to form the basic structure for resource prediction.

4. The method for predicting cold data computing resource occupancy based on machine learning according to claim 1, characterized in that, The determination of the executable resource event intensity specifically includes: Resource activation relationships, prerequisite dependencies, and task resource status data are extracted from the resource prediction infrastructure to determine the executable exposure intervals corresponding to task segments. The durations of task segments within the executable exposure intervals are sequentially connected to form the executable time axis of the task segments. A resource activation risk set is then established based on the executable time axis of the task segments. The resource activation events in the resource activation relationship are mapped to the executable time axis of the task segment according to the corresponding collection time. The resource activation events that occur repeatedly in the same task segment are accumulated according to the task segment identifier. When the computing power node of the task segment is migrated, the executable exposure interval corresponding to the original computing power node is terminated. When the target computing power node meets the resource execution conditions, a new executable exposure interval is established. The accumulated results of the resource activation events formed by the task segment and the processing progress of the task segment are transferred to the target computing power node to generate a continuous event sequence across computing power nodes. The executable timeline of task segments is used as the counting time benchmark for the Andersen-Gill counting process, replacing the natural acquisition time. Event counting intervals are divided according to the executable exposure interval. The event counting intervals corresponding to the original computing power node and the target computing power node for the same task segment are continuously connected. The cumulative number of resource activation events and the processing progress of task segments are transferred to the target computing power node. The task segments participating in intensity estimation are determined according to the resource activation risk set. Based on the occurrence density of resource activation events within the executable exposure interval and the difference in processing progress of task segments, the baseline resource event intensity and state influence relationship are updated. The intensity of repeated resource events in a unit executable time is determined. The intensity of repeated resource events is aggregated according to computing power resource type and computing power node, and time mapping is performed in combination with the length of the executable exposure interval within the prediction period to obtain the executable resource event intensity.

5. The method for predicting cold data computing resource occupancy based on machine learning according to claim 1, characterized in that, The determination of the single-event resource occupancy intensity specifically includes: Extract the preceding dependencies and task resource status data from the resource prediction infrastructure, use task fragments as partial order elements, determine the partial order direction according to the preceding dependencies, arrange the task fragments according to the execution order in the cross-border task processing chain, establish the task fragment partial order structure, and extract the cumulative processing amount of the task fragments at the time of collection from the task resource status data. Along the partial order structure of task segments, the source of the cumulative processing volume is traced from the starting task segment to the ending task segment. The source task segment identifier and the transmission path identifier are configured for the processing volume from the previous task segment to the current task segment. At the branch position, the previous contribution volume leading to different downstream task segments is retained. At the merging position, the same source previous contribution volume arriving through different transmission paths is merged according to the source task segment identifier to generate a previous contribution record. Task segments are processed sequentially according to the partial order hierarchy in the partial order structure of the task segments. The previous contribution amount corresponding to the previous contribution record is deducted from the cumulative processing amount of the current task segment using partial order Möbius inversion. The same source previous contribution amount that is repeatedly written through different transmission paths is deducted once. The repeated deduction amount caused by the overlap of partial order hierarchy is replenished according to the partial order hierarchy. The processing amount after deduction and replenishment is determined as the marginal processing amount of the task segment. According to the data transmission direction in the cross-border task processing chain, the marginal processing amount of the task segment is written into the transmission connection corresponding to the adjacent downstream task segment. The marginal processing amount of the task segment in the transmission connection is adjusted according to the data conversion relationship. At the branch position, the marginal processing amount of the task segment is allocated according to the actual receiving ratio of the downstream task segment. At the merging position, the duplicate incoming amount is removed according to the source task segment identifier and the remaining incoming amount is collected to generate the marginal processing amount transmission result. Based on the marginal processing volume transfer results, determine the actual marginal processing volume undertaken by the task segment in a single resource activation event. Extract the positive resource occupancy corresponding to the resource activation event from the task resource status data. Establish a resource occupancy conversion relationship between marginal processing volume and positive resource occupancy according to the task segment identifier, computing power resource type, and computing power node. Use the resource occupancy conversion relationship to convert the actual marginal processing volume undertaken by the task segment in a single resource activation event into single-event resource occupancy intensity.

6. The method for predicting cold data computing resource occupancy based on machine learning according to claim 1, characterized in that, The acquisition of the resource occupancy prediction trajectory specifically includes: Based on the computing power node, computing power resource type, and collection time, the intensity of executable resource events, the intensity of single event resource occupancy, and the task resource status data are correlated. The actual resource occupancy corresponding to the collection time is used as the supervision label to form Tweedie regression training samples. The intensity of executable resource events is accumulated over time according to the collection time. The accumulated time result is converted into Tweedie composite Poisson parameters. Training samples with supervision labels greater than zero are selected from the Tweedie regression training samples to form positive resource occupancy samples. Tweedie composite gamma parameters are generated based on the intensity of single event resource occupancy and the dispersion of positive resource occupancy samples. The Tweedie composite Poisson parameters and Tweedie composite gamma parameters are written into the corresponding Tweedie regression training samples. Based on the Tweedie composite Poisson parameter and the Tweedie composite gamma parameter, a dual-intensity resource prediction feature is established. The dual-intensity resource prediction feature and the task resource status data are input into the Tweedie regression model to obtain the resource occupancy prediction value. At the same time, a supervised training method is used to fit the mapping relationship between the resource prediction feature and the supervision label. The model parameters are updated based on the prediction deviation between the predicted resource usage and the supervision label. A dual-intensity consistency constraint is established based on the deviation between the number of resource activation events corresponding to the predicted resource usage and the intensity of executable resource events, and the deviation between the positive continuous resource usage and the intensity of single event resource usage. The model parameter update is repeated until the prediction deviation and the dual-intensity consistency deviation meet the training convergence condition, and the trained Tweedie regression model is obtained. Based on the computing power node, computing power resource type, and collection time within the prediction period, generate dual-intensity resource prediction features corresponding to the prediction stage. Input the dual-intensity resource prediction features and task resource status data into the trained Tweedie regression model to obtain the corresponding zero occupancy probability, positive continuous resource occupancy level, and resource occupancy prediction value. Connect the resource occupancy prediction values ​​according to the time sequence of the collection time to generate the resource occupancy prediction trajectory.

7. The method for predicting cold data computing resource occupancy based on machine learning according to claim 1, characterized in that, The acquisition of the unfulfilled resource usage trajectory specifically includes: Extract node resource status from task resource status data, perform node executable resource quantity generation processing, and obtain node executable resource trajectory; Based on the executable resource trajectory and predicted resource occupancy trajectory of the node, perform resource supply and demand alignment processing to obtain a resource supply and demand alignment sequence; Skorokhod reflection mapping is performed on the resource supply and demand alignment sequence, and non-negative boundary constraints and unfulfilled resource occupancy recursion processing are performed to obtain the unfulfilled resource occupancy trajectory.

8. The method for predicting cold data computing resource occupancy based on machine learning according to claim 1, characterized in that, The specific methods for obtaining the computing resource occupancy prediction results include: Based on the execution status of the task segment, the unfulfilled resource usage in the unfulfilled resource usage trajectory is allocated to the corresponding task segment and status-based processing is performed to obtain the status-based resource usage. Perform intensity conversion processing on the resource occupancy of each state and obtain the conversion result; Input the direction conversion results and task resource status data into the trained Tweedie regression model to generate an updated resource occupancy prediction trajectory. Input the updated resource occupancy prediction trajectory and the node executable resource trajectory into Skorokhod reflection mapping to generate an updated unfulfilled resource occupancy trajectory. Repeat the state direction conversion process and intensity conversion process based on the updated unfulfilled resource occupancy trajectory. Compare the resource usage prediction trajectories formed by two adjacent iterations, normalize the differences in resource usage prediction values ​​corresponding to different computing power resource types, and stop iterating when the differences in the normalized resource usage prediction trajectories are not greater than the corresponding trajectory convergence threshold, and output the computing power resource usage prediction results.

Images