Machine learning based cloud storage capacity intelligent prediction and elastic expansion system

Abstract

The application discloses a cloud storage capacity intelligent prediction and elastic expansion system based on machine learning and relates to the technical field of cloud computing.The system comprises a data arrangement module, a settlement tracking module, an unsettled arrangement module, a capacity prediction module, a section positioning module and a branch generation module.The data arrangement module is used to form capacity basic data;the settlement tracking module is used to form a settlement chain;the unsettled arrangement module is used to form an unsettled sequence;the capacity prediction module is used to input the unsettled sequence into an improved MICN model and output a predicted unsettled sequence;the section positioning module is used to form a settlement section;the branch generation module is used to form a candidate expansion branch;and the expansion execution module is used to screen a target expansion branch and issue an elastic expansion instruction to a cloud storage resource management interface according to the target expansion branch.The application can improve the capacity prediction accuracy and the elastic expansion reliability.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and in particular to a cloud storage capacity intelligent prediction and elastic expansion system based on machine learning. Background Technology

[0002] With the continuous deployment of object storage, distributed file storage, and cloud disk services, data write, overwrite, migration, replication, and deletion operations in cloud storage clusters are constantly increasing, resulting in significant time-series fluctuations in storage capacity. Existing cloud storage capacity management typically triggers expansion based on capacity utilization thresholds, historical average growth rates, or fixed-period predictions, and accesses storage resources according to preset capacity increments. This approach struggles to differentiate the different impacts of capacity formation, data migration, data migration, and space release on the actual available capacity. It easily includes incomplete data deletion or migration / release in the available capacity prematurely, or ignores short-term capacity increases caused by concentrated writes and batch migrations. Furthermore, existing capacity prediction models often use parallel extraction and unified aggregation methods for changing characteristics at different time scales, easily leading to duplicate features and prediction time-series bias. Expansion decisions typically only select resources based on predicted capacity gaps, lacking joint verification of access duration, migration conditions, fault domain distribution, and post-expansion capacity status. This easily results in problems such as delayed expansion, over-expansion, or expanded resources being unable to effectively carry data.

[0003] Therefore, how to provide a cloud storage capacity intelligent prediction and elastic expansion system based on machine learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a machine learning-based intelligent prediction and elastic expansion system for cloud storage capacity. This invention constructs a settlement chain corresponding to capacity formation, bearer migration, and space release, and uses an improved MICN model to perform multi-scale capacity prediction on unsettled chain segments, thereby improving the accuracy of capacity change prediction. At the same time, it locates the timing of capacity shortages and capacity gaps through settlement cross-sections, and combines candidate expansion branches to deduce and screen target expansion resources, thereby reducing expansion delays, erroneous expansions, and over-configuration of capacity.

[0005] The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to an embodiment of the present invention includes: The data consolidation module is used to acquire cloud storage capacity operation data and expansion constraint data, and to consolidate the data according to the storage object's ownership and status to form basic capacity data. The settlement tracking module is used to track the capacity formation, capacity migration, and space release processes caused by storage operations, and to connect the capacity change states in the order of their occurrence to form a settlement chain; The unresolved orchestration module is used to extract unclosed chain segments from the settlement chain that extend from the capacity formation position to the current advancement position, and to orchestrate them according to the chain position order to form an unresolved sequence; The capacity prediction module is used to input unfinished sequences into the improved MICN model. The improved MICN model changes the parallel multi-scale convolutional branches in the original MICN model to hierarchical equidistant convolutional chains, sets residual propagation paths between adjacent scales, and sets a unified reconstruction layer at the end of the hierarchical equidistant convolutional chains to output predicted unfinished sequences. The cross-section positioning module is used to connect the unsettled sequence with the predicted unsettled sequence in time and position, advance the corresponding settlement chain, and intercept the chain segments that stop advancing due to insufficient carrying capacity or delayed space release to form a settlement cross-section. The branch generation module is used to configure candidate expansion units for the interrupted chain position of the settlement section, fill the candidate expansion units into the corresponding bearing position, and continue the capacity migration and space release process to form candidate expansion branches. The expansion execution module is used to re-advance the settlement chain after the access candidate expansion branches, eliminate candidate expansion branches that have not eliminated settlement sections or have generated new settlement sections, screen the target expansion branches, and issue elastic expansion instructions to the cloud storage resource management interface according to the target expansion branches.

[0006] Optionally, the cloud storage capacity operation data includes storage operation data, storage resource status data, and capacity migration data, and the expansion constraint data includes resource access constraints, capacity security constraints, data migration constraints, and redundant bearing constraints.

[0007] Optionally, the formation of capacity basis data includes: Read the storage object ownership and operation time bit from the storage operation data, and group the storage resource status data and capacity migration data corresponding to the same storage object into the same object record; Unify the time base for recording all objects, arrange the capacity states according to the order of their occurrence, and record the time and location of each capacity state. The expansion constraint data is associated with the corresponding storage resources to form the basic capacity data.

[0008] Optionally, the settlement tracking module specifically includes: Extract the capacity formation status, bearer migration status, and space release status corresponding to the same storage object from the capacity basic data, group them according to the bearer change process, and arrange the capacity status of each group according to the time of occurrence. Set the capacity formation state as the head of the chain, connect the adjacent capacity states, and take over the migration out state with the corresponding migration in state; Mark the last state in each group of capacity states as the current advance position; when the last state is a space release state and the remaining actual occupied capacity of the corresponding storage object in the bearing position is 0, set it as the end of the chain; otherwise, mark it as an unclosed chain position to form a settlement chain.

[0009] Optionally, the unfinished orchestration module specifically includes: Traverse the settlement chain, select the settlement chain whose last capacity state is marked as an unclosed chain position, and extract from the capacity formation state to the unclosed chain position to form an unclosed chain segment; Each unclosed chain segment is arranged into a time series sample according to the unified state time position. The capacity value of each chain position is written into the corresponding state time position, and the capacity state category, carrying position, settlement chain ownership and unclosed chain position are bound to the corresponding time series sample. Collect samples from various time series to form unfinished sequences.

[0010] Optionally, the improved MICN model includes a multi-scale hybrid decomposition block, a trend cycle prediction block, and a seasonal prediction block; The seasonal prediction block includes a sequence embedding layer, multiple improved MIC layers and a prediction projection layer connected in sequence. Each improved MIC layer includes a hierarchical equidistant convolutional chain, a unified reconstruction layer and a feedforward residual layer. The hierarchical equidistant convolutional chain is formed by connecting multiple scale convolutional units in scale order, and a residual propagation path is set between adjacent scale convolutional units. The unfinished sequence is input into a multi-scale hybrid decomposition block for multi-scale decomposition to form trend periodic components and seasonal components. The trend cycle component is input into the trend cycle prediction block for mapping to form the trend prediction result. The seasonal components are encoded by a sequence embedding layer and then input into multiple improved MIC layers for hierarchical scale convolution and unified reconstruction. After being processed by a prediction projection layer, the seasonal prediction results are formed. The trend forecast results and seasonal forecast results are superimposed according to the corresponding state time position to form the predicted capacity sequence corresponding to each time series sample. The capacity state category, carrying location and settlement chain affiliation of each time series sample are retained, and they are aggregated to form the predicted unsettled sequence.

[0011] Optionally, the step of encoding the seasonal components via a sequence embedding layer and then inputting them into multiple improved MIC layers for hierarchical scale convolution and unified reconstruction, followed by processing through a prediction projection layer to form a seasonal prediction result, includes: The sequence embedding layer embeds and encodes the seasonal components to form a seasonal embedding sequence, and then passes the seasonal embedding sequence into the first improved MIC layer; The first-level scale convolutional unit performs scale compression and isometric convolution on the input state of the current improved MIC layer to form the first-level cumulative scale state; The next-scale convolutional unit compresses the input state of the current improved MIC layer according to the corresponding scale. The residual propagation path aligns the previous-level cumulative scale state to the compressed state and subtracts the aligned cumulative scale state from the compressed state to form the next-scale residual. The next scale convolutional unit performs equidistant convolution on the next scale residual and superimposes the convolution result with the aligned cumulative scale state to form the next scale cumulative scale state, and processes it step by step to the last scale convolutional unit. The unified reconstruction layer restores the final-level cumulative scale state to the sequence length corresponding to the current improved MIC layer input state; The feedforward residual layer merges the recovered state with the input state of the current improved MIC layer and passes the merge result to the next improved MIC layer. After processing multiple improved MIC layers, the prediction projection layer projects the output of the final improved MIC layer to form the seasonal prediction result.

[0012] Optionally, the section positioning module specifically comprises: According to the settlement chain affiliation and status, each predicted capacity sequence in the predicted unsettled sequence is connected to the unsettled chain position of the corresponding settlement chain to form a continuous advancement segment; Read the current carrying capacity of each carrying location under the capacity safety constraint, collect the capacity status of the same carrying location according to the state time position and advance it hour by hour; when advancing to the capacity formation state or the migration state, summarize the capacity value to be deducted at the corresponding state time position; when advancing to the space release state, add the corresponding capacity value to the current carrying capacity. When the value of the capacity to be deducted is greater than the current carrying capacity of the corresponding carrying position, the advancement is stopped and the corresponding interruption chain position is marked. Merge consecutive interrupted chain positions at the same bearing location, record the difference between the capacity to be deducted and the current bearing capacity as the capacity gap, and record the interrupted time position to form a settlement section.

[0013] Optionally, the branch generation module specifically comprises: Read the interruption chain bit, bearing position, capacity gap and interruption time bit of the settlement section, and read the available capacity increment, access position and access duration of available expansion resources from the cloud storage resource management interface; Based on the resource access constraints, expansion resources that can access the bearer location are screened out, and one or more expansion resources whose total available capacity increment is not less than the capacity gap and can complete access before the interruption time are combined into candidate expansion units. The candidate expansion unit is added to the bearing position corresponding to the interrupted chain position, the capacity status to be taken by the interrupted chain position is connected to the candidate expansion unit, and the capacity status is continued in the original settlement chain order. According to the data migration constraints and redundancy bearing constraints, the settlement chain after the continuation is checked, and the candidate expansion units that meet the constraints and their continuation status are arranged as candidate expansion branches.

[0014] Optionally, the expanded execution module specifically comprises: The candidate expansion branch is connected to the interrupted chain position of the corresponding settlement section, and when the connection of the candidate expansion unit is completed, the corresponding available capacity increment is added to the current bearable capacity of the corresponding bearer position. Based on the capacity status advancement method of the cross-section positioning module, the settlement chain after connecting the candidate expansion branch is re-deduced, and the interruption chain position is repositioned. Candidate expansion branches that have not been eliminated from the original settlement section or have generated new settlement sections are removed. The remaining candidate expansion branches are arranged in ascending order of available capacity increment. When the available capacity increment is the same, they are arranged in ascending order of access completion time. The first candidate expansion branch is determined as the target expansion branch. Generate elastic expansion instructions according to the target expansion branch and send them to the cloud storage resource management interface. Receive expansion execution receipts and update the current bearable capacity of the corresponding bearer location according to the actual available capacity increment.

[0015] The beneficial effects of this invention are: 1. This invention constructs a settlement chain with chain position connection by tracking the formation, migration and release process of cloud storage capacity, and extracts unsettled sequences from the settlement chain, so that the scattered storage operations, migration status and space release status can be uniformly associated according to the storage object ownership and status position, providing a continuous and traceable data foundation for capacity prediction and expansion judgment.

[0016] 2. This invention replaces the parallel multi-scale convolutional branches in the original MICN model with hierarchical equidistant convolutional chains, and sets up residual propagation paths between adjacent scales and a unified reconstruction layer at the end, so that the subsequent scale focuses on processing the capacity changes that have not been explained by the previous scale, reducing the repetitive expression and reconstruction time offset between different scales, and improving the accuracy and stability of capacity formation, capacity migration and spatial release prediction.

[0017] 3. This invention reduces capacity lag, erroneous expansion, and over-configuration of capacity by advancing the prediction of unfinished sequences to locate the interruption time and capacity gap corresponding to insufficient capacity, and generates candidate expansion branches based on resource access, data migration, and redundancy bearing constraints. After re-deduction and screening of target expansion branches, it ensures that expansion resources can actually access and carry data. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a structural diagram of the cloud storage capacity intelligent prediction and elastic expansion system based on machine learning proposed in this invention. Figure 2 This is a structural diagram of the improved MICN model for the cloud storage capacity intelligent prediction and elastic expansion system based on machine learning proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-2 A machine learning-based intelligent cloud storage capacity prediction and elastic scaling system includes: The data consolidation module is used to acquire cloud storage capacity operation data and expansion constraint data, and to consolidate the data according to the storage object's ownership and status to form basic capacity data. The settlement tracking module is used to track the capacity formation, capacity migration, and space release processes caused by storage operations, and to connect the capacity change states in the order of their occurrence to form a settlement chain; The unresolved orchestration module is used to extract unclosed chain segments from the settlement chain that extend from the capacity formation position to the current advancement position, and to orchestrate them according to the chain position order to form an unresolved sequence; The capacity prediction module is used to input unfinished sequences into the improved MICN model. The improved MICN model changes the parallel multi-scale convolutional branches in the original MICN model to hierarchical equidistant convolutional chains, sets residual propagation paths between adjacent scales, and sets a unified reconstruction layer at the end of the hierarchical equidistant convolutional chains to output predicted unfinished sequences. The cross-section positioning module is used to connect the unsettled sequence with the predicted unsettled sequence in time and position, advance the corresponding settlement chain, and intercept the chain segments that stop advancing due to insufficient carrying capacity or delayed space release to form a settlement cross-section. The branch generation module is used to configure candidate expansion units for the interrupted chain position of the settlement section, fill the candidate expansion units into the corresponding bearing position, and continue the capacity migration and space release process to form candidate expansion branches. The expansion execution module is used to re-advance the settlement chain after the access candidate expansion branches, eliminate candidate expansion branches that have not eliminated settlement sections or have generated new settlement sections, screen the target expansion branches, and issue elastic expansion instructions to the cloud storage resource management interface according to the target expansion branches.

[0021] In this embodiment, cloud storage capacity operation data includes storage operation data, storage resource status data, and capacity migration data, while expansion constraint data includes resource access constraints, capacity security constraints, data migration constraints, and redundant bearing constraints.

[0022] In practice, storage operation data is read from the object storage gateway logs and metadata service. Each storage operation record includes the storage object identifier, tenant identifier, storage space, operation type, operation start bit, operation completion bit, logical capacity value, and actual occupied capacity value. Operation types include write, overwrite, copy, migration, and delete. Write, overwrite, and copy operations correspond to the capacity formation process, migration operations are used to associate with the capacity migration process, and delete operations correspond to the space release process.

[0023] Storage resource status data is read from the cloud storage monitoring interface. Each resource status record includes the bearer location identifier, the resource pool to which it belongs, the fault domain, the storage media type, the nominal total capacity, the used capacity, the reserved capacity, the capacity being migrated in, the capacity being migrated out, and the resource health status. The bearer location is recorded according to three levels: resource pool, storage node, and object storage device. The resource health status is divided into normal, degraded, and inaccessible. Storage resources in an inaccessible state do not participate in the candidate expansion unit configuration.

[0024] Capacity migration data is read from the migration scheduling service. Each capacity migration record includes a migration task identifier, corresponding storage object, migration-out location, migration-in location, migration capacity value, task start time, task completion time, migration progress, and space release status of the migration-out location. When the migration progress reaches 100% and data deletion at the migration-out location is completed, the corresponding migration capacity value is included in the space release amount of the migration-out location. If the migration is not yet completed or data deletion at the migration-out location is not yet completed, the current available capacity of the migration-out location is not increased.

[0025] Resource access constraints record the allowed resource pools, regions, fault domains, storage media types, minimum available capacity increments per instance, and resource initialization durations for expansion resources. Capacity safety constraints set the upper limit of capacity usage for each storage resource to 85% of the nominal total capacity. The currently available capacity is obtained by subtracting used capacity, reserved capacity, and unused capacity from the upper limit. Capacity already written to and counted as used capacity during the migration process is not deducted again as unused capacity; a result less than 0 is taken as 0. The capacity value for capacity formation status uses the increase in actual occupied capacity at the corresponding carrying location. The logical capacity value is only used to verify the attribution relationship between storage operations and storage objects. Replica occupancy generated by replication operations and redundant occupancy generated by erasure coding are recorded according to the actual increase in occupied capacity in the storage resource status data, and are not repeatedly accumulated according to the logical capacity in the storage operations.

[0026] Data migration constraints record the storage media compatibility between the migration source and the migration target, the allowed migration direction, and the upper limit of migration bandwidth. Redundancy bearing constraints adopt a three-replica bearing method, with three replicas of the same storage object deployed on different storage nodes and distributed in at least two different fault domains; expansion resources that cannot meet the redundancy bearing requirements are not included in the candidate expansion resource range.

[0027] In this embodiment, the basic capacity data includes: Read the storage object ownership and operation time bit from the storage operation data, and group the storage resource status data and capacity migration data corresponding to the same storage object into the same object record; Unify the time base for recording all objects, arrange the capacity states according to the order of their occurrence, and record the time and location of each capacity state. The expansion constraint data is associated with the corresponding storage resources to form the basic capacity data.

[0028] In practice, the stored objects are specifically sharded groups within the object storage system. The system reads the original file object identifier, its bucket, and the operation time from the object storage gateway log. It then queries the shard group identifier and hosting device identifier corresponding to the original file object using the object placement metadata. Storage operation records, shard group capacity statistics records, and capacity migration records corresponding to the same shard group are grouped into the same object record. The shard group identifier serves as the primary index for the object record, and different replicas of the shard group record their hosting locations according to their respective object storage devices.

[0029] The time fields of all data sources are uniformly converted to millisecond-level timestamps, and the status time digits are divided into 5-minute intervals. For the same object storage device, among multiple resource status records falling into the same status time digit, the one with the latest collection time is retained; resource status records of different object storage devices are retained separately without cross-device overwriting; multiple progress records of the same capacity migration task are arranged according to the collection time, and the status records corresponding to task start, migration progress, migration completion, and space release are retained respectively.

[0030] After time correction is completed, the segment group capacity statistics records are grouped according to the segment group identifier and the carrier device identifier, and the actual occupied capacity corresponding to the adjacent state time position is read. When the actual occupied capacity of the later state time position is greater than that of the previous state time position, the increase is set as a positive capacity change; when the actual occupied capacity of the later state time position is less than that of the previous state time position, the decrease is set as a negative capacity change.

[0031] For a shard group that already has actual occupied capacity at the start of the observation, read its actual occupied capacity at the first state and generate a baseline capacity formation state. Set the baseline capacity formation state as the head of the chain corresponding to the carrying change process. The baseline capacity formation state is only used to carry out subsequent capacity migration and space release processes and is not counted as a duplicate write operation that occurs within the first state.

[0032] For positive capacity changes, perform task verification. If an inbound task exists and the inbound task identifier, inbound location, and capacity value correspond, record the positive capacity change as an inbound status; otherwise, record it as a capacity formation status. For negative capacity changes, perform task verification. If a migration task has been completed and the copy of the outbound location has been deleted, record it as a space release status after migration; if a deletion task has been completed, record it as a space release status after deletion. No space release status is generated if no actual reduction in occupied capacity is recorded.

[0033] The migration status is written into the settlement chain at the start of the migration task to indicate that capacity migration has started, but the migration status itself does not increase the current carrying capacity of the migration location; only when the actual occupied capacity of the migration location has decreased and a space release status has been formed, will the current carrying capacity of the corresponding carrying location be increased.

[0034] Expansion constraints are associated with storage resources based on resource pool identifier, storage node identifier, fault domain, and storage media type. Resource access constraints specify the allowed resource range and initialization duration; capacity security constraints specify the maximum capacity usage; data migration constraints specify the allowed migration direction and bandwidth limit; and redundancy constraints specify the number of replicas and fault domain distribution requirements. After the association is completed, the object record, capacity status, and corresponding expansion constraints are encapsulated into basic capacity data.

[0035] In this embodiment, the settlement tracking module specifically includes: Extract the capacity formation status, bearer migration status, and space release status corresponding to the same storage object from the capacity basic data, group them according to the bearer change process, and arrange the capacity status of each group according to the time of occurrence. Set the capacity formation state as the head of the chain, connect the adjacent capacity states, and take over the migration out state with the corresponding migration in state; Mark the last state in each group of capacity states as the current advance position; when the last state is a space release state and the remaining actual occupied capacity of the corresponding storage object in the bearing position is 0, set it as the end of the chain; otherwise, mark it as an unclosed chain position to form a settlement chain.

[0036] In the specific implementation process, the basic capacity data is first grouped by the storage object identifier, and then the capacity change process is divided according to the change of the carrying location. For capacity states that are always located in the same carrying location, they are grouped into the same capacity change process; for capacity states that have been migrated, the migration-out location, migration-in location, and migration task identifier are used as continuity verification fields, and the migration-out status and migration-in status are grouped into the same capacity change process.

[0037] The capacity states within each capacity change process are arranged in ascending order of their occurrence time. Capacity states at the same time point are arranged in the order of capacity formation, migration out, migration in, and space release; when there are multiple records for the same state category, they are arranged according to the generation time of the original record. The earliest occurring capacity formation state is set as the head of the chain, and the preceding and following chain positions are recorded for each capacity state.

[0038] For each migration state, the outgoing position, incoming position, and migration capacity value are read from the same migration task. When the outgoing position, incoming position, and migration capacity value match, the next bit of the outgoing state is set to point to the corresponding incoming state; if no corresponding incoming state is found, the outgoing state is kept as the current last bit and is not linked to the capacity state of other migration tasks.

[0039] After completing the chain connection, the last state in each carrying process is marked as the current advancement position. When the last state is a space release state, the remaining actual occupied capacity of the shard group on the corresponding object storage device is read; when the remaining actual occupied capacity is 0, the space release state is set as the chain tail; when the remaining actual occupied capacity is greater than 0, the space release state is retained as the current advancement position and marked as an unclosed chain position. When the last state is a capacity formation state, migration out state, or migration in state, it is also marked as an unclosed chain position. Finally, the chain head, each capacity state, chain connection relationship, current advancement position, chain tail, or unclosed chain position are collectively encapsulated into a settlement chain.

[0040] In this embodiment, the unfinished orchestration module specifically refers to: Traverse the settlement chain, select the settlement chain whose last capacity state is marked as an unclosed chain position, and extract from the capacity formation state to the unclosed chain position to form an unclosed chain segment; Each unclosed chain segment is arranged into a time series sample according to the unified state time position. The capacity value of each chain position is written into the corresponding state time position, and the capacity state category, carrying position, settlement chain ownership and unclosed chain position are bound to the corresponding time series sample. Collect samples from various time series to form unfinished sequences.

[0041] In the specific implementation process, all settlement chains are traversed, and the current progress position marker of each settlement chain is read. When the current progress position is marked as an unclosed chain position, the settlement chain is retained; when the current progress position has been set as the chain tail, it is not entered into the unclosed arrangement. For the retained settlement chain, starting from the capacity formation state corresponding to the chain head, the capacity state is read sequentially according to the chain position connection relationship until the unclosed chain position is reached, and the unclosed chain segment is intercepted.

[0042] The unified state time position is set to 5 minutes. The earliest occurrence time position in the unclosed chain segment is used as the sample starting time position. Each chain position is mapped to the corresponding state time position according to the state occurrence time. When the occurrence time of a capacity state falls between two state time positions, the later state time position is written. When there are multiple records of the same capacity state category at the same state time position and the same carrying location, their capacity values ​​are added together and written to the corresponding capacity state channel. Different capacity state categories are written to the capacity formation channel, migration out channel, migration in channel, and space release channel respectively, and the chain position index corresponding to the original record is retained.

[0043] For each unclosed chain segment, a timing sample is created, and the capacity value corresponding to each chain bit is written to the corresponding state timing bit. The capacity formation state is written with the newly occupied capacity value, the migration out and migration in states are written with the corresponding migration capacity value, and the space release state is written with the actual released capacity value; the state timing bit with no capacity is written with 0, and a vacancy mark is set.

[0044] In each time-series sample, the capacity status category, carrying location, settlement chain identifier, and unclosed chain position corresponding to each state are recorded simultaneously. The capacity status category is marked as capacity formation, migration out, migration in, or space release; the carrying location follows the resource pool, storage node, and object storage device positions in the capacity base data; the settlement chain identifier adopts the chain identifier of the corresponding settlement chain; and the unclosed chain position is recorded using the sequence number of that chain position in the settlement chain.

[0045] After completing the time-series processing of all unclosed chain segments, each time series sample is arranged according to the uniform state time position length, and each time series sample and its bound capacity state category, bearing location, settlement chain affiliation and unresolved chain position are gathered together to form an unresolved sequence.

[0046] In this embodiment, the improved MICN model includes a multi-scale hybrid decomposition block, a trend cycle prediction block, and a seasonal prediction block; The seasonal prediction block includes sequentially connected sequence embedding layers, multiple improved MIC layers, and a prediction projection layer. Each improved MIC layer includes a hierarchical equidistant convolutional chain, a unified reconstruction layer, and a feedforward residual layer. The hierarchical equidistant convolutional chain is formed by connecting multiple scale convolutional units in scale order, and residual propagation paths are set between adjacent scale convolutional units. The unfinished sequence is input into a multi-scale hybrid decomposition block for multi-scale decomposition to form trend periodic components and seasonal components. The trend cycle component is input into the trend cycle prediction block for mapping to form the trend prediction result. The seasonal components are encoded by a sequence embedding layer and then input into multiple improved MIC layers for hierarchical scale convolution and unified reconstruction. After being processed by a prediction projection layer, the seasonal prediction results are formed. The trend forecast results and seasonal forecast results are superimposed according to the corresponding state time position to form the predicted capacity sequence corresponding to each time series sample. The capacity state category, carrying location and settlement chain affiliation of each time series sample are retained, and they are aggregated to form the predicted unsettled sequence.

[0047] In this embodiment, the seasonal components are encoded by a sequence embedding layer and then input into multiple improved MIC layers for hierarchical scale convolution and unified reconstruction. After processing by a prediction projection layer, a seasonal prediction result is formed, including: The sequence embedding layer embeds and encodes the seasonal components to form a seasonal embedding sequence, and then passes the seasonal embedding sequence into the first improved MIC layer; The first-level scale convolutional unit performs scale compression and isometric convolution on the input state of the current improved MIC layer to form the first-level cumulative scale state; The next-scale convolutional unit compresses the input state of the current improved MIC layer according to the corresponding scale. The residual propagation path aligns the cumulative scale state of the previous stage to the compressed state and subtracts the aligned cumulative scale state from the compressed state to form the residual of the next scale. The next scale convolutional unit performs equidistant convolution on the next scale residual and superimposes the convolution result with the aligned cumulative scale state to form the next scale cumulative scale state, and processes it step by step to the last scale convolutional unit. The unified reconstruction layer restores the final-level cumulative scale state to the sequence length corresponding to the current improved MIC layer input state; The feedforward residual layer merges the recovered state with the input state of the current improved MIC layer and passes the merge result to the next improved MIC layer. After processing multiple improved MIC layers, the prediction projection layer projects the output of the final improved MIC layer to form the seasonal prediction result.

[0048] In the specific implementation, each MIC layer of the original MICN model has multiple parallel scale convolutional branches. Each scale convolutional branch performs scale compression, isometric convolution, and sequence length recovery on the same input state, and then the outputs of the scale convolutional branches are merged. This structure can extract capacity change features from different time scales, but the scale convolutional branches are independent of each other. Capacity changes extracted at a coarser scale may still be processed repeatedly at a finer scale, and each scale convolutional branch needs to recover the sequence length separately.

[0049] This implementation retains the original MICN model's multi-scale hybrid decomposition block, trend period prediction block, sequence embedding layer, feedforward residual layer, and prediction projection layer. It replaces the parallel-scale convolutional branches in each MIC layer with a hierarchical equidistant convolutional chain connected in order from coarse to fine scales. A residual propagation path is established between adjacent scale convolutional units, allowing the subsequent scale convolutional unit to subtract the previous stage's cumulative scale state from the current scale compression state, performing equidistant convolution only on capacity changes not yet explained by the previous scale. Simultaneously, the processing path for each scale convolutional unit to recover sequence length separately is eliminated; a unified reconstruction layer is placed at the end of the hierarchical equidistant convolutional chain to perform a single sequence length recovery on the final stage's cumulative scale state.

[0050] Read the capacity formation state, migration out state, migration in state, and space release state of each time series sample in the unfinished sequence, and write them into the capacity formation channel, migration out channel, migration in channel, and space release channel respectively according to the state time position; if the corresponding capacity state does not appear at the same state time position, write 0 in the corresponding channel.

[0051] Each channel records its capacity value and bearer location. The capacity formation channel is bound to the object storage device where capacity is actually increased; the migration out channel is bound to the migration out device in the migration task; the migration in channel is bound to the migration in the migration task; and the space release channel is bound to the object storage device where space is actually released. Bearer locations are encoded using a three-level system: resource pool identifier, storage node identifier, and object storage device identifier.

[0052] The capacity values ​​and corresponding carrier position codes of the four channels are arranged in state time position order, and then written into the state time position code, hour, day of the week, and intraday time position to form the input tensor of the improved MICN model. For each time series sample, the 288 consecutive state time positions before the unchained position are selected as the input interval, and the capacity change of the 72 state time positions after the unchained position is predicted. If there are fewer than 288 valid state time positions before the unchained position, placeholder time positions with both capacity value and carrier position code of 0 are added to the beginning of the input interval, and an invalid position marker is set; if there are more than 288 valid state time positions, the 288 state time positions closest to the unchained position are retained. The invalid position marker is used to mask the added positions, but does not mask 0 values ​​in the actual state time positions.

[0053] For the predicted state bit after the unchained bit, if there is an incomplete migration task, the migration-out and migration-in channels are activated according to the migration-out and migration-in positions recorded in the migration task record, and the migration-out and migration-in predicted amounts are limited to the incomplete capacity of the corresponding migration task, respectively; if there is no incomplete migration task, the migration-out and migration-in channels are set as invalid channels, the capacity value and the carrier position code of the invalid channel are both set to 0, and an invalid channel mark is written, and the corresponding prediction results are not written to the settlement chain.

[0054] When there are incomplete deletion tasks or relocation site cleanup tasks, the space release channel is activated, limiting the predicted space release amount to within the capacity value that has not yet been released for the corresponding task, and binding it to the release location corresponding to the task; when there are no incomplete deletion tasks or relocation site cleanup tasks, the space release channel is set to an invalid channel, and its prediction results are not added to the current carrying capacity. The capacity formation channel is bound to the current carrying location of the unlinked position.

[0055] Normalize the four capacity status channels separately. Read the upper limit of capacity usage for the corresponding bearing location under capacity safety constraints, divide each capacity value by the upper limit of capacity usage, and obtain the model input value. If the normalization result is greater than 1, retain the actual ratio. Multiply the model output by the same upper limit of capacity usage to restore the actual capacity value.

[0056] The improved MICN model includes a multi-scale hybrid decomposition block, a trend cycle prediction block, and a seasonal prediction block. The multi-scale hybrid decomposition block sets three moving average windows of 13, 37, and 73, respectively, and performs boundary-filled moving averages on the four capacity state channels of the unfinished sequence. The three moving average results are then averaged state-by-state to form the trend cycle component. The unfinished sequence is then subtracted from the trend cycle component state-by-state to form the seasonal component.

[0057] The trend cyclical component is used to characterize the slow changes resulting from continuous capacity formation, continuous space release, and the accumulation of capacity migration. The seasonal component is used to characterize the local fluctuations caused by batch storage operations, centralized migration tasks, timed deletion, and space release delays. The four capacity state channels, the number of state time points, and the time point order are maintained unchanged during the multi-scale hybrid decomposition process.

[0058] The trend cycle prediction block uses a time-dimensional linear mapping to map 288 input state time points to 72 predicted state time points. The four capacity state channels are each configured with output mapping parameters to predict the trend changes corresponding to capacity formation, migration out, migration in, and space release, thus obtaining the trend prediction results.

[0059] The seasonal prediction block consists of a sequence embedding layer, two improved MIC layers, and a prediction projection layer connected in sequence. The sequence embedding layer first concatenates 72 predicted placeholder time positions after 288 historical state time positions to form a seasonal input sequence of length 360. The four capacity values ​​of the predicted placeholder time positions are set to 0, and the positional encoding, positional encoding, and temporal features are written normally according to the state template corresponding to the predicted state time positions.

[0060] The sequence embedding layer projects four types of capacity values, carrier position encoding, state time position encoding and time features into a 64-dimensional feature space to form a seasonal embedding sequence, and then feeds the seasonal embedding sequence into the first improved MIC layer.

[0061] Each improved MIC layer consists of a hierarchical isometric convolutional chain comprising three-level scale convolutional units. The one-dimensional compressed convolutional kernel lengths and strides of these three-level scale convolutional units are set to 24, 12, and 6, respectively, with no padding in the temporal direction. The output feature channels of each compressed convolution are set to 64. For the current improved MIC layer input state of length 360, the three-level scale convolutional units form compressed states of lengths 15, 30, and 60, respectively.

[0062] For the first-level compressed state with a length of 15, 14 zero-value positions are padded to the left end of the time direction, and a one-dimensional isometric convolution with a kernel length of 15 and a stride of 1 is used to maintain the convolution result at a length of 15. For the second-level scale residual with a length of 30, 29 zero-value positions are padded to the left end of the time direction, and a one-dimensional isometric convolution with a kernel length of 30 and a stride of 1 is used to maintain the convolution result at a length of 30. For the third-level scale residual with a length of 60, 59 zero-value positions are padded to the left end of the time direction, and a one-dimensional isometric convolution with a kernel length of 60 and a stride of 1 is used to maintain the convolution result at a length of 60. The number of feature channels for all three scale states remains at 64.

[0063] The first-level scale convolutional unit compresses the input state of the current improved MIC layer through a one-dimensional convolution with a stride of 24, forming the first-level compressed state. The first-level scale convolutional unit performs isometric convolution on the first-level compressed state, so that the convolution kernel covers the entire time range of the first-level compressed state. Then, the isometric convolution result is residually superimposed with the first-level compressed state and layer normalized to form the first-level cumulative scale state.

[0064] The second-level scale convolutional unit compresses the input state of the current improved MIC layer through a one-dimensional convolution with a stride of 12, forming the second-level compressed state. Since the time length of the second-level compressed state is longer than that of the first-level cumulative scale state, the residual propagation path uses one-dimensional linear interpolation to align the first-level cumulative scale state to the time length of the second-level compressed state.

[0065] The aligned first-level cumulative scale state is subtracted from the second-level compressed state to form the second-level scale residual. The second-level scale convolutional unit performs isometric convolution only on the second-level scale residual, and superimposes the convolution result with the aligned first-level cumulative scale state to form the second-level cumulative scale state.

[0066] The third-level scale convolutional unit forms the third-level compressed state through a one-dimensional convolution with a stride of 6. It aligns the second-level cumulative scale state to the third-level compressed state by the time length required to do so. Then, it subtracts the aligned second-level cumulative scale state from the third-level compressed state to form the third-level scale residual. The third-level scale convolutional unit performs equidistant convolution on the third-level scale residual and superimposes the convolution result with the aligned second-level cumulative scale state to form the final-level cumulative scale state.

[0067] The residual formation and cumulative transfer relationship between adjacent scales is as follows: ; in, This indicates the compression state formed by the convolutional unit at the current scale. This indicates the cumulative scale state of the previous level. This indicates the time required to align the previous cumulative scale state to the current compressed state. Indicates the residual at the current scale. This indicates the isometric convolution performed by the convolutional unit at the current scale. This indicates the cumulative scale state at the current scale. The isotropic convolution results are activated before being stacked, and layer normalization is performed after stacking.

[0068] Through hierarchical computation, the first-level cumulative scale state represents the capacity variation explained by the coarser scale, the second-level scale residual represents the medium-range variation that the first-level scale failed to explain, and the third-level scale residual represents the short-term variation that neither of the first two scales could explain. The objects processed by the convolutional units at subsequent scales shrink step by step to the unexplained capacity residual.

[0069] After the final cumulative scale state is formed, it is input into the unified reconstruction layer. The unified reconstruction layer uses a one-dimensional transposed convolution with a kernel length and stride of 6 and no padding in the temporal direction to directly restore the final cumulative scale state with a length of 60 and 64 feature channels to a reconstructed state with a length of 360.

[0070] The feedforward residual layer first adds the unified reconstruction result to the input state of the current improved MIC layer position by position and performs layer normalization; then it completes feature dimension expansion and recovery through two one-dimensional convolutions with a kernel length of 1, and performs non-linear activation between the two convolutions; then it adds the feedforward processing result to the previous residual result to form the output state of the current improved MIC layer.

[0071] The output state of the first improved MIC layer is fed into the second improved MIC layer. The second improved MIC layer performs three-level scale compression, cumulative scale state alignment, scale residual subtraction, isometric convolution, cumulative stacking, and unified reconstruction in the same manner to obtain the output state of the final improved MIC layer.

[0072] The prediction projection layer projects the 64-dimensional features output from the final improved MIC layer onto four capacity state channels and extracts the last 72 state time positions to form capacity formation predictions, migration out predictions, migration in predictions, and space release predictions. The seasonal prediction results of the four channels are added to the trend prediction results of the corresponding channels for each state time position, and results less than 0 are corrected to 0 to form the predicted capacity sequence corresponding to each time series sample.

[0073] Based on the sample mapping table, each predicted capacity sequence is re-associated with its corresponding settlement chain affiliation, channel carrying position, and unresolved chain position. The predicted capacity formation and effective migration-in quantities serve as the capacity to be deducted when advancing to the corresponding carrying position; the predicted effective space release quantity is written to the space release chain position according to the predicted completion time of the corresponding deletion task or migration-out position cleanup task; the migration-out quantity is only used to indicate the migration progress of existing migration tasks. Invalid migration-out channels, migration-in channels, and space release channels are not written to the settlement chain.

[0074] Training samples are constructed by selecting the input interval and the subsequent 72 consecutive state time points from the historical settlement chain, where actual data recording has been completed. It is not required that the corresponding settlement chain has progressed to the space release state. For each historical chain segment, the four-channel capacity values ​​of the first 288 state time points are read as model input, and the four-channel capacity values ​​of the subsequent 72 state time points that have actually occurred are read as supervision labels. Historical chain segments whose supervision interval has not ended or have missing records are not included in the training samples.

[0075] The training samples are sorted according to the time position of the last state in the supervision interval. The first 70% of the samples with the earliest last state time are assigned to the training set, the middle 15% to the validation set, and the last 15% to the test set. All training windows generated by the same settlement chain are uniformly assigned to the same dataset according to the time position of the last supervision interval of that settlement chain.

[0076] The mean absolute error of the masks for the capacity formation, migration out, migration in, and space release channels is calculated separately. The losses of the four channels are then averaged, and this average result is used as the model training loss for backpropagation. The mask only hides the input window padding positions and missing supervision data positions; zero values ​​in the real state participate normally in the loss calculation. Backpropagation updates the parameters of the sequence embedding layer, the three-level scale convolutional unit, the residual propagation path, the unified reconstruction layer, the feedforward residual layer, and the prediction projection layer.

[0077] Training employed the AdamW optimizer with an initial learning rate of 0.001, a batch size of 64, and a maximum of 100 training epochs. Training was halted if the validation set loss did not decrease for 10 consecutive epochs, and the model parameters corresponding to the epoch with the lowest validation set loss were retained. During the testing phase, the mean absolute error of the four capacity state channels was calculated, and the accuracy of the predicted capacity sequence in reconnecting according to the settlement chain affiliation, carrying position, and unconnected chain position was verified.

[0078] With the above improvements, the original MICN model's method of processing the same complete input state in parallel across multiple scale branches has been replaced by a method that progressively passes the accumulated state from the coarse scale to the residual at the fine scale. This ensures that the convolutional unit at the next scale only supplements the capacity changes that were not explained by the previous scale, reducing redundant representations between different scales.

[0079] Meanwhile, the method of recovering complete sequences at each scale and then merging them has been replaced by a unified reconstruction of the final cumulative scale state, which reduces the temporal offset between multiple reconstruction results and enables the output capacity prediction results to be directly connected to the corresponding settlement chain according to the unified state temporal position.

[0080] In this embodiment, the cross-section positioning module specifically includes: According to the settlement chain affiliation and status, each predicted capacity sequence in the predicted unsettled sequence is connected to the unsettled chain position of the corresponding settlement chain to form a continuous advancement segment; Read the current carrying capacity of each carrying location under the capacity safety constraint, collect the capacity status of the same carrying location according to the state time position and advance it hour by hour; when advancing to the capacity formation state or the migration state, summarize the capacity value to be deducted at the corresponding state time position; when advancing to the space release state, add the corresponding capacity value to the current carrying capacity. When the value of the capacity to be deducted is greater than the current carrying capacity of the corresponding carrying position, the advancement is stopped and the corresponding interruption chain position is marked. Merge consecutive interrupted chain positions at the same bearing location, record the difference between the capacity to be deducted and the current bearing capacity as the capacity gap, and record the interrupted time position to form a settlement section.

[0081] In the specific implementation process, the settlement chain affiliation, carrying position of each capacity status channel, effective channel marker, and prediction status time position corresponding to each predicted capacity sequence in the predicted unresolved sequence are read. The unresolved chain position of the corresponding settlement chain is found according to the settlement chain affiliation, and the predicted capacity status corresponding to the effective channel is continuating after the unresolved chain position according to the prediction status time position. Capacity formation predictions are written to the capacity formation chain position, migration in predictions are written to the migration in chain position, migration out predictions are written to the migration out chain position, and space release predictions are written to the space release chain position. The unresolved chain position and the subsequent predicted chain position together constitute a continuous advancement segment.

[0082] A 5-minute status time point is used as the unit of advancement. For each carrying location, the most recent storage resource status record before the current advancement time is read, and the current carrying capacity of that carrying location is obtained from it. Object storage devices in the cloud storage platform are determined as the minimum capacity metering unit, and the current carrying capacity is maintained on each object storage device; resource pools and storage nodes are only used for resource grouping, fault domain verification, and expansion resource screening, and capacity deduction is not performed repeatedly.

[0083] The capacity status in the continuous advancement segment is read from first to last along the state time position. The actual capacity status retains the original millisecond-level occurrence time and is advanced sequentially according to the occurrence time; for the predicted state that lacks the occurrence time within the state time position, the predicted capacity formation and the predicted effective migration within the same bearing location and the same state time position are added together to obtain the capacity to be deducted, and the predicted effective space release is added together to obtain the capacity to be added, and the process is advanced in the order of the capacity to be deducted before the capacity to be added.

[0084] When advancing the actual capacity status, if the space release status occurs earlier than the capacity formation status or the relocation status, the corresponding released capacity will be added to the current carrying capacity first; if the space release status occurs later than the capacity formation status or the relocation status, it will not participate in the capacity verification of the aforementioned capacity status.

[0085] When the capacity to be deducted is not greater than the current carrying capacity, the capacity to be deducted is deducted from the current carrying capacity, and the corresponding capacity forms a chain position and a migration chain position marked as advanced; when there is still effective space to release the predicted amount in the current state position, the capacity to be added is added to the current carrying capacity after deduction, and the updated capacity is passed to the next state position.

[0086] When the capacity to be deducted exceeds the current carrying capacity, partial deduction will not be performed. The corresponding capacity forming chain position and migration chain position will be marked as interrupted chain positions, and the portion of the capacity to be deducted that exceeds the current carrying capacity will be recorded as the current capacity gap. The corresponding settlement chain will stop advancing from the interrupted chain position, and the capacity status after the interrupted chain position will not participate in this round of capacity update.

[0087] Other settlement chains that have not been interrupted continue to advance according to their status. Space release states that have not yet been executed after the interruption chain position must not be prematurely added to the current carrying capacity to prevent the use of subsequent release capacity before the previous capacity is formed or the migration process is completed.

[0088] Check all interrupted chain positions according to their bearing positions. Interrupted chain positions belonging to the same bearing position and state, and without any intermediate completed advance states, are merged into the same interrupted chain segment; if there are interrupted chain positions with completed advance states, they are retained as different interrupted chain segments.

[0089] When an interrupted chain segment contains multiple state positions, the current carrying capacity prior to the starting state position of the interrupted chain segment is used as the initial capacity. All capacity states involved in the interrupted chain segment are rearranged according to the state positions and the order of occurrence within each state position. For each state, the effective space release amount is added to the capacity balance, while the capacity formation and migration amounts are deducted from the capacity balance. When the capacity balance is negative, its absolute value is recorded. The largest absolute negative value occurring within the entire interrupted chain segment is used as the capacity gap, thus the capacity gap represents the minimum additional carrying capacity required to propel all capacity states within the interrupted chain segment through continuously.

[0090] Finally, the carrying position, interruption start time, interruption end time, settlement chain affiliation, interruption chain position, and maximum capacity gap corresponding to the interrupted chain segment are recorded to form a settlement section. Insufficient carrying capacity is manifested as the capacity to be deducted exceeding the current carrying capacity; delayed space release is manifested as the released capacity not being added to the current carrying capacity at the corresponding state time position, thus causing insufficient capacity. Both situations are located through the same capacity advancement process.

[0091] In this embodiment, the branch generation module specifically includes: Read the interruption chain bit, bearing position, capacity gap and interruption time bit of the settlement section, and read the available capacity increment, access position and access duration of available expansion resources from the cloud storage resource management interface; Based on resource access constraints, expansion resources that can be accessed at the bearing location are screened out, and one or more expansion resources whose total available capacity increment is not less than the capacity gap and can be accessed before the interruption time are combined into candidate expansion units. The candidate expansion unit is added to the bearing position corresponding to the interrupted chain position, the capacity status to be taken by the interrupted chain position is connected to the candidate expansion unit, and the capacity status is continued in the original settlement chain order. According to the data migration constraints and redundancy bearing constraints, the settlement chain after the continuation is checked, and the candidate expansion units that meet the constraints and their continuation status are arranged as candidate expansion branches.

[0092] In the specific implementation process, for each settlement section, the recorded bearing location, interruption time, capacity gap, interruption chain position, and settlement chain affiliation are read, and the cloud storage resource management interface is queried for expansion resources in an accessible state. The interface returns the resource identifier, resource pool to which it belongs, storage medium type, fault domain, available capacity increment, allowed access location, estimated initialization time, and resource health status for each expansion resource. Expansion resources with an abnormal health status or an available capacity increment of 0 are not processed.

[0093] The allowed access locations of expansion resources are verified based on the carrying location of the settlement section. If the resource pool, storage medium type, and fault domain of the expansion resource meet the resource access constraints, and the expected access completion time, calculated from the current time, is no later than the interruption time of the settlement section, the expansion resource is retained. The expected access completion time is obtained by adding the current time to the expected initialization duration; expansion resources that cannot complete access before the interruption occurs are discarded.

[0094] Reserved expansion resources are arranged in ascending order of available capacity increment. Each expansion resource is checked sequentially; if the available capacity increment of a single expansion resource is not less than the capacity shortfall, it is recorded as a candidate expansion unit.

[0095] While retaining single-resource candidate expansion units, the expansion resources are combined without duplication. First, one expansion resource is selected as the starting resource for each combination. Then, other expansion resources not yet included in the current combination are added sequentially. When the total available capacity increment of the combined resources first reaches the capacity gap, the current combination is recorded as a candidate expansion unit, and adding more expansion resources to the current combination is stopped. This process is repeated after changing the starting resource and subsequent added resources to form multi-resource candidate expansion units.

[0096] When a candidate expansion unit contains multiple expansion resources, the latest expected access completion time is used as the access completion time of the candidate expansion unit; candidate expansion units with a total available capacity increment not less than the capacity gap and an access completion time not later than the interruption time are entered into subsequent constraint verification.

[0097] Read the capacity status category of the interrupted chain bit. When the interrupted chain bit is in the capacity formation state, add the candidate expansion unit to the selectable bearer range of the corresponding resource pool, and re-determine the bearer device for the capacity to be written according to the object placement rules; when the interrupted chain bit is in the migration-in state, set the candidate expansion unit as the new migration-in bearer location, and retain the original migration-out location and migration capacity value. When there is a space release state after the interrupted chain bit, maintain its original chain bit succession relationship, release time bit, and release capacity value unchanged. After the candidate expansion unit is connected, continue the capacity status that has not yet been promoted according to the original settlement chain order.

[0098] Then, the data migration constraints are checked. If the storage media between the migration location and the candidate expansion unit is incompatible, the migration direction is not allowed, or the migration completion time calculated according to the available migration bandwidth is later than the advance time required by the corresponding capacity status, the continuation relationship corresponding to the candidate expansion unit is deleted.

[0099] Continue to verify the redundancy constraints. Compare the storage nodes and fault domains after the candidate expansion unit is connected with the storage nodes and fault domains where the existing replicas of the same storage object are located; if the candidate expansion unit causes the replicas of the same storage object to fall into the same storage node, or fails to meet the fault domain dispersion requirement, the corresponding candidate expansion unit is removed.

[0100] After constraint verification, the resource composition, access completion time, total available capacity increment, interrupted chain position, and capacity status after reconnection of the candidate expansion units are recorded together and connected according to the chain position direction of the original settlement chain to form candidate expansion branches. When no candidate expansion unit with a total available capacity increment not less than the capacity gap and capable of completing access before the interruption time position is formed, the corresponding settlement section and its interrupted chain position are retained, a resource insufficiency flag is generated, and a capacity risk alarm is sent to the cloud storage resource management interface; no candidate expansion branches are generated in this round, and no elastic expansion instructions are issued. Each candidate expansion branch corresponds to only one settlement section and one expansion resource combination, which are used by the expansion execution module to access the settlement chain and re-determine.

[0101] In this embodiment, the expanded execution module specifically refers to: The candidate expansion branch is connected to the interrupted chain position of the corresponding settlement section, and when the connection of the candidate expansion unit is completed, the corresponding available capacity increment is added to the current bearable capacity of the corresponding bearer position. Based on the capacity status advancement method of the cross-section positioning module, the settlement chain after connecting the candidate expansion branch is re-deduced, and the interruption chain position is repositioned. Candidate expansion branches that have not been eliminated from the original settlement section or have generated new settlement sections are removed. The remaining candidate expansion branches are arranged in ascending order of available capacity increment. When the available capacity increment is the same, they are arranged in ascending order of access completion time. The first candidate expansion branch is determined as the target expansion branch. Generate elastic expansion instructions according to the target expansion branch and send them to the cloud storage resource management interface. Receive expansion execution receipts and update the current bearable capacity of the corresponding bearer location according to the actual available capacity increment.

[0102] In the specific implementation process, candidate expansion branches are read sequentially. Only one candidate expansion branch is connected in each simulation, and the current carrying capacity, settlement chain progress status and interrupted chain position of the corresponding settlement section are copied to form an independent simulation copy, so as to avoid the overlap of capacity increments and chain position status between different candidate expansion branches.

[0103] According to the access completion time bit of the candidate expansion branch record, the available capacity increment of each expansion resource in the candidate expansion unit is written to the current bearable capacity of its corresponding object storage device. When the interruption chain bit is in the capacity formation state, the bearer device is determined from the object storage device corresponding to the candidate expansion unit according to the object placement rules. The capacity formation chain bit after the bearer device is re-determined is connected to the determined object storage device, and the corresponding capacity value is deducted from the current bearable capacity of the object storage device. When the interruption chain bit is in the migration state, a new migration device is determined from the object storage device corresponding to the candidate expansion unit according to the data migration constraints, and the migration capacity value is deducted from the current bearable capacity of the migration device. The current bearable capacity of the object storage device corresponding to the original settlement section remains unchanged.

[0104] When resuming the process, the capacity status progression rules in the cross-section positioning module are followed. The actual capacity status is progressed sequentially according to the original millisecond-level occurrence time; for predicted statuses with missing occurrence times within a given location, the progression is made in the order of capacity formation status and migration status preceding space release status. Only space release amounts that occurred earlier than the status to be deducted and are already in a valid state can be added to the current carrying capacity used in this capacity verification.

[0105] If all interrupted chain positions in the original settlement section can be advanced, and their capacity status can continue to advance according to the original chain position order, the candidate expansion branch is marked as having eliminated the original settlement section. If at least one interrupted chain position in the original settlement section still cannot be advanced, the candidate expansion branch is marked as having not eliminated the original settlement section, and the simulation of the candidate expansion branch is stopped.

[0106] After the original settlement section is eliminated from the candidate expansion branch, other settlement chains related to the bearer location continue to be promoted. When a capacity status that could previously be promoted normally becomes insufficient due to the access of candidate expansion units, capacity migration, or redundant bearer adjustments, and a new interruption chain position is formed, the corresponding interruption chain segment is recorded as a new settlement section. Candidate expansion branches that generate new settlement sections will not enter the target expansion branch screening.

[0107] After completing the simulation of a candidate expansion branch, save its original settlement section elimination results, new settlement section detection results, available capacity increment and access completion time. Then restore the current carrying capacity and settlement chain status before the simulation and continue to perform the same simulation for the next candidate expansion branch.

[0108] Candidate expansion branches that have not eliminated the original settlement interface or have created a new settlement interface will be deleted. For the remaining candidate expansion branches, they will first be arranged in ascending order of total available capacity increment to reduce the expansion capacity exceeding the actual capacity gap; if the total available capacity increment is the same, they will be arranged in ascending order of access completion time to prioritize candidate expansion branches that can resume settlement chain progress earlier. The candidate expansion branch ranked first will be determined as the target expansion branch.

[0109] After identifying the target expansion branch, a target expansion resource reservation request is submitted to the cloud storage resource management interface. The cloud storage resource management interface returns a resource reservation identifier for successfully reserved expansion resources and removes the expansion resources from the available expansion resources in other settlement sections during the reservation period. When the target expansion branch contains multiple expansion resources, an elastic expansion instruction is generated only after all expansion resources are successfully reserved; if any expansion resource reservation fails, the obtained resource reservation identifier is released, the current target expansion branch is deleted, and the branch ranked first is reselected from the remaining candidate expansion branches.

[0110] Elastic expansion instructions are generated based on the target expansion branch. Each elastic expansion instruction includes at least the target expansion resource identifier, target resource pool, access location, available capacity increment, required completion time, and corresponding settlement segment identifier, and is sent to the resource orchestration service via the cloud storage resource management interface. When the target expansion branch contains multiple expansion resources, access sub-instructions are generated for each expansion resource, and they are associated using the same settlement segment identifier.

[0111] After the cloud storage resource management interface returns a successful instruction acceptance, the current available capacity of the corresponding bearer location is not immediately modified. The system continues to read the resource mounting status, resource pool addition status, resource health status, and newly added available capacity returned by the cloud storage monitoring interface. Once the expanded resource is successfully mounted, added to the target resource pool, and its health status is normal, and the cloud storage monitoring interface returns the actual available capacity increment, the actual available capacity increment is added to the current available capacity of the actual access target storage device. Based on the bearer adjustment relationship recorded in the target expansion branch, the bearer location of the corresponding capacity forming link or migration link is updated to the actual access target storage device. The resource reservation identifier is changed to an occupied state, and the candidate access status in the target expansion branch is replaced with the actual access status.

[0112] When the actual available capacity increment is less than the planned available capacity increment, the corresponding settlement chain is restarted according to the actual available capacity increment. If the original settlement gap is still not eliminated after restarting, the remaining capacity gap is recorded, and the process proceeds to the next round of candidate expansion branch generation.

[0113] If the expanded resource is not mounted, not added to the target resource pool, is not in a normal health state, or the cloud storage monitoring interface does not return the newly added available capacity before the required completion time, the expansion will be marked as a failure, the corresponding resource reservation identifier will be released, the failed resource will be removed from the available expansion resources in this round, the current available capacity will not be modified, and the original settlement section will be retained. Only after the cloud storage monitoring interface confirms that the resource has reached a normal health state and returns the actual available capacity increment will the newly added capacity be written to the current available capacity of the actual access object storage device.

[0114] Example 1: To verify the feasibility of this invention in practice, it was applied to the object storage center of a large e-commerce platform. This storage center stores product images, product videos, live stream replays, user review attachments, and business backup data. During operation, it experiences concentrated writes during promotional periods, batch backups at night, cross-node capacity migration, and delayed release of deleted files. The original capacity management method triggered expansion when the object storage device's capacity utilization reached 80%, increasing storage resources in fixed increments. This method was prone to premature expansion just before data was to be released, or expansion delays when batch writes and capacity migration occurred simultaneously.

[0115] This embodiment selects one resource pool within the object storage center for testing. This resource pool includes 20 storage nodes and 160 object storage devices, each with a nominal capacity of 64TB. Object data is carried using a three-replica configuration, with the three replicas of the same shard group located on different storage nodes and distributed across at least two fault domains. Capacity safety constraints set the maximum capacity usage of each object storage device to 85% of its nominal capacity.

[0116] The cloud storage monitoring interface collects the used capacity, reserved capacity, pending migration capacity, and health status of object storage devices every 5 minutes; the object storage gateway log records write, overwrite, copy, and delete operations; and the migration scheduling service records the migration location, migration location, migration capacity, and space release status. It continuously collects 60 days of operational data and generates capacity formation status, migration status, migration status, and space release status according to the shard group identifier and object storage device identifier.

[0117] For each time series sample, the first 288 states are used to predict the next 72 states, corresponding to using the capacity states from the first 24 hours to predict the capacity changes over the next 6 hours. The improved MICN model uses two improved MIC layers with a feature dimension of 64, and the compression scales of the three-level scale convolutional units are set to 24, 12, and 6 respectively. The model uses the AdamW optimizer with an initial learning rate of 0.001 and a training batch size of 64. Training stops when the loss on the validation set does not decrease after 10 consecutive rounds.

[0118] After the model outputs the predicted unfinished sequence, it advances the capacity formation, effective migration, and effective space release of the object storage device hourly. When the capacity to be deducted exceeds the current carrying capacity, the interruption time and capacity gap are recorded to form a settlement section.

[0119] During a test prior to a promotional event, the current capacity of the object storage device OSD-086 was 21.7TB. The model predicted a capacity build-up of 31.8TB and a valid migration-in of 12.6TB over the next 6 hours, with 8.4TB of space available for release corresponding to incomplete deletion tasks. Following the status timeline, the object storage device experienced a capacity shortfall of 14.3TB at 17:40.

[0120] The cloud storage resource management interface returned three accessible expansion resources with available capacity increments of 16TB, 24TB, and 32TB, respectively. The 16TB expansion resource is located in the same fault domain as the existing replicas in the target shard group, thus failing to meet redundancy constraints. Both the 24TB and 32TB expansion resources could be accessed before the interruption occurred. After re-analyzing the two candidate expansion branches by connecting them to the settlement chain, both eliminated the original settlement gaps without creating new ones. Therefore, the 24TB expansion resource with the smaller available capacity increment was selected as the target expansion branch.

[0121] After the cloud storage monitoring interface confirms that the target expansion resource has been successfully mounted, added to the resource pool, and is in normal health status, it records the actual available capacity increment as 23.5TB and adjusts the corresponding capacity into chain positions and migration chain positions to the newly connected device according to the object placement rules. After restarting the process, the original 14.3TB capacity gap was eliminated.

[0122] Historical data from seven consecutive days of operation were reviewed, and a total of 42 sections with insufficient capacity risk were identified. The traditional fixed threshold expansion method, the original MICN model prediction method, and the system of this invention were compared, and the results are shown in Table 1.

[0123] Table 1 Comparison of Cloud Storage Capacity Prediction and Expansion Results

[0124] Among them, the 6-hour prediction error is the average of the absolute value of the difference between the predicted capacity change and the actual capacity change when the actual capacity change is not zero, divided by the absolute value of the actual capacity change; the state when the actual capacity change is zero is not included in the calculation of this indicator; the cross-section identification recall rate is the ratio between the number of correctly identified capacity insufficiency risk cross-sections and the total number of actual capacity insufficiency risk cross-sections; the number of false expansions is the number of times expansion orders were issued even though no capacity insufficiency risk actually occurred.

[0125] As shown in Table 1, the fixed threshold expansion method cannot distinguish between capacity formation, capacity migration, and space release processes, resulting in a prediction error of 12.36% and 12 instances of erroneous expansion. The original MICN model can extract multi-scale capacity change features, but each scale branch is processed and reconstructed independently, resulting in a prediction error of 7.48%. This invention reduces the prediction error to 4.12% by passing unexplained scale residuals through hierarchical equidistant convolutional chains and recovering the prediction sequence through a unified reconstruction layer; simultaneously, by combining settlement cross-section and expansion branch derivation, the number of erroneous expansions is reduced to 2.

[0126] This embodiment demonstrates that the present invention can use the operational data of e-commerce platform object storage to predict capacity changes, locate the interruption points and capacity gaps corresponding to insufficient capacity, and select expansion resources under the conditions of meeting resource access, data migration and redundancy bearing constraints, thereby reducing capacity prediction deviation and unnecessary expansion operations.

[0127] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A cloud storage capacity intelligent prediction and elastic expansion system based on machine learning, characterized in that, include: The data consolidation module is used to acquire cloud storage capacity operation data and expansion constraint data, and to consolidate the data according to the storage object's ownership and status to form basic capacity data. The settlement tracking module is used to track the capacity formation, capacity migration, and space release processes caused by storage operations, and to connect the capacity change states in the order of their occurrence to form a settlement chain; The unresolved orchestration module is used to extract unclosed chain segments from the settlement chain that extend from the capacity formation position to the current advancement position, and to orchestrate them according to the chain position order to form an unresolved sequence; The capacity prediction module is used to input unfinished sequences into the improved MICN model. The improved MICN model changes the parallel multi-scale convolutional branches in the original MICN model to hierarchical equidistant convolutional chains, sets residual propagation paths between adjacent scales, and sets a unified reconstruction layer at the end of the hierarchical equidistant convolutional chains to output predicted unfinished sequences. The cross-section positioning module is used to connect the unsettled sequence with the predicted unsettled sequence in time and position, advance the corresponding settlement chain, and intercept the chain segments that stop advancing due to insufficient carrying capacity or delayed space release to form a settlement cross-section. The branch generation module is used to configure candidate expansion units for the interrupted chain position of the settlement section, fill the candidate expansion units into the corresponding bearing position, and continue the capacity migration and space release process to form candidate expansion branches. The expansion execution module is used to re-advance the settlement chain after the access candidate expansion branches, eliminate candidate expansion branches that have not eliminated settlement sections or have generated new settlement sections, screen the target expansion branches, and issue elastic expansion instructions to the cloud storage resource management interface according to the target expansion branches.

2. The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to claim 1, characterized in that, The cloud storage capacity operation data includes storage operation data, storage resource status data, and capacity migration data. The expansion constraint data includes resource access constraints, capacity security constraints, data migration constraints, and redundancy bearing constraints.

3. The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to claim 1, characterized in that, The basic data for forming capacity includes: Read the storage object ownership and operation time bit from the storage operation data, and group the storage resource status data and capacity migration data corresponding to the same storage object into the same object record; Unify the time base for recording all objects, arrange the capacity states according to the order of their occurrence, and record the time and location of each capacity state. The expansion constraint data is associated with the corresponding storage resources to form the basic capacity data.

4. The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to claim 1, characterized in that, The settlement tracking module specifically includes: Extract the capacity formation status, bearer migration status, and space release status corresponding to the same storage object from the capacity basic data, group them according to the bearer change process, and arrange the capacity status of each group according to the time of occurrence. Set the capacity formation state as the head of the chain, connect the adjacent capacity states, and take over the migration out state with the corresponding migration in state; Mark the last state in each group of capacity states as the current advance position; when the last state is a space release state and the remaining actual occupied capacity of the corresponding storage object in the bearing position is 0, set it as the end of the chain; otherwise, mark it as an unclosed chain position to form a settlement chain.

5. The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to claim 1, characterized in that, The unfinished orchestration module specifically includes: Traverse the settlement chain, select the settlement chain whose last capacity state is marked as an unclosed chain position, and extract from the capacity formation state to the unclosed chain position to form an unclosed chain segment; Each unclosed chain segment is arranged into a time series sample according to the unified state time position. The capacity value of each chain position is written into the corresponding state time position, and the capacity state category, carrying position, settlement chain ownership and unclosed chain position are bound to the corresponding time series sample. Collect samples from various time series to form unfinished sequences.

6. The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to claim 1, characterized in that, The improved MICN model includes a multi-scale hybrid decomposition block, a trend cycle prediction block, and a seasonal prediction block; The seasonal prediction block includes a sequence embedding layer, multiple improved MIC layers and a prediction projection layer connected in sequence. Each improved MIC layer includes a hierarchical equidistant convolutional chain, a unified reconstruction layer and a feedforward residual layer. The hierarchical equidistant convolutional chain is formed by connecting multiple scale convolutional units in scale order, and a residual propagation path is set between adjacent scale convolutional units. The unfinished sequence is input into a multi-scale hybrid decomposition block for multi-scale decomposition to form trend periodic components and seasonal components. The trend cycle component is input into the trend cycle prediction block for mapping to form the trend prediction result. The seasonal components are encoded by a sequence embedding layer and then input into multiple improved MIC layers for hierarchical scale convolution and unified reconstruction. After being processed by a prediction projection layer, the seasonal prediction results are formed. The trend forecast results and seasonal forecast results are superimposed according to the corresponding state time position to form the predicted capacity sequence corresponding to each time series sample. The capacity state category, carrying location and settlement chain affiliation of each time series sample are retained, and they are aggregated to form the predicted unsettled sequence.

7. The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to claim 6, characterized in that, The process involves encoding the seasonal components via a sequence embedding layer, inputting them into multiple improved MIC layers for hierarchical scale convolution and unified reconstruction, and then processing them through a prediction projection layer to form a seasonal prediction result, including: The sequence embedding layer embeds and encodes the seasonal components to form a seasonal embedding sequence, and then passes the seasonal embedding sequence into the first improved MIC layer; The first-level scale convolutional unit performs scale compression and isometric convolution on the input state of the current improved MIC layer to form the first-level cumulative scale state; The next-scale convolutional unit compresses the input state of the current improved MIC layer according to the corresponding scale. The residual propagation path aligns the previous-level cumulative scale state to the compressed state and subtracts the aligned cumulative scale state from the compressed state to form the next-scale residual. The next scale convolutional unit performs equidistant convolution on the next scale residual and superimposes the convolution result with the aligned cumulative scale state to form the next scale cumulative scale state, and processes it step by step to the last scale convolutional unit. The unified reconstruction layer restores the final-level cumulative scale state to the sequence length corresponding to the current improved MIC layer input state; The feedforward residual layer merges the recovered state with the input state of the current improved MIC layer and passes the merge result to the next improved MIC layer. After processing multiple improved MIC layers, the prediction projection layer projects the output of the final improved MIC layer to form the seasonal prediction result.

8. The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to claim 1, characterized in that, The section positioning module is specifically: According to the settlement chain affiliation and status, each predicted capacity sequence in the predicted unsettled sequence is connected to the unsettled chain position of the corresponding settlement chain to form a continuous advancement segment; Read the current carrying capacity of each carrying location under the capacity safety constraint, collect the capacity status of the same carrying location according to the state time position and advance it hour by hour; when advancing to the capacity formation state or the migration state, summarize the capacity value to be deducted at the corresponding state time position; when advancing to the space release state, add the corresponding capacity value to the current carrying capacity. When the value of the capacity to be deducted is greater than the current carrying capacity of the corresponding carrying position, the advancement is stopped and the corresponding interruption chain position is marked. Merge consecutive interrupted chain positions at the same bearing location, record the difference between the capacity to be deducted and the current bearing capacity as the capacity gap, and record the interrupted time position to form a settlement section.

9. The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to claim 1, characterized in that, The branch generation module is specifically as follows: Read the interruption chain bit, bearing position, capacity gap and interruption time bit of the settlement section, and read the available capacity increment, access position and access duration of available expansion resources from the cloud storage resource management interface; Based on the resource access constraints, expansion resources that can access the bearer location are screened out, and one or more expansion resources whose total available capacity increment is not less than the capacity gap and can complete access before the interruption time are combined into candidate expansion units. The candidate expansion unit is added to the bearing position corresponding to the interrupted chain position, the capacity status to be taken by the interrupted chain position is connected to the candidate expansion unit, and the capacity status is continued in the original settlement chain order. According to the data migration constraints and redundancy bearing constraints, the settlement chain after the connection is checked and the candidate expansion units that meet the constraints and their connection status are arranged as candidate expansion branches.

10. The cloud storage capacity intelligent prediction and elastic expansion system based on machine learning according to claim 1, characterized in that, The expanded execution module is specifically: The candidate expansion branch is connected to the interrupted chain position of the corresponding settlement section, and when the connection of the candidate expansion unit is completed, the corresponding available capacity increment is added to the current bearable capacity of the corresponding bearer position. Based on the capacity status advancement method of the cross-section positioning module, the settlement chain after connecting the candidate expansion branch is re-deduced, and the interruption chain position is repositioned. Candidate expansion branches that have not been eliminated from the original settlement section or have generated new settlement sections are removed. The remaining candidate expansion branches are arranged in ascending order of available capacity increment. When the available capacity increment is the same, they are arranged in ascending order of access completion time. The first candidate expansion branch is determined as the target expansion branch. Generate elastic expansion instructions according to the target expansion branch and send them to the cloud storage resource management interface. Receive expansion execution receipts and update the current bearable capacity of the corresponding bearer location according to the actual available capacity increment.

Images