Resource scheduling method and device, storage medium and electronic equipment

Abstract

The invention discloses a resource scheduling method and device, a storage medium and electronic equipment, and relates to the technical field of cloud storage, and the method comprises the steps: firstly obtaining historical request information of a user for a cloud storage service; on the basis of historical request information, tide prediction confidence is generated; based on the tide prediction confidence, judging whether the current service load state meets a triggering condition of a target fluctuation scheduling period or not; if the current service load state meets the triggering condition of the target fluctuation scheduling period, respectively calculating a resource scheduling weight and a migration performance factor corresponding to the storage node; and triggering a corresponding resource scheduling operation based on the resource scheduling weight and the migration performance factor. Compared with the prior art, the method has the advantages that a multi-stage dynamic scheduling mechanism is constructed based on the tide prediction confidence coefficient, and accurate triggering of scheduling operations such as data migration and message queue optimization is realized in combination with the resource scheduling weight and the migration performance factor, so that the system resource utilization rate is effectively improved, and the fluctuation of user access delay is reduced.

Description

A resource scheduling method, apparatus, storage medium, and electronic device Technical Field

[0001] This application relates to the field of cloud storage technology, specifically to a resource scheduling method, apparatus, storage medium, and electronic device. Background Technology

[0002] With the rapid development of cloud computing and edge computing technologies, cloud storage systems are being widely applied in various high-concurrency, periodically cyclical business scenarios. In these scenarios, user access behavior often exhibits a significant "tidal effect"—that is, the number of requests surges during specific periods (such as peak activity periods) and drops sharply at other times. This periodic fluctuation places higher demands on the scheduling efficiency and service quality of cloud storage resources, urgently requiring an optimization mechanism that can dynamically sense load changes and adaptively adjust resource allocation.

[0003] Currently, cloud storage systems generally employ static resource allocation strategies or simple scaling mechanisms based on real-time load to cope with access pressure. However, this approach cannot predict the intensity and timing of tidal loads, resulting in low resource utilization and significant fluctuations in user access latency. Summary of the Invention

[0004] In view of this, this application provides a resource scheduling method, apparatus, storage medium and electronic device, the main purpose of which is to improve the technical problem of low resource utilization and large fluctuations in user access latency caused by the inability to predict the intensity and timing of tidal loads.

[0005] In a first aspect, this application provides a resource scheduling method, comprising: acquiring historical request information of users for cloud storage services; generating a tidal prediction confidence level based on the historical request information; determining whether the current business load status meets the triggering conditions of a target fluctuation scheduling cycle based on the tidal prediction confidence level; if the current business load status meets the triggering conditions of the target fluctuation scheduling cycle, calculating the resource scheduling weight and migration performance factor corresponding to the storage node respectively; and triggering a corresponding resource scheduling operation based on the resource scheduling weight and the migration performance factor.

[0006] Secondly, this application provides a resource scheduling apparatus, comprising: an acquisition module configured to acquire historical request information of users for cloud storage services; a generation module configured to generate a tidal prediction confidence level based on the historical request information; a judgment module configured to determine, based on the tidal prediction confidence level, whether the current business load state meets the triggering conditions of a target fluctuation scheduling cycle; a calculation module configured to calculate, if the current business load state meets the triggering conditions of the target fluctuation scheduling cycle, the resource scheduling weight and migration performance factor corresponding to the storage node; and a triggering module configured to trigger a corresponding resource scheduling operation based on the resource scheduling weight and the migration performance factor.

[0007] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0008] Fourthly, this application provides an electronic device including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect.

[0009] Fifthly, this application provides a computer program product having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method of the first aspect.

[0010] By employing the aforementioned technical solution, this application provides a resource scheduling method, apparatus, storage medium, and electronic device. First, it acquires historical request information from users for cloud storage services. Then, based on this historical request information, it generates a tidal prediction confidence level. Based on the tidal prediction confidence level, it determines whether the current business load status meets the triggering conditions of the target fluctuation scheduling cycle. If the current business load status meets the triggering conditions of the target fluctuation scheduling cycle, it calculates the resource scheduling weight and migration performance factor corresponding to each storage node. Finally, based on the resource scheduling weight and migration performance factor, it triggers the corresponding resource scheduling operation. Compared with existing technologies, this application constructs a multi-stage dynamic scheduling mechanism based on tidal prediction confidence, deeply integrating user behavior characteristics, business page access patterns, and storage node status. This achieves forward-looking prediction of tidal load intensity and timing, and on this basis, it calculates the resource scheduling weight and migration performance factor in conjunction, driving the precise triggering of scheduling operations such as data migration and message queue optimization. This avoids resource mismatch caused by static or delayed scheduling strategies, effectively improving the resource utilization of the cloud storage system and reducing the volatility of user access latency.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 shows a flowchart of a resource scheduling method provided in an embodiment of this application; Figure 2 shows a flowchart of another resource scheduling method provided in an embodiment of this application; Figure 3 shows a flowchart of an example provided in an embodiment of this application; Figure 4 shows a structural schematic diagram of a resource scheduling device provided in an embodiment of this application. Detailed Implementation

[0015] The embodiments of this application will now be described in more detail with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0016] In current cloud storage architectures, resource scheduling strategies generally employ static allocation or simple dynamic scaling mechanisms based on current load. This approach has significant limitations when facing periodic or sudden surges in requests (such as e-commerce promotions or holiday events). For example, during peak periods, storage nodes may become overloaded due to a surge in requests, leading to increased response latency and decreased service quality; conversely, during off-peak periods, a large amount of resources remain idle, resulting in resource waste. Furthermore, in multi-tenant environments, different tenants have different load patterns, making fine-grained scheduling difficult with traditional methods, resulting in low resource utilization.

[0017] For example, static resource scheduling strategies lack modeling and analysis of the periodic characteristics of user behavior, making it impossible to predict tidal loads (such as peak periods during marketing activities and trough periods during non-activity periods), resulting in unreasonable resource allocation; the resource allocation strategy does not consider the matching relationship between the physical characteristics of solid state drives (SSDs) and hard disk drives (HDDs) and the load, resulting in low hardware lifespan and load matching, affecting the lifespan of storage media and scheduling efficiency.

[0018] To address the technical problem of unpredictable tidal load intensity and timing, leading to low resource utilization and significant fluctuations in user access latency, this embodiment provides a resource scheduling method applicable to the main control server's memory cache node. As shown in Figure 1, the method includes: Step 101, obtaining historical request information from users for cloud storage services.

[0019] In some examples, historical request information may include a sequence of timestamps of requests initiated by the user within a preset time period, the request frequency per unit time, and the number of concurrent requests; request types and their corresponding operation characteristics, such as file reading, uploading, previewing, or deleting, as well as the size, format, and popularity attributes of the accessed data; system response latency data, and user cancellation, retry, or redirection behaviors caused by excessive latency; usage logs of the terminal's local cache, covering cache hit counts, cache replacement frequency, cache read / write request ratios, and their trends over time; and optional device and network context information, such as terminal model, operating system version, network access type, and geographical location, to help identify the periodic patterns and tidal characteristics of user access behavior.

[0020] Step 102: Generate tidal prediction confidence based on historical request information.

[0021] For example, the confidence level for tidal prediction is a value between 0 and 1, used to quantify the predictability of user access behavior exhibiting periodic tidal characteristics over time.

[0022] In some examples, the tidal prediction confidence score can be generated by fusing three factors: the request tidal gradient, the user latency-concurrency adaptation coefficient calculated based on Shannon entropy, and the mobile local cache utilization volatility. The request tidal gradient reflects the steepness of the change in user request volume between peaks and troughs within a unit time window, the user latency-concurrency adaptation coefficient characterizes the user's tolerance for response latency and the stability of their behavior in high-concurrency scenarios, and the cache utilization volatility reflects the time-series stability of the terminal's local cache usage. The above three indicators are weighted and combined with configurable weight coefficients, and then combined with a nonlinear activation function and normalization processing (using the historical maximum cache volatility as a reference) to finally output the tidal prediction confidence score.

[0023] Step 103: Based on the tidal prediction confidence level, determine whether the current business load status meets the triggering conditions of the target fluctuation scheduling cycle.

[0024] In some examples, when the confidence level of the tidal prediction exceeds a preset threshold, the system can consider the user behavior to be periodically predictable, and then collect relevant tidal fluctuation characteristic parameters, including page request density fluctuation rate, page active period offset, edge storage tidal idle rate and storage load periodicity fitting degree, etc., and determine whether the current business load status meets the triggering conditions of the target fluctuation scheduling cycle based on the tidal fluctuation characteristic parameters.

[0025] For example, if the current business load status meets the triggering conditions of the target fluctuation scheduling cycle, it means that the system has entered the high fluctuation scheduling mode and the corresponding resource scheduling strategy needs to be adopted. Otherwise, the normal scheduling strategy is maintained to avoid unnecessary resource disturbance.

[0026] Step 104: If the current business load status meets the triggering conditions of the target fluctuation scheduling cycle, calculate the resource scheduling weight and migration performance factor corresponding to the storage node respectively.

[0027] In some examples, resource scheduling weights are generated by cloud HDD storage nodes to characterize the scheduling priority and data migration drive of the cloud storage layer under the current tidal load; migration performance factors are calculated by the master server memory cache nodes to evaluate the feasibility and timeliness of performing data migration operations on the edge side.

[0028] Step 105: Based on the resource scheduling weight and migration performance factor, trigger the corresponding resource scheduling operation.

[0029] In some examples, resource scheduling operations may include collaborative optimization processes such as cold and hot data migration, cache preheating advance calculation, and Kafka message path reconstruction.

[0030] For example, when the ratio of migration performance factor to resource scheduling weight exceeds a preset threshold, a cold and hot data migration operation is triggered; otherwise, the migration is skipped and the process directly enters the message queue optimization process.

[0031] Compared with existing technologies, the technical solution of this embodiment first obtains historical request information of users for cloud storage services; then, based on the historical request information, a tidal prediction confidence score is generated; based on the tidal prediction confidence score, it is determined whether the current business load status meets the triggering conditions of the target fluctuation scheduling cycle; if the current business load status meets the triggering conditions of the target fluctuation scheduling cycle, the resource scheduling weight and migration performance factor corresponding to the storage node are calculated respectively; then, based on the resource scheduling weight and migration performance factor, the corresponding resource scheduling operation is triggered. A multi-stage dynamic scheduling mechanism is constructed based on the tidal prediction confidence score, deeply integrating user behavior characteristics, business page access patterns, and storage node status, achieving forward-looking prediction of tidal load intensity and time points. Based on this, the resource scheduling weight and migration performance factor are calculated in conjunction, driving the precise triggering of scheduling operations such as data migration and message queue optimization. This avoids resource mismatch caused by static or lagging scheduling strategies, effectively improving the resource utilization of the cloud storage system and reducing the volatility of user access latency.

[0032] To further illustrate the specific implementation process of the method in this embodiment, this embodiment provides a method as shown in Figure 2, which includes: step 201, obtaining the user's historical request information for cloud storage services.

[0033] For example, historical request information may include timestamps of requests initiated by users within a preset time period, request frequency, and number of concurrent requests; request types and their corresponding data access characteristics, such as read, write, or preview operations, as well as the size and popularity attributes of the accessed data; system response latency data and user cancellation, redirection, or retry behavior caused by excessive latency; usage logs of the terminal's local cache, including cache hit counts, cache replacement frequency, and cache read / write request ratio; and optional device and network context information, such as terminal type, network access method, and geographical location, to assist in analyzing the periodicity and stability of user access behavior.

[0034] Step 202: Generate tidal prediction confidence based on historical request information.

[0035] Optionally, step 202 may specifically include: extracting the tidal gradient corresponding to the user request through Fourier analysis based on historical request information; generating the target adaptation coefficient according to the user's request latency tolerance and the peak concurrency ratio corresponding to the concurrent requests; performing fluctuation analysis on the local cache usage time series reported by the terminal to calculate the cache utilization volatility; and fusing the tidal gradient, the target adaptation coefficient and the cache utilization volatility to generate the tidal prediction confidence.

[0036] For example, the terminal (taking a mobile phone as an example) request tidal gradient refers to the ratio of the difference between the peak and trough of user request volume within a unit time window to the time span, used to quantify the intensity of fluctuations and the steepness of the trend in user request behavior. Request latency tolerance and peak concurrency ratio refer to collecting user request response times during peak periods (e.g., concurrent access exceeding a certain threshold) and trough periods, and recording the number of times users abandoned operations due to excessive latency (e.g., page redirection, request cancellation), forming a latency tolerance behavior curve. Simultaneously, the mobile terminal calculates the ratio of concurrent requests during peak periods to those during trough periods, obtaining the peak concurrency ratio. By normalizing the latency tolerance curve and the peak concurrency ratio, a user latency-concurrency adaptation coefficient is generated. Mobile local cache utilization volatility refers to the mobile terminal recording parameters such as cache hit count, cache replacement frequency, and cache read / write request ratio within each time window (e.g., hour), generating a local cache usage time series. Subsequently, the standard deviation of the time series is calculated using a sliding window to obtain the cache utilization volatility, which is used to determine the stability and predictability of the user's local cache.

[0037] In some examples, the mobile terminal can collect all historical request information of users, extract the mobile terminal request tidal gradient from it through Fourier analysis, calculate the request latency tolerance and peak concurrency ratio based on Shannon entropy, and calculate the mobile local cache utilization volatility through time series fluctuation analysis. The three are combined to generate the first tidal prediction confidence. If the first tidal prediction confidence is greater than the preset threshold, it indicates that it has high predictability and triggers the subsequent process. Otherwise, the process ends and prompts that as much user data as possible needs to be collected before triggering this step.

[0038] For example, the confidence level of the first tide prediction The expression is as follows:

[0039] in, Request the tidal gradient for the mobile device; To request the Shannon entropy values ​​for latency tolerance and peak concurrency ratio; Utilize volatility for local caching on mobile phones; The weighting coefficients are configurable and are dynamically updated based on historical scheduling feedback. The historical maximum value of the mobile phone cache volatility is used as a normalization reference.

[0040] Step 203: If the confidence level of the tidal prediction is greater than the preset confidence threshold, then collect the tidal fluctuation characteristic parameters corresponding to the current business load status.

[0041] For example, the above The confidence index integrates the periodic intensity of user behavior, the trend of latency tolerance changes, and cache fluctuation characteristics. Through a non-linear activation function and normalization, it generates a value between 0 and 1. If this value exceeds a preset threshold (e.g., 0.75), it indicates that the current user behavior has significant tidal characteristics and predictability, triggering subsequent scheduling processes. If it does not exceed the threshold, the process terminates, and the end user is prompted to provide more behavioral data for analysis.

[0042] Among them, the tidal fluctuation characteristic parameters may include page request density fluctuation rate, page active period offset, edge storage tidal idle rate, and storage load periodicity fit.

[0043] For example, Page Request Density Volatility refers to collecting requests per second (QPS) and calculating the standard deviation of request density within a fixed time window (e.g., 5 minutes) as the raw value of volatility. Subsequently, a sliding window weighted average method is used to smooth the volatility across multiple time windows, generating the Page Request Density Volatility index. Page Activity Period Shift refers to recording the time distribution of page visits over the past N days, extracting historical peak periods (e.g., 19:00-21:00 daily), and comparing them with the current peak period to calculate the time difference (in minutes). Edge Storage Tidal Idle Rate refers to the proportion of free storage space available for release during off-peak hours on current edge storage nodes. This is calculated by statistically analyzing storage usage during off-peak hours (e.g., early morning) and calculating the ratio of free capacity to total capacity. Storage Load Periodicity Fit refers to collecting the load time series over the past N days, extracting its periodicity features through Fourier transform, and then performing correlation analysis between the current load and historical periodic loads. The closer the result is to 1, the higher the consistency between the current load and historical patterns; otherwise, it indicates a sudden change in load.

[0044] Step 204: Construct a multivariate tidal scheduling trigger function based on tidal prediction confidence and tidal fluctuation characteristic parameters.

[0045] For example, multivariate tidal scheduling trigger function The expression is as follows:

[0046] in, Page request density volatility; Offset during peak page activity period; Store tidal idle rates at the edge; For storage load periodicity fit; The weighting coefficients are adjustable and dynamically optimized based on historical scheduling feedback and A / B test data. This is the Sigmoid activation function, used to map continuous values ​​to Boolean probability values ​​between 0 and 1.

[0047] Step 205: Based on the multivariate tidal scheduling trigger function, determine whether the current business load status meets the trigger conditions of the target fluctuation scheduling cycle.

[0048] For example, if If a threshold (such as 0.65) is set, the current business load status is determined to meet the triggering conditions of the target fluctuation scheduling cycle, that is, the current period enters the high fluctuation scheduling cycle.

[0049] In some examples, when determining whether to activate the resource scheduling mechanism, the system first collects access characteristic data of the current marketing campaign's HTML page, including the page request density fluctuation rate calculated based on requests per second, and the page activity time offset determined by comparing the offset between historical peak times and the current peak access time. Then, the system integrates the above page-level indicators with the first tidal prediction confidence score generated by the user terminal, and further introduces infrastructure-level state parameters—namely, the proportion of free storage space that edge SSD storage nodes can release during off-peak hours (edge ​​storage tidal idle rate) and the correlation between the current load of cloud HDD storage nodes and historical periodic patterns (storage load periodic fit). These multi-source heterogeneous parameters serve as input variables to construct a multivariate tidal scheduling trigger function. This function outputs a probability value between 0 and 1 through weighted combination and nonlinear mapping (such as Sigmoid activation). When this output value exceeds a preset threshold, the system determines that the current business scenario has entered a high-fluctuation scheduling cycle, thereby triggering the subsequent edge-cloud collaborative scheduling process.

[0050] Step 206: If the current business load status meets the triggering conditions of the target fluctuation scheduling cycle, then calculate the resource scheduling weight and migration performance factor corresponding to the storage node respectively.

[0051] Optionally, step 206 may specifically include: obtaining the tidal load status characteristic parameters corresponding to the cloud storage node, wherein the tidal load status characteristic parameters include the throughput tidal fluctuation coefficient, the cold and hot data resident ratio, and the synchronous message queue tidal throughput ratio; and generating resource scheduling weights based on the tidal load status characteristic parameters.

[0052] For example, cloud-based HDD storage nodes calculate the HDD throughput tidal fluctuation coefficient based on their historical throughput time series to characterize the stability of read / write throughput under tidal loads. Simultaneously, by analyzing the spatial proportion of high-frequency and low-frequency access data within a unit time window, a hot / cold data residency ratio is generated to reflect the current distribution of storage content's popularity. Furthermore, the tidal throughput ratio of the synchronous message queue reported by the Kafka server—the ratio of message production to consumption rates during peak and off-peak periods—is used to measure the dynamic demand for data flow between the edge and the cloud. The cloud-based HDD storage nodes use these three metrics as input, generate a third cloud storage elastic scheduling weight through convolutional fusion, and send this weight to the master server's memory cache node. Based on this, the master server drives the generation of a hot / cold data tiered migration strategy, prioritizing the migration of high-frequency, latency-sensitive hot data from the HDD to the low-latency edge SSD storage layer, thereby optimizing the overall storage architecture's response efficiency and resource matching.

[0053] Optionally, the aforementioned acquisition of the tidal load status characteristic parameters corresponding to the cloud storage node may specifically include: determining the throughput tidal fluctuation coefficient based on the throughput changes of the cloud storage node in a first preset time period; determining the hot and cold data residency ratio based on the proportion of data with access frequency higher than a preset frequency threshold in a second preset time period; and determining the synchronous message queue tidal throughput ratio based on the changes in message production and consumption rates of the message queue server in a third preset time period.

[0054] For example, the HDD throughput tidal fluctuation coefficient refers to the ratio of the standard deviation to the average of the peak throughput and the trough throughput over the past N days (such as hourly read and write volume), used to evaluate the throughput stability of HDDs under tidal loads; the hot and cold data residency ratio refers to the ratio of the space occupied by data with access frequency above a threshold within a unit time window (hot data) and below a threshold (cold data) through access log analysis, generating the hot and cold data residency ratio index; the synchronous message queue tidal throughput ratio refers to the ratio of the production and consumption rate of messages collected by Kafka nodes within a unit time window, calculated as the ratio of peak throughput to trough throughput, used to reflect the dynamic data flow demand between the edge and the cloud.

[0055] For example, the third cloud storage elastic scheduling weight The generative expression is as follows:

[0056] in, This refers to the HDD throughput tidal fluctuation coefficient. This refers to the ratio of hot to cold data residence time in HDDs. This refers to the tidal throughput ratio of the Kafka synchronous message queue. Configurable weighting coefficients; This is a smoothing factor to prevent division by zero errors; This is a temporal infinitesimal element used for convolution to fuse historical and current data. Ultimately, this... The weight values ​​are normalized and then sent to the master server's memory cache node to drive the hierarchical migration strategy of hot and cold data and optimize the scheduling efficiency of cloud storage resources.

[0057] Optionally, step 206 may further include: obtaining the edge storage tidal idle rate and fragmented hot-cold ratio corresponding to the edge storage node; calculating the edge resource elastic redundancy value based on the edge storage tidal idle rate and fragmented hot-cold ratio; and determining the migration performance factor based on the edge resource elastic redundancy value, the hot data migration lag time of the edge storage node, and the cache hit volatility of the cache node.

[0058] Optionally, obtaining the edge storage tidal idle rate and fragmentation hot-cold ratio corresponding to the edge storage node may specifically include: determining the edge storage tidal idle rate based on the proportion of free storage space available in the edge storage node; and determining the fragmentation hot-cold ratio based on the ratio of hot data to cold data in the fragmented space of the edge storage node.

[0059] For example, the edge SSD storage nodes can report two data points: the current edge storage tidal idle rate and the SSD fragmentation hot-cold ratio. An adaptive weighted algorithm is used to generate a second edge resource elastic redundancy value, and the result is sent to the master server memory cache node.

[0060] In some examples, edge storage tidal idle rate refers to the proportion of free storage space that edge SSD storage nodes can release during off-peak hours. It is calculated by statistically analyzing the storage usage of edge nodes during low-load periods (such as early morning) and calculating the ratio of their free capacity to total capacity, reflecting the resource redundancy capability of edge nodes during off-peak hours. SSD fragmentation hot-cold ratio refers to the ratio of hot data to cold data in the fragmented space of edge SSD storage nodes, used to assess the current degree of storage space fragmentation and its impact on the efficiency of hot data access.

[0061] For example, the second edge resource elastic redundancy value The generative expression is as follows:

[0062] in, Store tidal idle rates at the edge; For SSD fragmentation, improve the hot-to-cold ratio; This represents the largest fragmented heat-to-cold ratio in history, used for normalization processing; These are adaptive weighting coefficients. The redundancy value ranges from [0,1]. A higher value indicates that the edge node has a higher resource release capability and lower fragmentation interference, making it suitable as a cache or migration target node.

[0063] For example, hot data migration lag time refers to the time latency required for an edge SSD storage node to migrate frequently accessed hot data from the cold storage layer to the hot storage layer, used to measure the responsiveness of edge nodes when performing migration operations. Cache hit volatility refers to the standard deviation of the cache hit rate fluctuation of the master server's memory cache nodes during peak and off-peak periods, used to evaluate the stability and adaptability of caching strategies.

[0064] For example, the fourth migration delay ratio co-factor (i.e., migration performance factor). The expression is as follows:

[0065] in, Lag time for hot data migration of edge SSD nodes (in milliseconds); Cache hit volatility (in percentage) of the main server's memory cache nodes; This is the second edge resource elastic redundancy value generated in step three. If it is not generated, the default value of 1 is used. This is the volatility weighting coefficient, used to adjust the impact of cache volatility on the migration delay co-factor; It should be a very small positive number to prevent the denominator from being zero. The larger the value, the more severe the migration delay or the more drastic the cache fluctuation, requiring priority to be given to cache preheating and edge resource scheduling optimization; the smaller the value, the higher the migration efficiency and the more stable the cache, and the message queue optimization stage can be entered.

[0066] Step 207: Based on the resource scheduling weight and migration performance factor, trigger the corresponding resource scheduling operation.

[0067] Optionally, step 207 may specifically include: if the ratio of migration performance factor to resource scheduling weight exceeds a preset migration threshold, then perform the corresponding data migration operation; if the ratio of migration performance factor to resource scheduling weight does not exceed the preset migration threshold, then dynamically adjust the message distribution path of the message queue server.

[0068] For example, based on the edge resource elastic redundancy value (if none, take the default value of 1), the hot data migration lag time of the edge SSD storage node and the cache hit volatility of the master server memory cache node can be integrated to calculate the fourth migration latency ratio coordination factor. Then, based on the ratio of the fourth migration latency ratio coordination factor to the third cloud storage elastic scheduling weight, it can be determined whether the current node load level needs to perform a cold data to hot data migration operation.

[0069] In some examples, the determination of whether to perform a cold data to hot data migration operation is as follows:

[0070] in, The fourth migration delay ratio is the co-factor. The third cloud storage elastic scheduling weight generated in step four; The threshold value is a configurable threshold value, i.e., a preset migration threshold value.

[0071] For example, when When the ratio exceeds the set threshold, it indicates that the edge node migration delay is large or the cache hit is unstable. It is necessary to migrate the edge cold data to the cloud first to free up continuous storage space and improve the access efficiency of edge hot data. If the ratio does not exceed the threshold, it indicates that the edge node scheduling capability is good. The cold data migration can be skipped and the Kafka message queue optimization process can be directly entered.

[0072] Optionally, the above-mentioned data migration operations may include: generating a scheduling response distribution probability based on the migration performance factor and the cache hit tidal fluctuation rate of the cache node; dynamically adjusting the migration probability of each storage unit in the edge storage node according to the scheduling response distribution probability; and performing the corresponding data migration operation according to the migration probability.

[0073] In some examples, the master server's memory cache node uses two key indicators—cache hit tidal fluctuation rate and fourth migration latency ratio—to perform nonlinear normalization and fusion processing on the two using a hyperbolic tangent function, mapping the original values ​​to the [0,1] interval to generate a fifth scheduling response distribution probability. This distribution probability is not a static weight, but a probability distribution that dynamically changes with the system load state, used to characterize the relative priority of each storage unit being selected to perform migration operations under the current tidal fluctuation environment. Based on this, the master server assigns differentiated migration probabilities to all storage units in the edge SSD storage node according to this probability, and performs a data migration operation once according to this probability in each scheduling time unit. For example, it migrates hot data blocks with high probability from the cloud HDD to the edge SSD in advance, or sinks low-activity cold data back.

[0074] For example, cache hit tidal volatility refers to the fluctuation of cache hit rate of master server memory cache nodes during peak and trough periods. The standard deviation of the hit rate is calculated in multiple tidal cycles (such as hourly and daily peaks), and a volatility index between 0 and 1 is generated through Gaussian normalization to measure the stability of the caching strategy under periodic load.

[0075] For example, the probability distribution of the fifth scheduling response. The expression is as follows:

[0076] in, To cache hit tidal volatility; The fourth migration delay ratio is the co-factor. , These are adjustable weighting coefficients; Let be the hyperbolic tangent function, mapping the linear combination to response probability values ​​between [-1, 1]. The probability value is further mapped to the [0,1] interval, serving as the trigger weight for the migration operation of each storage unit on the edge SSD node. Based on this probability value, the master server's memory cache node performs migration evaluation and scheduling operations on the storage units of the edge storage node every unit of time (e.g., every minute) to ensure improved cache hit rate and optimized edge storage fragmentation.

[0077] In some examples, within each scheduling time unit, after the master server's memory cache node completes the cache policy update, the system synchronously adjusts the pre-read data window of the user's mobile terminal to further optimize the access experience on the terminal side. This adjustment is based on the probability distribution of the fifth scheduling response, combined with the ratio between the cache preheating task completion delay recorded by the master server and the peak time of user requests (i.e., the preheating delay peak ratio), as well as the request latency tolerance and peak concurrency ratio determined by the historical behavior of the user's mobile terminal. The master server uses the above multi-dimensional parameters to dynamically calculate the cache scheduling advance Δt, which represents the time offset required to start data preheating before the predicted access peak arrives. By sending Δt to the user terminal, the system ensures that the corresponding amount of data has been preloaded before the actual request peak occurs, so that high-frequency access content resides in the local cache or edge SSD in advance, thereby effectively alleviating the concentrated read and write pressure on edge SSD and cloud HDD storage nodes during peak periods and improving overall response stability and resource utilization efficiency.

[0078] For example, the preheating delay peak ratio refers to the main control server recording the initiation time and completion time of each cache preheating task, calculating its delay time, and performing a ratio calculation with the peak time of user requests (determined by the ratio of the mobile terminal request tidal gradient in step one to the page active period offset in step two) to generate the preheating delay peak ratio index.

[0079] For example, the generation expression of the cache scheduling advance Δt is as follows:

[0080] in, The cache warm-up delay time (in milliseconds) for the main control server's memory cache node; The user's mobile terminal's request latency tolerance (unit: milliseconds) during high-concurrency periods is generated by the definition in step one; The fifth scheduling response distribution probability reflects the stability of the current scheduling response and the migration capability of edge nodes; This is a warm-up time adjustment factor; the lead time Δt is dynamically updated in each scheduling cycle. The master control server sends Δt to the user's mobile terminal, and the terminal loads the pre-read data according to this Δt time, ensuring that the cache warm-up is completed when the actual peak arrives, thus improving the user access experience.

[0081] Optionally, the above-mentioned dynamic adjustment of the message distribution path of the message queue server may specifically include: obtaining the number of partition consumption threads of the message queue server; and dynamically adjusting the message distribution path of the message queue server based on the number of partition consumption threads, resource scheduling weight, and migration performance factor.

[0082] For example, if there is no need to perform cold data to hot data migration operation, the Kafka server can be used to combine the standard deviation of message queue consumption lag and the average message write displacement distance, and dynamically determine the number of Kafka partition consumption threads based on the Poisson distribution model. After obtaining the number of partition consumption threads, the Kafka message distribution path topology is reconstructed based on the third cloud storage elastic scheduling weight and the fourth migration latency ratio coordination factor, so that it accurately matches the real-time load status of edge SSD or cloud HDD storage nodes.

[0083] For example, the standard deviation of message queue consumption lag refers to the time-series data of producer write rate and consumer processing rate collected by the Kafka server in each partition, calculating the difference between the two over all time units, and calculating its standard deviation through a sliding window. This is used to assess message backlog and whether the consumption threads need dynamic scaling. The average message write offset distance refers to the average offset distance between adjacent write operations when writing Kafka messages to the log file. By recording the offset of each message when writing to the log file, the difference between the write offsets of adjacent messages is calculated, and the average of the offsets is taken through a sliding window. This is used to measure the stability of message writing and disk I / O pressure.

[0084] For example, the number of consumer threads for Kafka partitions. The generative expression is as follows:

[0085] in, The standard deviation of message queue consumption lag; Average message write offset distance; Fourth migration delay ratio co-factor (generated in step five); The expected value is a Poisson distribution, dynamically adjusted based on historical consumption efficiency and message backlog. This is a rounding function used to ensure the thread count is an integer; the thread count... The generation mechanism integrates lag period, write distance and migration factor through nonlinear functions to achieve adaptive scaling of consumer threads, ensuring that message processing capacity matches the overall system load.

[0086] Optionally, the aforementioned partition consumption thread count, resource scheduling weight, and migration performance factor dynamically adjust the message distribution path of the message queue server. Specifically, this may include: determining a message path allocation decision factor based on the partition consumption thread count, resource scheduling weight, and migration performance factor; if the message path allocation decision factor is greater than a preset routing threshold, then the corresponding message partition is routed to a cloud storage node; if the message path allocation decision factor is less than or equal to the preset routing threshold, then the corresponding message partition is routed to an edge storage node.

[0087] In some examples, based on the number of partition consumption threads Third cloud storage elastic scheduling weight and the fourth migration delay ratio co-factor The specific method for reconstructing the Kafka message distribution path topology is as follows: The Kafka server assigns decision factors to each message partition when constructing the message path.

[0088] in, This is used to determine whether a message should be distributed to an edge SSD node or a cloud HDD node. If If the current partition message is not found, it will be routed to the cloud HDD storage node first, and sorted according to the hot / cold data resident ratio, with priority given to HDD nodes with higher values; if If the message is not found, it will be routed to the edge SSD storage node and sorted according to the hot data migration lag time, with priority given to SSD nodes with higher values.

[0089] Optionally, the method in this embodiment may further include: obtaining the difference between read and write operations of edge storage nodes during peak and off-peak hours, and the fragmentation hot-cold ratio of cloud storage nodes; generating a dynamic allocation coefficient based on the difference between read and write operations and the fragmentation hot-cold ratio; and periodically updating the cache scheduling advance configured on the terminal according to the dynamic allocation coefficient.

[0090] In some examples, edge SSD nodes periodically report poor read / write performance during peak / off-peak periods, while cloud HDD nodes periodically report fragmented hot / cold ratios. These metrics are combined to generate a dynamic allocation coefficient, which is used to periodically update the cache scheduling lead time Δt and the number of Kafka partition consumer threads. These key parameters enable continuous optimization of edge-cloud resource utilization. Specifically, the peak / off-peak read / write operation difference refers to the difference in the total number of read / write requests for edge SSD storage nodes during peak and off-peak periods, used to quantify the load fluctuations of edge nodes at different times; the fragmentation hot / cold ratio refers to the ratio of hot data to cold data space in storage fragments of cloud HDD storage nodes, used to measure the degree of matching between cloud storage fragmentation and data heat distribution.

[0091] For example, dynamic allocation coefficient The generation method is as follows:

[0092] in, The peak / off-peak read / write operations of edge SSD storage nodes are poor; Fragmented hot-cold ratio for cloud-based HDD nodes; The maximum value of the historical fragmented heat-to-cold ratio is used for normalization; This is an amplification factor for edge load fluctuations, used to adjust the weight of the load difference on scheduling; in some examples, it updates the cache scheduling lead Δt and the number of Kafka partition consumer threads. The method is as follows:

[0093]

[0094] in, Lead time for new cache scheduling; This is the lead time for the current cache scheduling; This is the scheduling lead adjustment factor, a configurable parameter with a value between [0,1]. Number of consumption threads for the new partition; This represents the current number of consumer threads for the Kafka partition. As the third cloud storage elastic scheduling weight; The fourth migration delay ratio is the co-factor. To prevent small positive numbers from being divided by zero.

[0095] In some examples, as shown in Figure 3, this embodiment proposes a cloud storage resource utilization optimization method based on a dynamic tidal model. Through an edge-cloud resource collaborative scheduling mechanism and cache preheating delay control, it achieves adaptive and efficient resource scheduling. This embodiment mainly involves core components such as user mobile terminals, marketing campaign HTML pages, Kafka servers, edge SSD storage nodes, cloud HDD storage nodes, and master server memory cache nodes. The specific interaction logic is as follows: The user mobile terminal collects historical request information and uploads it to the master server memory cache node. Simultaneously, it generates a first tidal prediction confidence score based on Fourier analysis and time series fluctuation analysis. The master server combines the page request density fluctuation rate and page active time offset of the current marketing campaign HTML page to construct a multivariate tidal scheduling trigger function. It then inputs the edge storage tidal idle rate of the edge SSD node and the storage load periodicity fit of the cloud HDD node to determine whether to proceed. The system enters a high-fluctuation scheduling cycle. If it does, the edge SSD nodes report the peak / valley read / write operation difference, and the cloud HDD nodes report the fragmented hot / cold ratio. The master control server merges these to generate a dynamic allocation coefficient, which is used to update the cache scheduling lead Δt and the number of Kafka partition consumer threads N_consumer. The master control server dynamically adjusts the migration probability of the edge storage units according to the fifth scheduling response distribution probability, and converts the cache hit tidal fluctuation rate and the fourth migration delay ratio co-factor into a migration strategy through the hyperbolic tangent function. The Kafka server receives messages and distributes them according to the reconstructed consumption path topology, ultimately completing the data migration, preheating window update, and resource scheduling closed loop, achieving collaborative optimization of edge and cloud resources.

[0096] Compared with existing technologies, this embodiment introduces a probability-driven migration mechanism. By dynamically adjusting the migration probability of edge storage units through the distribution probability of scheduling response, it achieves fine-grained migration control and precise matching of scheduling time windows. This effectively alleviates resource contention and cache hit rate decline caused by rigid strategies, significantly improving the elasticity and load response efficiency of edge-side data scheduling. Simultaneously, addressing the preheating lag problem caused by traditional cache preheating relying on fixed time windows or simple event triggers, this embodiment innovatively implements dynamic generation and closed-loop feedback control of the cache scheduling lead time Δt, enabling the preheating operation to be completed before the peak request period, thereby reducing... This reduces the read / write pressure on edge SSDs and cloud HDD storage nodes during periods of low to high concurrency, improving the stability of user access responses. Furthermore, unlike existing Kafka message routing strategies that lack awareness of storage layer status, this embodiment dynamically binds message distribution paths to the real-time load status of edge and cloud storage nodes (such as resource scheduling weights and migration performance factors), constructing an intelligent collaborative mechanism between message flow and storage flow. This solves the problems of low resource utilization and latency fluctuations caused by the disconnect between message scheduling and storage resources, comprehensively enhancing the collaborative scheduling efficiency between the Kafka message queue and storage system, as well as the overall architecture's elastic processing capabilities.

[0097] Furthermore, as a specific implementation of the methods shown in Figures 1 and 2, this embodiment provides a resource scheduling device, as shown in Figure 4. The device includes: an acquisition module 31, a generation module 32, a judgment module 33, a calculation module 34, and a triggering module 35.

[0098] The acquisition module 31 is configured to acquire historical request information of users for cloud storage services; the generation module 32 is configured to generate tidal prediction confidence based on the historical request information; the judgment module 33 is configured to determine whether the current business load status meets the triggering conditions of the target fluctuation scheduling cycle based on the tidal prediction confidence; the calculation module 34 is configured to calculate the resource scheduling weight and migration performance factor corresponding to the storage node if the current business load status meets the triggering conditions of the target fluctuation scheduling cycle; and the triggering module 35 is configured to trigger the corresponding resource scheduling operation based on the resource scheduling weight and the migration performance factor.

[0099] In some examples of this embodiment, the generation module 32 is further configured to extract the tidal gradient corresponding to the user request through Fourier analysis based on the historical request information; generate a target adaptation coefficient according to the user's request latency tolerance and the peak concurrency ratio corresponding to the concurrent requests; perform fluctuation analysis on the local cache usage time series reported by the terminal to calculate the cache utilization volatility; and fuse the tidal gradient, the target adaptation coefficient and the cache utilization volatility to generate the tidal prediction confidence.

[0100] In some examples of this embodiment, the judgment module 33 is further configured to: if the tidal prediction confidence level is greater than a preset confidence threshold, collect tidal fluctuation characteristic parameters corresponding to the current business load state, the tidal fluctuation characteristic parameters including page request density fluctuation rate, page active period offset, edge storage tidal idle rate, and storage load periodicity fit; construct a multivariate tidal scheduling trigger function based on the tidal prediction confidence level and the tidal fluctuation characteristic parameters; and determine whether the current business load state meets the triggering conditions of the target fluctuation scheduling cycle according to the multivariate tidal scheduling trigger function.

[0101] In some examples of this embodiment, the calculation module 34 is further configured to obtain the tidal load status characteristic parameters corresponding to the cloud storage node, the tidal load status characteristic parameters including the throughput tidal fluctuation coefficient, the cold and hot data resident ratio and the synchronous message queue tidal throughput ratio; and generate the resource scheduling weight based on the tidal load status characteristic parameters.

[0102] In some examples of this embodiment, the calculation module 34 is further configured to determine the throughput tidal fluctuation coefficient based on the throughput changes of the cloud storage node in a first preset time period; and to determine the hot and cold data residency ratio based on the proportion of data with access frequency higher than a preset frequency threshold in a second preset time period; and to determine the synchronous message queue tidal throughput ratio based on the changes in the message production and consumption rate of the message queue server in a third preset time period.

[0103] In some examples of this embodiment, the calculation module 34 is further configured to obtain the edge storage tidal idle rate and fragmented hot-cold ratio corresponding to the edge storage node; calculate the edge resource elastic redundancy value based on the edge storage tidal idle rate and the fragmented hot-cold ratio; and determine the migration performance factor based on the edge resource elastic redundancy value, the hot data migration lag time of the edge storage node, and the cache hit volatility of the cache node.

[0104] In some examples of this embodiment, the calculation module 34 is further configured to determine the edge storage tidal idle rate based on the proportion of free storage space available in the edge storage node; and to determine the fragmentation hot-cold ratio based on the ratio of hot data to cold data in the fragmented space of the edge storage node.

[0105] In some examples of this embodiment, the trigger module 35 is further configured to perform a corresponding data migration operation if the ratio of the migration performance factor to the resource scheduling weight exceeds a preset migration threshold; and to dynamically adjust the message distribution path of the message queue server if the ratio of the migration performance factor to the resource scheduling weight does not exceed the preset migration threshold.

[0106] In some examples of this embodiment, the trigger module 35 is further configured to generate a scheduling response distribution probability based on the migration performance factor and the cache hit tidal fluctuation rate of the cache node; dynamically adjust the migration probability of each storage unit in the edge storage node according to the scheduling response distribution probability; and perform corresponding data migration operations according to the migration probability.

[0107] In some examples of this embodiment, the triggering module 35 is further configured to obtain the number of partition consumption threads of the message queue server; and dynamically adjust the message distribution path of the message queue server based on the number of partition consumption threads, the resource scheduling weight, and the migration performance factor.

[0108] In some examples of this embodiment, the triggering module 35 is further configured to determine a message path allocation decision factor based on the number of partition consumption threads, the resource scheduling weight, and the migration performance factor; if the message path allocation decision factor is greater than a preset routing threshold, the corresponding message partition is routed to a cloud storage node; if the message path allocation decision factor is less than or equal to the preset routing threshold, the corresponding message partition is routed to an edge storage node.

[0109] In some examples of this embodiment, the trigger module 35 is further configured to obtain the difference between read and write operations of the edge storage node during peak and off-peak hours, and the fragmentation hot-cold ratio of the cloud storage node; generate a dynamic allocation coefficient based on the read and write operation difference and the fragmentation hot-cold ratio; and periodically update the cache scheduling advance configured on the terminal according to the dynamic allocation coefficient.

[0110] It should be noted that the other corresponding descriptions of the various functional units involved in the resource scheduling device provided in this embodiment can be found in the corresponding descriptions in Figures 1 and 2, and will not be repeated here.

[0111] Based on the methods shown in Figures 1 and 2, this embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the methods shown in Figures 1 and 2.

[0112] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0113] Based on the methods shown in Figures 1 and 2, and the virtual device embodiment shown in Figure 4, in order to achieve the above objectives, this application also provides an electronic device, such as a personal computer, server, laptop computer, intelligent robot, and other intelligent terminal. The device includes a storage medium and a processor; the storage medium is used to store computer programs; the processor is used to execute the computer programs to implement the methods shown in Figures 1 and 2.

[0114] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0115] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0116] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the solution of this embodiment, compared with the existing technology, this embodiment collects user request behavior, marketing page access characteristics, and real-time status of edge SSD and cloud HDD storage nodes, and integrates the dynamic scheduling mechanism of Kafka message queues to build a multi-dimensional perception system covering terminal-edge-cloud. It realizes predictive identification, adaptive response and closed-loop optimization of tidal loads, effectively solving the core pain points in the current collaborative scheduling of edge computing and cloud storage, such as resource matching lag, rigid scheduling strategy, high response latency during peak user access, and inefficient hierarchical scheduling of hot and cold data. In internal applications, this embodiment is suitable for high-concurrency edge storage scenarios such as cloud disks and cloud storage platforms. It can significantly improve the resource release efficiency of edge SSD nodes, the pre-read hit rate of user terminals, and the intelligent routing capability of Kafka message streams. In typical business scenarios such as marketing peaks, live interactive events, and cloud collaboration, it can achieve adaptive resource scheduling and low-latency response, reduce systemic lag rates, and optimize the end-to-end user access experience. In external applications, this embodiment can be extended to a wide range of scenarios such as CDN edge caching scheduling systems, short video platform content preheating strategies, and elastic storage scheduling services during e-commerce promotions. It can be seamlessly embedded into a Kafka-driven distributed processing architecture, improving the load matching degree between edge and cloud storage and the efficiency of hot and cold data tiered scheduling. It is especially suitable for business scenarios with instantaneous high concurrency and significant tidal fluctuations, such as live streaming, e-commerce flash sales, and video-on-demand preloading, demonstrating good cross-platform adaptability and engineering implementation value. Furthermore, this embodiment can improve the cache hit rate by more than 30% in the edge-cloud hybrid storage architecture, reduce storage node resource waste by 25% to 40%, and improve Kafka message consumption efficiency by about 20%. During peak marketing campaigns, it can significantly reduce service interruption and resource contention rates caused by unpreheated caches or overloaded edge nodes, providing efficient, stable, and intelligent underlying scheduling support for high-concurrency cloud services.

[0118] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0119] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A resource scheduling method, characterized in that, include: Obtain users' historical request information for cloud storage services; Based on the historical request information, generate a tidal prediction confidence level; Based on the tidal prediction confidence level, determine whether the current service load status meets the triggering conditions of the target fluctuation scheduling cycle; If the current business load status meets the triggering conditions of the target fluctuation scheduling cycle, then calculate the resource scheduling weight and migration performance factor corresponding to the storage node respectively; Based on the resource scheduling weight and the migration performance factor, the corresponding resource scheduling operation is triggered.

2. The method according to claim 1, characterized in that, Based on the historical request information, a tidal prediction confidence score is generated, including: extracting the tidal gradient corresponding to the user request through Fourier analysis based on the historical request information; generating a target adaptation coefficient based on the user's request latency tolerance and the peak concurrency ratio corresponding to the concurrent requests; performing fluctuation analysis on the local cache usage time series reported by the terminal to calculate the cache utilization volatility; and fusing the tidal gradient, the target adaptation coefficient, and the cache utilization volatility to generate the tidal prediction confidence score.

3. The method according to claim 1, characterized in that, Based on the tidal prediction confidence level, determine whether the current business load status meets the triggering conditions of the target fluctuation scheduling cycle, including: if the tidal prediction confidence level is greater than a preset confidence threshold, collect tidal fluctuation characteristic parameters corresponding to the current business load status, the tidal fluctuation characteristic parameters including page request density volatility, page active period offset, edge storage tidal idle rate, and storage load periodicity fit; construct a multivariate tidal scheduling trigger function based on the tidal prediction confidence level and the tidal fluctuation characteristic parameters; and determine whether the current business load status meets the triggering conditions of the target fluctuation scheduling cycle according to the multivariate tidal scheduling trigger function.

4. The method according to claim 1, characterized in that, Calculating the resource scheduling weights corresponding to storage nodes includes: obtaining the tidal load status characteristic parameters corresponding to cloud storage nodes, wherein the tidal load status characteristic parameters include the throughput tidal fluctuation coefficient, the cold and hot data resident ratio, and the synchronous message queue tidal throughput ratio; and generating the resource scheduling weights based on the tidal load status characteristic parameters.

5. The method according to claim 4, characterized in that, The method of obtaining tidal load status characteristic parameters corresponding to cloud storage nodes includes: determining the throughput tidal fluctuation coefficient based on the throughput changes of cloud storage nodes in a first preset time period; determining the hot and cold data resident ratio based on the proportion of data with access frequency higher than a preset frequency threshold in a second preset time period; and determining the synchronous message queue tidal throughput ratio based on the changes in message production and consumption rates of message queue servers in a third preset time period.

6. The method according to claim 1, characterized in that, Calculating the migration performance factor corresponding to the storage node includes: obtaining the edge storage tidal idle rate and fragmented hot-cold ratio corresponding to the edge storage node; calculating the edge resource elastic redundancy value based on the edge storage tidal idle rate and the fragmented hot-cold ratio; and determining the migration performance factor based on the edge resource elastic redundancy value, the hot data migration lag time of the edge storage node, and the cache hit volatility of the cache node.

7. The method according to claim 6, characterized in that, Obtaining the edge storage tidal idle rate and fragmented hot-cold ratio corresponding to the edge storage node includes: determining the edge storage tidal idle rate based on the proportion of free storage space available in the edge storage node; and determining the fragmented hot-cold ratio based on the ratio of hot data to cold data in the fragmented space of the edge storage node.

8. The method according to claim 1, characterized in that, Based on the resource scheduling weight and the migration performance factor, a corresponding resource scheduling operation is triggered, including: if the ratio of the migration performance factor to the resource scheduling weight exceeds a preset migration threshold, then a corresponding data migration operation is performed; if the ratio of the migration performance factor to the resource scheduling weight does not exceed the preset migration threshold, then the message distribution path of the message queue server is dynamically adjusted.

9. The method according to claim 8, characterized in that, Performing corresponding data migration operations includes: generating a scheduling response distribution probability based on the migration performance factor and the cache hit tidal fluctuation rate of the cache node; dynamically adjusting the migration probability of each storage unit in the edge storage node according to the scheduling response distribution probability; and performing corresponding data migration operations according to the migration probability.

10. The method according to claim 8, characterized in that, Dynamically adjusting the message distribution path of the message queue server includes: obtaining the number of partition consumption threads of the message queue server; and dynamically adjusting the message distribution path of the message queue server based on the number of partition consumption threads, the resource scheduling weight, and the migration performance factor.

11. The method according to claim 10, characterized in that, The message distribution path of the message queue server is dynamically adjusted based on the number of partition consumption threads, the resource scheduling weight, and the migration performance factor, including: determining a message path allocation decision factor based on the number of partition consumption threads, the resource scheduling weight, and the migration performance factor; if the message path allocation decision factor is greater than a preset routing threshold, the corresponding message partition is routed to a cloud storage node; if the message path allocation decision factor is less than or equal to the preset routing threshold, the corresponding message partition is routed to an edge storage node.

12. The method according to claim 1, characterized in that, The method further includes: obtaining the difference between read and write operations of edge storage nodes during peak and off-peak hours, and the fragmentation hot-cold ratio of cloud storage nodes; generating a dynamic allocation coefficient based on the read and write operation difference and the fragmentation hot-cold ratio; and periodically updating the cache scheduling advance configured on the terminal according to the dynamic allocation coefficient.

13. A resource scheduling device, characterized in that, include: The acquisition module is configured to acquire historical request information from users for cloud storage services. The generation module is configured to generate tidal prediction confidence based on the historical request information; The judgment module is configured to determine whether the current service load status meets the triggering conditions of the target fluctuation scheduling cycle based on the tidal prediction confidence level. The calculation module is configured to calculate the resource scheduling weight and migration performance factor corresponding to the storage node if the current business load status meets the triggering conditions of the target fluctuation scheduling cycle. The triggering module is configured to trigger corresponding resource scheduling operations based on the resource scheduling weight and the migration performance factor.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 12.

15. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 12.

16. A computer program product having a computer program stored thereon, characterized in that, When the computer program product is executed by a processor, it implements the method of any one of claims 1 to 12.

Images