A random online cloud edge-end collaborative data storage method and system

Abstract

The application discloses a kind of randomness online cloud edge-end cooperation's data storage method and system, method includes: first selecting research object, in a regional range, randomly select an available area as object, the quantity n of edge node in the available area AZ is counted, then relevant parameters are set, subsequently set the size of sliding window size and statistics relevant data, set the size of sliding window z, the data request situation of real-time monitoring terminal equipment, then calculate elastic balance point β e , construct probability density function, obtain random factor k, finally design randomness online cloud edge-end cooperation data storage algorithm, whether the current data needs to be migrated by the algorithm, it provides an intelligent, flexible, efficient data storage strategy, while guaranteeing data storage performance greatly reduces the deployment cost of cloud storage service, improves the deployment efficiency of cloud storage service.

Description

Technical Field

[0001] This invention belongs to the field of cloud-edge-device collaboration and cloud storage technology, and in particular, it is a random online cloud-edge-device collaborative data storage method and system. Background Technology

[0002] In recent years, with the rapid development of the digital economy, cloud computing has maintained a strong momentum of development and is one of the hottest topics in the field of IT technology. As a product of the long-term evolution of computing technology and production practices, cloud computing can not only effectively reduce service costs but also significantly improve resource utilization. In addition, cloud computing also has many advantages such as elastic scaling, disaster recovery, and on-demand access.

[0003] As cloud computing continues to deepen and expand in the IaaS, PaaS, and SaaS fields, the surge in data volume has brought unprecedented storage pressure to major cloud service providers. Compared to the high hardware resource costs and maintenance expenses of self-built storage servers, more internet companies are choosing cloud storage systems, which offer lower storage costs and greater intelligence. Thanks to the high efficiency, low cost, intelligence, and high reliability of cloud storage, it is widely used in various production environments, such as general storage, big data analytics, database services, and microservices, playing a crucial role.

[0004] Today, with more and more smart devices connecting to the network, traditional cloud computing paradigms face significant challenges in data storage, workload, and bandwidth resources, leading to numerous problems in low-latency, high-bandwidth applications such as connected vehicles and intelligent monitoring. For data stored in the cloud, cloud-edge-device collaboration can more efficiently store data requiring real-time computation and analysis closer to the terminal devices, ensuring real-time data processing. In the "cloud-edge-device" architecture, the "cloud" is the central node responsible for managing edge computing, the "edge" is the edge side of cloud computing responsible for local data analysis, and the "device" is the terminal device responsible for data collection, sensing, and other operations. This cloud-edge-device collaboration mechanism not only effectively reduces storage costs for cloud users but also provides them with higher-quality services in both temporal and spatial dimensions.

[0005] For cloud service providers, storing data in the cloud is often more cost-effective than deploying numerous storage servers at the edge. If certain data is frequently accessed at the edge over a period of time, storing it there is clearly more suitable. This ensures data real-time performance during transmission and effectively reduces bandwidth overhead. Conversely, if certain data is accessed infrequently at the edge, storing it in the cloud is more appropriate. This not only reduces data storage costs but also alleviates the pressure on edge data storage.

[0006] In existing cloud-edge collaboration solutions, patent CN111800486A provides a resource scheduling method and system for cloud-edge collaboration. This invention makes reservation decisions in real time based on the actual bandwidth usage in different regions, effectively reducing bandwidth cost surges caused by short-term spikes in access numbers. Starting with access bandwidth, this invention monitors the access intensity of each resource within the service range of each MEC server in real time to determine if the access bandwidth of a certain resource in a certain region has reached a set threshold. It then determines whether to rent an MEC server in that region or, within the reserved period for renting a MEC server, collaboratively use both the MEC server and the cloud server to provide access. However, this invention does not fully consider the QoS of users using cloud storage services, limiting its application scenarios. Furthermore, the proposed cloud-edge collaboration resource scheduling method is relatively simple, leaving significant room for optimization in algorithm design to save on storage costs.

[0007] Patent CN113157446A provides a method, apparatus, device, and medium for resource allocation in a cloud-edge collaborative manner. This invention uses blockchain and a cloud-edge collaboration scheme to limit and control resources, initiates all sub-tasks according to coupled rule logic, and finally aggregates the results of all sub-tasks to complete the overall task. First, the cloud-edge collaboration platform analyzes the tasks requiring resource allocation, determines the task parameters and minimum split subset of the resources to be allocated, and then determines the cloud-edge collaboration scheme based on the task parameters and a task resource collaboration algorithm. Finally, based on the blockchain and the cloud-edge collaboration scheme, the tasks with the resources to be allocated are allocated resources. However, this invention introduces blockchain technology, which leads to significant energy consumption. With the surge in data volume, it can also generate performance issues of varying degrees. Furthermore, this invention does not consider reducing data storage costs, merely proposing a single, idealized cloud-edge collaborative resource allocation strategy.

[0008] In recent years, cloud service providers have increasingly focused on optimizing cloud costs, controlling cloud waste, and improving cloud deployment efficiency. Coupled with the growing number of smart devices connecting to the network, the demand for data perception at the edge is becoming increasingly strong. Therefore, how to effectively reduce cloud storage costs and improve the deployment efficiency of cloud storage services from the perspective of cloud-edge-device collaboration is of significant research importance and value. Current cloud-edge collaborative resource scheduling mechanisms are insufficient to meet the task processing needs of current application scenarios. Summary of the Invention

[0009] The purpose of this invention is to provide a randomized online cloud-edge-device collaborative data storage method and system to address the shortcomings of existing technologies. Based on the historical storage behavior of data and the processing frequency of data on both the cloud and edge sides, it provides an intelligent, flexible, and efficient data storage strategy in real time, which greatly reduces the deployment cost of cloud storage services and improves the deployment efficiency of cloud storage services while ensuring data storage performance.

[0010] One embodiment of this application provides a data storage method for randomized online cloud-edge-device collaboration, the method comprising:

[0011] Step 1: Start by selecting the research object. Within a geographical area, randomly select an availability zone as the object and count the number n of edge nodes in that availability zone AZ.

[0012] Step 2: Set the relevant parameters, let b center The amount of data stored in the cloud makes Let p be the amount of data stored at the edge. center The unit price for storing data in the cloud, Let be the unit price of data stored at the edge, where i∈[1,n], let q be the unit price of data access, let t be the storage duration, and let T be the storage billing period. Assume that the data was stored in the cloud during the past storage period, and let its storage cost be C. center Assuming the data was stored at the edge during the previous storage cycle, let its storage cost be C. edge .

[0013] Step 3: Set the sliding window size and collect relevant data. Set the sliding window size z, monitor the data requests from terminal devices in real time, record and count the number m of data access requests initiated by cloud users on the terminal within the sliding window area, and count the number of times these data are processed in the cloud and at the edge. The number of times data is processed in the cloud is denoted as h. center The number of times data processing is performed at the edge is denoted as h. edge .

[0014] Step 4: Calculate the elastic equilibrium point β eConstruct the probability density function and obtain the random factor k;

[0015] Step 5: Design a randomized online cloud-edge-device collaborative data storage algorithm;

[0016] Step 6: Determine whether the current data needs to be migrated using the aforementioned machine-based online cloud-edge-device collaborative data storage algorithm.

[0017] Optionally, the random online cloud-edge-device collaborative data storage method further includes:

[0018] If the current data needs to be migrated, the data stored at the edge will be migrated to the cloud, or the data in the cloud will be migrated to the edge. In the current storage period T, the random online cloud-edge collaborative data storage method is completed. In the next storage period T', the method is repeated. If the current data does not need to be migrated, the process will end directly.

[0019] Furthermore, the calculation of the elastic equilibrium point β e This is a preliminary step in proposing a stochastic online cloud-edge-device collaborative data storage algorithm, where the elastic equilibrium point β... e Using the parameters set in steps 1-3, the cost of storing data in the cloud during the previous storage billing cycle T is calculated to be C. center =b center ·p center ·t+z·h center / T·q, the cost of storing data at the edge is

[0020] Furthermore, the calculation of the elastic equilibrium point β e The storage cost of cloud data for Lieutenant General C center The storage cost C for edge data edge Establish the association and set the elastic interval s as [l, r], where l = min(z / T·h) center ,m), r=max(z / T·h edge ,m).

[0021] Furthermore, the elastic equilibrium point β e Calculations show that:

[0022]

[0023] Furthermore, in step 4, the constructed probability density function is obtained by combining the calculation formula for data storage cost with the set relevant parameters and the calculated elastic equilibrium point β. e Among them, The constructed probability density function is as follows:

[0024]

[0025] Where e is the natural index, δ(·) is the Dirac delta function, and k is a random factor.

[0026] Furthermore, the constructed probability density function is used to obtain the random factor k by calculating its inverse function, where let Mathematical expectation of function f(y) The inverse function of f(k) is used to obtain the random factor k, and the formula for calculating the random factor k is as follows:

[0027]

[0028] Furthermore, in step 5, the design of the stochastic online cloud-edge-device collaborative data storage algorithm includes the following steps:

[0029] At the current time t0, count the number of times h that the data was accessed in the cloud and at the edge during the period from time t0-z+1 to time t0. center and h edge And thus calculate the storage cost of the data during this period. That for

[0030]

[0031] If the current data is stored at the edge, compare the storage cost of the data at the current time t0. and the magnitude of the random factor k;

[0032] If the current data is stored in the cloud, compare the storage costs of the data at the current time t0. and the magnitude of the random factor k;

[0033] A multi-level queue coordination mechanism is set up, creating n-level storage queues. At the current time t0, the storage costs of the data are compared. The size of the random factor k determines whether the data should be stored in the cloud or at the edge. For data that needs to be physically migrated, it is first stored in queue n based on the principle of locality. i The data is cached in the cache and then migrated.

[0034] Another embodiment of this application provides a randomized online cloud-edge-device collaborative data storage system, the system comprising:

[0035] The data acquisition module obtains resource request information from the cloud, edge, and terminal, and receives data access record information connected to the server.

[0036] The real-time monitoring module monitors data access requests from the cloud, edge, and terminal in real time, including parameters such as request time and data volume.

[0037] The data storage module stores the collected data, set parameters, and real-time monitoring information.

[0038] The calculation module is responsible for key data such as calculating the elastic equilibrium point, constructing the probability density function, obtaining random factors, calculating the inverse function, and calculating the mathematical expectation.

[0039] The algorithm generation module generates a randomized online cloud-edge-device collaborative data storage algorithm to determine whether the current data should be stored in the cloud or at the edge.

[0040] The collaborative scheduling module migrates data stored in the cloud to the edge, or vice versa.

[0041] The log module records information such as end-user resource requests and data acquisition, and also records the migration of data between the cloud and the edge.

[0042] Another embodiment of this application provides a storage medium for storing computer instructions, which, when executed by a processor, complete a data storage method for random online cloud-edge-device collaboration.

[0043] Another embodiment of this application provides a terminal device including a memory, a processor, and a computer program, wherein the memory stores the computer program, and the processor is configured to run the computer program to implement the method described in any of the preceding claims.

[0044] Compared with existing technologies, this invention can make reasonable decisions in real time about whether data should be stored at the edge or in the cloud, minimize the deployment cost of cloud storage services, improve the deployment efficiency of cloud storage services, and ensure QoS for users when using cloud storage services. A stochastic online cloud-edge-device collaborative data storage algorithm is proposed. By combining this algorithm with probability distribution and mathematical expectation, the deployment cost of cloud storage services can be minimized. The random factors generated by the algorithm can make more reasonable decisions in real time about whether data should be stored at the edge or in the cloud. This paper introduces the concept of a "sliding window" to achieve real-time monitoring of data stored in the cloud and at the edge. It supports customizable sliding window sizes, adaptively adjusting the perception range of stored data at both the cloud and edge. By combining a stochastic online cloud-edge collaborative data storage algorithm with the sliding window, the accuracy and flexibility of the algorithm's predictions are effectively improved, easily adapting to different production environment requirements and enhancing the deployment efficiency of cloud storage services. The paper also proposes the concept of "elastic break-even points." Typically, break-even points are fixed values ​​calculated based on an equation, simplifying the research model. However, in applications sensitive to latency, designing a break-even point with a flexible range allows for more rational decisions on whether data should be stored at the edge or in the cloud, significantly reducing data storage costs while ensuring data storage performance. Finally, a multi-level queue coordination mechanism is introduced. By setting up multi-level queues, priority caching of data stored in the cloud and at the edge is implemented based on the principle of locality, resulting in greater stability during data access and use, thus fully guaranteeing QoS for users using cloud storage services. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the random online cloud-edge-device collaborative data storage method provided in Embodiment 1 of the present invention;

[0046] Figure 2 This is a schematic diagram of the random online cloud-edge-device collaborative data storage algorithm provided in Embodiment 1 of the present invention;

[0047] Figure 3 This is a structural diagram of a randomized online cloud-edge-device collaborative data storage system provided in Embodiment 2 of the present invention. Detailed Implementation

[0048] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0049] This invention first provides a method for correcting errors in edited information based on a rich text editor. This method can be applied to terminal devices, such as computer terminals, specifically ordinary computers, quantum computers, etc.

[0050] The following detailed explanation uses a computer terminal as an example.

[0051] Example 1:

[0052] Figure 1 A random online cloud-edge-device collaborative data storage method provided in this embodiment of the invention includes the following steps:

[0053] Step 1: Select the research object. Within a region, randomly select an availability zone (AZ) as the object, and count the number n of edge nodes in that availability zone AZ.

[0054] Step 2: Set the relevant parameters. Let b center The amount of data stored in the cloud makes Let p be the amount of data stored at the edge. center The unit price for storing data in the cloud, Let be the unit price of data stored at the edge, where i∈[1,n], let q be the unit price of data access, let t be the storage duration, and let T be the storage billing period. Assume that the data was stored in the cloud during the past storage period, and let its storage cost be C. center Assuming the data was stored at the edge during the previous storage cycle, let its storage cost be C. edge .

[0055] Step 3: Set the sliding window size and collect relevant data. Set the sliding window size z and monitor the data requests from terminal devices in real time. Record and count the number m of data access requests initiated by cloud users on the terminal within the sliding window area. Count the number of times these data are processed in the cloud and at the edge. The number of times data is processed in the cloud is denoted as h. center The number of times data processing is performed at the edge is denoted as h. edge .

[0056] Step 4: Calculate the elastic equilibrium point β using mathematical operations based on probability theory and inverse functions. e We construct a probability density function and obtain the random factor k, which serves as preliminary preparation for proposing a random online cloud-edge-device collaborative data storage algorithm.

[0057] Elastic equilibrium point β e :

[0058] Using the parameters set in steps 1-3, the cost of storing data in the cloud during the previous storage billing cycle T can be calculated as C. center =b center ·p center ·t+z·h center / T·q, the cost of storing data at the edge is

[0059] Furthermore, the storage cost C of cloud data center Storage cost C for edge data edge Establish a correlation and set the elastic interval s to [l, r]. Where l = min(z / T·h) center ,m), r=max(z / T·h edge ,m), and then calculate the elastic equilibrium point β. e .

[0060]

[0061] Probability density function:

[0062] Combining the data storage cost calculation formula and the set relevant parameters, and the calculated elastic equilibrium point β e ,make Construct the following probability density function.

[0063]

[0064] Where e is the natural index, δ(·) is the Dirac delta function, and k is a random factor.

[0065] Random factor k:

[0066] By constructing the probability density function, we calculate its inverse function, and then obtain the random factor k through the inverse function. Let... Mathematical expectation of function f(y) The inverse function of f(k) yields the random factor k. Let F(k) = u, then the formula for calculating the random factor k is:

[0067]

[0068] Step 5: Random online cloud-edge-device collaborative data storage algorithm, the execution flow of which is as follows: Figure 2 As shown, it includes the following steps:

[0069] At the current time t0, count the number of times h that the data was accessed in the cloud and at the edge during the period from time t0-z+1 to time t0. center and h edge And thus calculate the storage cost of the data during this period.

[0070]

[0071] If the current data is stored at the edge, compare the storage cost of the data at the current time t0. And the size of the random factor k, if Therefore, storing data in the cloud would save more costs; if Storing the data at the edge would save even more costs.

[0072] If the current data is stored in the cloud, compare the storage cost w of the data at the current time t0. t0 And the size of the random factor k, if Therefore, storing data at the edge would save more costs; if Storing the data in the cloud would save even more costs.

[0073] A multi-level queue coordination mechanism is set up, creating n-level storage queues. At the current time t0, the storage costs of the data are compared. The size of the random factor k determines whether the data should be stored in the cloud or at the edge. For data that needs to be physically migrated, it is first stored in queue n based on the principle of locality. i The data is cached in the cache and then migrated.

[0074] Step 6: In the current storage period T, the random online cloud-edge-device collaborative data storage method has been executed. This method is repeated in the next storage period T'.

[0075] This embodiment describes in detail one implementation of the online hierarchical storage method for randomized hot and cold data in the cloud. When using the method described in this disclosure, users should not be limited to the method described in this embodiment, but can make appropriate adjustments according to their own business and actual situation.

[0076] Example 2:

[0077] See Figure 3 , Figure 3 This invention provides a randomized online cloud-edge-device collaborative data storage system, which includes seven modules: a data acquisition module, a real-time monitoring module, a data storage module, a computing module, an algorithm generation module, a collaborative scheduling module, and a log module.

[0078] Specifically, the data acquisition module can acquire resource request information from the cloud, edge, and terminal, and receive data access records connected to the server; the real-time monitoring module can monitor data access requests from the cloud, edge, and terminal in real time, including parameters such as request time and data volume; the data storage module is responsible for storing the acquired data, set parameters, and real-time monitoring information; the calculation module is mainly used to calculate key data such as the elastic-break-even-point, construct the probability density function, obtain random factors, calculate the inverse function, and calculate the mathematical expectation, providing support for generating a randomized online cloud-edge-terminal collaborative data storage algorithm; the algorithm generation module determines whether the current data should be stored in the cloud or the edge by generating a randomized online cloud-edge-terminal collaborative data storage algorithm; the collaborative scheduling module is responsible for migrating data stored in the cloud to the edge, or data stored in the edge to the cloud; and the log module is used to record information such as terminal user resource requests and data acquisition, and to record the migration of data on both the cloud and edge sides.

[0079] Example 3:

[0080] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform the method described in Embodiment 1.

[0081] It should be noted that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0082] More specifically, memory may include read-only memory and random access memory, providing instructions and data to the processor; a portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0083] In the implementation process, each step of the method in Embodiment 1 can be completed by the integrated logic circuit in the processor or by instructions in the form of software.

[0084] In further embodiments, the following is also provided:

[0085] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0086] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0087] The embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A randomized online cloud-edge-device collaborative data storage method, characterized in that, The method includes: Step 1: Start by selecting the research object. Within a geographical area, randomly select an availability zone as the object and count the number of edge nodes n in the availability zone AZ. Step 2: Set the relevant parameters, let b center The amount of data stored in the cloud makes Let p be the amount of data stored at the edge. center The unit price for storing data in the cloud, Let be the unit price of data stored at the edge, where i∈[1,n], let q be the unit price of data access, let t be the storage duration, and let T be the storage billing period; assuming that the data was stored in the cloud in the past storage period, let its storage cost be C. center Assuming the data was stored at the edge during the previous storage cycle, let its storage cost be C. edge ; Step 3: Set the sliding window size and collect relevant data. Set the sliding window size z, monitor the data requests from terminal devices in real time, record and count the number m of data access requests initiated by cloud users on the terminal within the sliding window area, and count the number of times these data are processed in the cloud and at the edge. The number of times data is processed in the cloud is denoted as h. center The number of times data processing is performed at the edge is denoted as h. edge ; Step 4: Calculate the elastic equilibrium point β e Construct the probability density function and obtain the random factor k; The calculation of the elastic equilibrium point β e This is a preliminary step in proposing a stochastic online cloud-edge-device collaborative data storage algorithm, where the elastic equilibrium point β... e Using the parameters set in steps 1-3, the cost of storing data in the cloud during the previous storage billing cycle T is calculated to be C. center =b center ·p center ·t+z·h center / T·q, the cost of storing data at the edge is The calculation of the elastic equilibrium point β e The storage cost of cloud data for Lieutenant General C center The storage cost C for edge data edge Establish the association and set the elastic interval s as [l, r], where l = min(z / T·h) center ,m), r=max(z / T·h edge ,m); The elastic equilibrium point β e Calculations show that: Step 5: Design a randomized online cloud-edge-device collaborative data storage algorithm; Step 6: Determine whether the current data needs to be migrated using the aforementioned machine-based online cloud-edge-device collaborative data storage algorithm.

2. The data storage method for random online cloud-edge-device collaboration according to claim 1, characterized in that, The method further includes: If the current data needs to be migrated, the data stored at the edge will be migrated to the cloud, or the data in the cloud will be migrated to the edge. In the current storage period T, the random online cloud-edge collaborative data storage method is completed. In the next storage period T', the method is repeated. If the current data does not need to be migrated, the process will end directly.

3. The data storage method for random online cloud-edge-device collaboration according to claim 1, characterized in that, In step 4, the constructed probability density function is obtained by combining the calculation formula for data storage cost with the set relevant parameters and the calculated elastic equilibrium point β. e Among them, The constructed probability density function is as follows: Where e is the natural index, δ(·) is the Dirac delta function, and k is a random factor.

4. The data storage method for random online cloud-edge-device collaboration according to claim 3, characterized in that, The constructed probability density function is used to obtain the random factor k by calculating its inverse function, where let Mathematical expectation of function f(y) The inverse function of f(k) is used to obtain the random factor k, and the formula for calculating the random factor k is as follows:

5. The data storage method for random online cloud-edge-device collaboration according to claim 1, characterized in that, In step 5, the design of the stochastic online cloud-edge-device collaborative data storage algorithm includes the following steps: At the current time t0, count the number of times h that the data was accessed in the cloud and at the edge during the period from time t0-z+1 to time t0. center and h edge And thus calculate the storage cost of the data during this period. That for If the current data is stored at the edge, compare the storage cost of the data at the current time t0. and the magnitude of the random factor k; If the current data is stored in the cloud, compare the storage costs of the data at the current time t0. and the magnitude of the random factor k; A multi-level queue coordination mechanism is set up, creating n-level storage queues. At the current time t0, the storage costs of the data are compared. The size of the random factor k determines whether the data should be stored in the cloud or at the edge. For data that needs to be physically migrated, it is first stored in queue n based on the principle of locality. i The data is cached in the cache and then migrated.

6. A terminal device, comprising: At least one processor, at least one memory, and at least one computer program, characterized in that the processor is connected to the memory, the memory stores the computer program, and the processor is configured to execute the computer program to implement the method of any one of claims 1 to 5.

Images