Multi-cloud storage processing method and device, equipment and storage medium

Abstract

The invention provides a multi-cloud storage processing method and device, equipment and a storage medium, and belongs to the technical field of multi-cloud storage. The method comprises the following steps: acquiring target data to be stored, and dividing the target data into a plurality of data segments; executing a multi-cloud storage segmentation online algorithm strategy, and determining respective target storage nodes of a plurality of data segments corresponding to the target data, including executing an allocation strategy and executing an allocation adjustment strategy; wherein the allocation strategy is used for determining respective target storage nodes of a plurality of data segments corresponding to the target data; the allocation adjustment strategy is used for adjusting target storage nodes of a plurality of data segments of the target data according to a storage balance point when any access request exists, and the storage balance point is used for representing a balance relationship between the access request frequency and the storage cost of the target data. The invention aims to reduce the storage cost.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of multi-cloud storage, and in particular, to a multi-cloud storage processing method, apparatus, device, and storage medium. Background Art

[0002] With the maturity and popularization of cloud computing technology, more and more companies and individuals have begun to store data in the cloud to obtain higher reliability, flexibility and scalability. Major companies have also begun to adopt a multi-cloud strategy to reduce data storage risks and improve data storage flexibility by storing data on multiple cloud platforms.

[0003] With the development of artificial intelligence and big data analysis technologies, multi-cloud storage has also begun to integrate intelligent functions to help users better manage and analyze stored data; multi-cloud storage strategies have many advantages. They can not only flexibly allocate storage resources among different cloud storage service providers according to storage needs, better meet business needs and cope with sudden traffic, but also disperse storage loads on different cloud storage platforms, optimize data access speed and performance, and improve application responsiveness.

[0004] In the current digital construction, multi-cloud storage strategies have broad development space in terms of technological innovation, security control, intelligence and ecosystem construction. However, with the continuous surge in data storage demand, and the fact that cloud service providers usually charge based on storage usage, high storage costs are often incurred in multi-cloud environments. In addition, in a multi-cloud environment, cloud service providers may adjust data storage pricing strategies from time to time, which may lead to instability in storage costs and result in higher storage costs. Summary of the invention

[0005] The embodiments of the present application provide a multi-cloud storage processing method, apparatus, device, and storage medium, aiming to reduce storage costs.

[0006] In a first aspect, an embodiment of the present application provides a multi-cloud storage processing method, the method comprising: Acquire target data to be stored, and divide the target data into multiple data segments; Execute the multi-cloud storage segmentation online algorithm strategy to determine the target storage nodes of the multiple data segments corresponding to the target data, including executing the allocation strategy and executing the allocation adjustment strategy; Among them, the allocation strategy is used to determine the target storage nodes of the multiple data segments corresponding to the target data; the allocation adjustment strategy is used to adjust the target storage nodes of the multiple data segments of the target data according to the storage balance point when there is any access request, and the storage balance point is used to characterize the balance relationship between the number of access requests for the target data and the storage cost.

[0007] Optionally, obtaining target data to be stored and dividing the target data into multiple data segments includes: Obtain target data to be stored; The target data is divided into a plurality of data segments according to the total amount of the target data, current usage information of each storage node and storage characteristics of the target data.

[0008] Optionally, execute an allocation strategy, including: Obtain the current usage information of each storage node and the remaining storage capacity of each storage node; For any data segment of the target data, when the usage information of any storage node is characterized as being in an idle state and the remaining storage capacity of the storage node is greater than or equal to the data capacity of the data segment, the storage node is used as the target storage node of the data segment; When the usage information of any storage node is characterized as a saturated state, or the remaining storage capacity of the storage node is less than the data capacity of the data segment, the next storage node is traversed until the target storage node of the data segment is determined.

[0009] Optionally, execute an allocation adjustment strategy, including: Acquire the remaining storage capacity of each storage node at the current moment in real time, and sort each storage node according to the size of the remaining storage capacity and add it to the storage node queue; When there is any access request, target storage nodes of multiple data segments of the target data are adjusted in the storage node queue according to the number of access requests for the target data and the storage balance point.

[0010] Optionally, adjusting the target storage nodes of the plurality of data segments of the target data in the storage node queue according to the number of access requests for the target data and the storage balance point includes: When the number of access requests for the target data is less than or equal to the storage balance point, any data segment of the target data is sequentially stored in the storage nodes in the storage node queue; Among them, when the remaining storage capacity of the current storage node in the storage node queue is less than or equal to the data capacity of the current data segment of the target data, the data to be migrated in the current storage node is determined according to the dynamic elastic window, and the data to be migrated is migrated to the storage node at the tail of the storage node queue, until the remaining storage capacity of the current storage node is greater than the data capacity of the current data segment, and the current storage node is used as the target storage node of the current data segment; When the number of access requests to the target data is greater than the storage balance point, the multi-cloud storage segmentation online algorithm strategy is re-executed on the target data.

[0011] Optionally, executing the allocation adjustment strategy further includes: Calculating the storage balance point; The calculation formula of the storage balance point is:

[0012] Wherein, Q is the target data; is the data capacity of the j-th data segment of the target data; is the storage unit price of storage node i; The storage duration of the target data; is the unit price of access request processing cost; n is the number of storage nodes; m is the number of data segments of the target data.

[0013] Optionally, executing the allocation adjustment strategy further includes: Calculating the size of the dynamic elastic window, where the size of the dynamic elastic window represents the amount of data stored per unit time; The calculation formula for the size of the dynamic elastic window is:

[0014] in, is the size of the dynamic elastic window; T is the previous storage cycle; The number of access requests for the target data; is the data capacity of the j-th data segment of the target data; Cost per unit for access request processing; is the sensitivity coefficient.

[0015] In a second aspect, an embodiment of the present application provides a multi-cloud storage processing device, the device comprising: An acquisition module, used for acquiring target data to be stored and dividing the target data into multiple data segments; A storage module, used to execute a multi-cloud storage segmentation online algorithm strategy, determine the target storage nodes of the multiple data segments corresponding to the target data, including executing an allocation strategy and executing an allocation adjustment strategy; Among them, the allocation strategy is used to determine the target storage nodes of the multiple data segments corresponding to the target data; the allocation adjustment strategy is used to adjust the target storage nodes of the multiple data segments of the target data according to the storage balance point when there is any access request, and the storage balance point is used to characterize the balance relationship between the number of access requests for the target data and the storage cost.

[0016] In a third aspect, an embodiment of the present application provides a computer device, comprising: at least one processor, and a memory, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the computer program, the multi-cloud storage processing method described in the first aspect of the embodiment is executed.

[0017] In a fourth aspect, an embodiment of the present application provides a non-volatile readable storage medium, wherein the non-volatile readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the multi-cloud storage processing method described in the first aspect of the embodiment is executed.

[0018] Beneficial effects: The target data to be stored is obtained, and the target data is divided into multiple data segments; a multi-cloud storage segmentation online algorithm strategy is executed to determine the target storage nodes of the multiple data segments corresponding to the target data, including executing an allocation strategy and executing an allocation adjustment strategy; wherein the allocation strategy is used to determine the target storage nodes of the multiple data segments corresponding to the target data; the allocation adjustment strategy is used to adjust the target storage nodes of the multiple data segments of the target data according to a storage balance point when there is any access request, and the storage balance point is used to characterize the balance relationship between the number of access requests for the target data and the storage cost.

[0019] The method provided in this embodiment divides the target data into multiple data segments according to the multi-cloud storage segmentation online algorithm when storing the target data, and then stores the multiple data segments in corresponding target storage nodes respectively. When the stored target data is accessed later, the target storage node of each data segment can be dynamically adjusted according to the number of access requests and the storage balance point, thereby reducing the data storage cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 is a flowchart of the steps of a multi-cloud storage processing method proposed in one embodiment of the present application; Figure 2 is a flowchart of the steps of executing the allocation strategy proposed in one embodiment of the present application; Figure 3 is a flowchart of the steps of executing the allocation adjustment strategy provided by an embodiment of the present application; Figure 4is an execution flow chart of a multi-cloud storage processing method proposed in an embodiment of the present application; Figure 5 It is a functional module diagram of a multi-cloud storage processing device proposed in one embodiment of the present application; Figure 6 is a schematic diagram of a computer device provided by an embodiment of the present application; Figure 7 It is a schematic diagram of a non-volatile readable storage medium proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] Segmented online algorithm: It is an algorithm that can respond to the dynamic changes of data streams in real time when processing service requests. When the data distribution changes suddenly or new data patterns appear, it can effectively deal with the situation where different characteristic segments appear in the data stream. Segmented online algorithm can make real-time analysis and effective decisions in many fields such as industrial production optimization and intelligent transportation systems where data streams change dynamically.

[0024] Multi-cloud storage: refers to cloud tenants storing their data and resources in multiple cloud service providers. Multi-cloud storage allows users to flexibly allocate storage resources between different cloud service providers based on storage needs, thereby better meeting business needs; by storing data on different cloud platforms, data backup and disaster recovery can be achieved.

[0025] Storage balance point: refers to the critical point calculated for the storage cost of different feature segments in the process of using online algorithms to process data streams. When calculating the storage balance point, it is necessary to take into account the response speed of the online algorithm and the accuracy of the processing results to meet the storage requirements of different application scenarios. The determination of the balance point also needs to comprehensively consider factors such as the characteristics of the data stream, system resource usage, and algorithm complexity.

[0026] Dynamic sliding window: It is a commonly used technical means in processing data streams. It differs from traditional sliding windows in that the dynamic sliding window can dynamically adjust the size or position of the window according to the characteristics of the data stream to adapt to the real-time changes of the data stream. It is often used in real-time data analysis, pattern recognition, and streaming computing. It can effectively process constantly changing data streams and extract useful information or features.

[0027] With the maturity and popularization of cloud computing technology, more and more companies and individuals have begun to store data in the cloud to obtain higher reliability, flexibility and scalability. Major companies have also begun to adopt a multi-cloud strategy to reduce data storage risks and improve data storage flexibility by storing data on multiple cloud platforms. With the development of artificial intelligence and big data analysis technology, multi-cloud storage has also begun to integrate intelligent functions to help users better manage and analyze stored data.

[0028] A multi-cloud storage strategy has many advantages. It can not only flexibly allocate storage resources among different cloud storage service providers according to storage needs to better meet business needs and cope with sudden traffic, but also optimize data access speed and performance and improve application responsiveness by dispersing storage loads on different cloud storage platforms.

[0029] In the current digital construction, multi-cloud storage strategies have broad room for development in terms of technological innovation, security control, intelligence, and ecosystem construction. However, with the continuous surge in data storage demand, and the fact that cloud service providers usually charge based on storage usage, high storage costs are often incurred in multi-cloud environments.

[0030] By optimizing multi-cloud storage costs, enterprises can not only reduce operating costs and use storage resources more effectively, but also maintain their advantages in market competition and achieve long-term sustainable development. Therefore, how to further optimize storage costs in a multi-cloud environment is an issue worthy of in-depth study.

[0031] In a multi-cloud environment, cloud service providers may adjust data storage pricing strategies from time to time, which may lead to instability in storage costs and unnecessary waste of storage resources.

[0032] Therefore, the embodiments of the present application provide a multi-cloud storage processing method, which can meet the storage needs of users while reducing storage costs.

[0033] Reference Figure 1 , shows a flowchart of a multi-cloud storage processing method in an embodiment of the present application, the method is applied to the user side, and the method may specifically include the following steps: S101: Acquire target data to be stored, and divide the target data into multiple data segments.

[0034] In the actual implementation process, users will store any data in the multi-cloud storage service according to the actual storage business needs. The multi-cloud storage service includes multiple storage nodes provided by different cloud service providers. Moreover, different storage nodes provided by the same cloud service provider may have different storage costs due to the different performance of each storage node. Because this method is based on a segmented online algorithm, which is a commonly used algorithm in data stream processing and is mainly used to process dynamically changing data sets, the algorithm divides the data stream into segments and calculates each segment of data independently, thereby achieving real-time processing and analysis of the data stream.

[0035] Therefore, when any user stores target data on a multi-cloud storage service, the target data sequence must first be divided into multiple data segments corresponding to the target data. For example, the target data is recorded as , the target data Divided into m data segments, any data segment is recorded as , j∈[1,m].

[0036] In a feasible implementation, in the process of obtaining the target data to be stored and dividing the target data into multiple data segments, the target data to be stored is first obtained, and then the target data is divided into multiple data segments according to the total amount of data of the target data, the current usage information of each storage node and the storage characteristics of the target data.

[0037] The usage information corresponding to each storage node represents the storage status of the storage node. For example, the i-th storage node among the n storage nodes in the multi-cloud storage service is recorded as , any storage node The usage information is recorded as ,when =1, represents the storage node In idle state, normal storage process can be performed; when =0, indicating that the storage node It is in a saturated state and cannot perform normal storage processes.

[0038] The storage characteristics of any data include the access frequency of the data, etc.

[0039] In the actual implementation process, a comprehensive analysis can be performed based on the total amount of target data, current usage information of each storage node, and storage characteristics of the target data to try to divide the target data into multiple data segments.

[0040] The specific division rules for the target data can be customized according to the actual storage requirements. For example, a division rule can be preset as follows: determine the total amount of data, the current usage information of each storage node, and the division weights of the storage characteristics of the data, and then determine the number of data segments corresponding to the division according to the range of the weighted results. In the actual implementation process, other division rules can also be used, which are not limited in this embodiment.

[0041] S102: Execute a multi-cloud storage segmentation online algorithm strategy to determine target storage nodes for each of the multiple data segments corresponding to the target data, including executing an allocation strategy and executing an allocation adjustment strategy.

[0042] The allocation strategy is used to determine the target storage nodes of the multiple data segments corresponding to the target data.

[0043] The allocation adjustment strategy is used to adjust the target storage nodes of multiple data segments of the target data according to the storage balance point when there is any access request. The storage balance point is used to characterize the balance relationship between the number of access requests for the target data and the storage cost.

[0044] Reference Figure 2 , shows a flowchart of the steps of executing the allocation strategy provided by an embodiment of the present application. In a feasible implementation, the process of executing the allocation strategy may include the following steps: A1: Obtain the current usage information of each storage node and the remaining storage capacity of each storage node.

[0045] Specifically, at any time, the usage information of each storage node in the previous storage cycle can be collected, and the remaining storage capacity of each storage node can be recorded respectively. The remaining storage capacity is .

[0046] The storage period is configured by the user. For example, the storage period may be one month or one year, etc., and may be defined according to the needs in the actual implementation process.

[0047] A2: For any data segment of the target data, when the usage information of any storage node is characterized as an idle state and the remaining storage capacity of the storage node is greater than or equal to the data capacity of the data segment, the storage node is used as the target storage node of the data segment.

[0048] After collecting and counting the usage information of each storage in the previous storage cycle and recording the remaining storage capacity of each storage node respectively, it is possible to determine whether any storage node is in an idle state and the remaining storage capacity of the storage node in an idle state, thereby determining whether any data segment of the target data can be stored in the storage node.

[0049] In the actual implementation process, since the target data corresponds to multiple data segments, each data segment can be traversed in sequence and the target storage node corresponding to the data segment can be determined.

[0050] For any data segment of the target node, the usage information and remaining storage capacity of each node are traversed. When a storage node is in an idle state, it indicates that the normal storage process can be carried out, and when the remaining storage capacity of the storage node is greater than or equal to the data capacity of the data segment, it indicates that the data segment can be stored in the storage node, then the storage node is used as the target storage node for the data segment.

[0051] A3: When the usage information of any storage node is characterized as saturated, or the remaining storage capacity of the storage node is less than the data capacity of the data segment, traverse the next storage node until the target storage node of the data segment is determined.

[0052] For any data segment of the target node, when traversing the usage information and remaining storage capacity of each node, when a storage node is in a saturated state, it means that the storage node cannot perform normal storage process, then skip the storage node and continue to determine whether the next storage node can be used as the target storage node for the data segment.

[0053] Alternatively, when a storage node is in an idle state, that is, the storage node can perform normal storage processes, but the remaining storage capacity of the storage node is less than the data capacity of the data segment, the data segment cannot be stored in the storage node, then the storage node is skipped and it is continued to determine whether the next storage node can be used as the target storage node for the data segment.

[0054] When a storage node is found to be idle and the remaining storage capacity of the storage node is greater than or equal to the data capacity of the data segment, the storage node is used as the target storage node of the data segment, and then the target storage node of the next data segment of the target data is determined.

[0055] When all data segments of the target data have determined the corresponding storage nodes, each data segment is stored in the target storage node, and the remaining storage capacity updated after each target storage node stores the corresponding data segment is recorded, so that the new target data to be stored is initially stored in the multi-cloud storage.

[0056] After the allocation strategy determines the target storage nodes for preliminary storage of all data segments of the target data, the storage node of any data segment is not fixed. In order to reduce the storage cost of data, the method provided in this embodiment also executes an allocation adjustment strategy. When the target data is subsequently accessed, the storage position of any data segment is adjusted according to the storage balance point to minimize the storage cost of the data.

[0057] Reference Figure 3, shows a flowchart of the steps of executing the allocation adjustment strategy provided by an embodiment of the present application. In a feasible implementation, the process of executing the allocation adjustment strategy may include the following steps: B1: The remaining storage capacity of each storage node at the current moment is obtained in real time, and each storage node is sorted according to the size of the remaining storage capacity and added to the storage node queue.

[0058] The remaining storage capacity of each storage node is updated in real time, and then all storage nodes are sorted in ascending order, and all sorted storage nodes are added to the storage node queue.

[0059] In multi-cloud storage, it is usually divided into hot storage layer, warm storage layer and cold storage layer. The storage nodes in the hot storage layer are usually nodes with fast storage speed but small storage capacity, which can quickly respond to the user's access request for any data; the cold storage layer is usually nodes with low storage speed but large storage capacity.

[0060] Therefore, after sorting by remaining storage capacity, the nodes at the head of the storage node queue are mostly nodes of the hot storage layer, and the nodes at the tail of the storage node queue are mostly nodes of the cold storage layer with larger storage capacity.

[0061] B2: When there is any access request, according to the number of access requests for the target data and the storage balance point, the target storage nodes of the multiple data segments of the target data are adjusted in the storage node queue.

[0062] After a target data is stored, the user may make an access request to the stored target data. The method may adjust the storage locations of multiple data segments of the target data according to the access to the target data.

[0063] During actual implementation, the storage balance point is calculated.

[0064] According to n storage nodes, set d storage balance points , k∈[1,d], n=d+1.

[0065] In the storage of multiple data segments of the target data, the total storage cost Z of the target data includes the data storage cost X and the data processing cost Y as well as other additional costs. The total storage cost Z is calculated as:

[0066] Specifically:

[0067] Where Z is the total storage cost; is the data capacity of the j-th data segment of the target data; is the storage unit price of storage node i; The storage duration of the target data; is the number of access requests; is the unit price of access request processing cost; n is the number of storage nodes; m is the number of data segments of the target data; For additional fees.

[0068] The additional fee includes other storage fees in the multi-cloud storage. For example, the additional fee includes a storage discount and the like.

[0069] Furthermore, according to the calculation formula of the total storage cost, the calculation formula of the storage balance point can be obtained as follows:

[0070] Wherein, Q is the target data; is the data capacity of the j-th data segment of the target data; is the storage unit price of storage node i; The storage duration of the target data; is the unit price of access request processing cost; n is the number of storage nodes; m is the number of data segments of the target data.

[0071] Then, according to the number of access requests for the target data and the storage balance point, the process of adjusting the target storage nodes of the multiple data segments of the target data in the storage node queue includes two cases.

[0072] The first case: when the number of access requests for the target data is less than or equal to the storage balance point, any data segment of the target data is sequentially stored in the storage nodes in the storage node queue.

[0073] That is, the number of access requests ≤ Storage balance point When the data segments of the target data are stored in the storage node queue, the target storage position of any data segment can be adjusted.

[0074] In the actual implementation process, for any data segment, each storage node in the storage node queue is traversed respectively.

[0075] When the remaining storage capacity of the current storage node in the storage node queue is less than or equal to the data capacity of the current data segment of the target data, the data to be migrated in the current storage node is determined according to the dynamic elastic window, and the data to be migrated is migrated to the storage node at the tail of the storage node queue.

[0076] Specifically, this embodiment also provides a formula for calculating the size of the dynamic elastic window, and the size of the dynamic elastic window represents the amount of data stored per unit time.

[0077] For example, starting from time t-T+1, the number of access requests in the previous storage cycle T is traversed, combined with the calculation formula of the data processing cost Y in the previous storage cycle T, that is:

[0078] The calculation formula for the size of the dynamic elastic window is:

[0079] Among them, among them, is the size of the dynamic elastic window; T is the previous storage cycle; The number of access requests for the target data; is the data capacity of the j-th data segment of the target data; is the data capacity of the j-th data segment of the target data; is the sensitivity coefficient.

[0080] The sensitivity system is the degree to which the dynamic elastic window changes according to the size of the storage load. The larger the storage load, the larger the window size. It can be set according to the needs of the actual application.

[0081] When the remaining storage capacity of the current storage node in the storage node queue is less than or equal to the data capacity of the current data segment of the target data, the data that has not been accessed in the previous storage cycle T can be used as the data to be migrated, and then the amount of data to be migrated is determined based on the size of the dynamic elastic window, and the data to be migrated is migrated to the storage node at the tail of the storage node queue.

[0082] Until the remaining storage capacity of the current storage node is greater than the data capacity of the current data segment, the current storage node is used as the target storage node of the current data segment.

[0083] In the actual implementation process, if the remaining storage capacity of the current storage node is still smaller than the current data segment after the data to be migrated is migrated to the storage node at the tail of the storage node queue through the dynamic elastic window, it is possible to traverse to the next storage node in the storage node queue to determine the adjusted target storage node corresponding to the data segment.

[0084] The dynamic elastic window can be used to adjust the number of access requests based on the current access request situation of the target data. ≤ Storage balance point When the accessed data is migrated to a storage node such as a cold storage layer, storage space is reserved for the data segment of the target data, and the size of the dynamic elastic window can be adjusted accordingly according to the access request of the current data segment of the target data.

[0085] Second case: when the number of access requests to the target data is greater than the storage balance point, the multi-cloud storage segmentation online algorithm strategy is re-executed for the target data, that is, the number of access requests Greater than storage balance point When , it indicates that the storage of the target data is unreasonable, which may result in a large storage cost. At this time, the target data will be re-divided into multiple data segments, and the multi-cloud storage segmentation online algorithm strategy will be executed again to determine the target storage node.

[0086] Reference Figure 4 , shows an execution flow chart of a multi-cloud storage processing method according to an embodiment of the present application. In a feasible implementation manner, the method may include the following steps: S1: Divide the acquired target data into multiple data segments.

[0087] For example, the target data is recorded as , the target data Divided into m data segments, any data segment is recorded as , the data capacity of any data segment j is , j∈[1,m].

[0088] S2: Execute the allocation strategy to allocate and store multiple data segments to their corresponding target storage nodes.

[0089] Collect the usage information of each storage node in the previous storage cycle T, and record the usage information of the i-th storage node respectively. Remaining storage capacity .

[0090] When the i-th storage node Remaining storage capacity Greater than or equal to the data capacity of any data segment j , and when =1, the storage node As the target storage node for the data segment; When the i-th storage node Remaining storage capacity Smaller than the data capacity of any data segment j , or when =1, traverse the next storage node.

[0091] S3: Calculates storage balance point and dynamic elastic window.

[0092] Specifically, the calculation formula of the storage balance point is:

[0093] Wherein, Q is the target data; is the data capacity of the j-th data segment of the target data; is the storage unit price of storage node i; The storage duration of the target data; is the unit price of access request processing cost; n is the number of storage nodes; m is the number of data segments of the target data.

[0094] The calculation formula for the size of the dynamic elastic window is:

[0095] Among them, among them, is the size of the dynamic elastic window; T is the previous storage cycle; The number of access requests for the target data; is the data capacity of the j-th data segment of the target data; is the data capacity of the j-th data segment of the target data; is the sensitivity coefficient.

[0096] S4: acquiring the remaining storage capacity of each storage node at the current moment in real time, and sorting each storage node according to the size of the remaining storage capacity and adding it to the storage node queue.

[0097] S5: When there is any access request, adjust the target storage node of the data segment according to the number of access requests for the target data and the storage balance point.

[0098] S6: Determine whether the number of access requests is less than or equal to the storage balance point.

[0099] If the number of access requests is less than or equal to the storage balance point, execute step S7; If the number of access requests is greater than the storage balance point, execute step S10; S7: Determine whether the remaining storage capacity of the current storage node is less than or equal to the data capacity of the current data segment.

[0100] If the remaining storage capacity of the current storage node is less than or equal to the data capacity of the current data segment, execute step S8; If the remaining storage capacity of the current storage node is greater than the data capacity of the current data segment, execute step S9.

[0101] S8: Determine the data to be migrated in the current storage node according to the dynamic elastic window, and migrate the data to be migrated to the storage node at the tail of the storage node queue.

[0102] S9: Store the current data segment in the current storage node.

[0103] Then the next data segment is taken as the current data segment and step S7 is continued.

[0104] S10: Re-divide the target data into multiple data segments and execute S2.

[0105] The method provided in this embodiment divides the target data into multiple data segments for storage and storage adjustment according to the multi-cloud storage segmentation online algorithm when storing the target data, which can not only quickly respond to changing storage needs, but also effectively reduce data storage costs and improve data storage performance and efficiency.

[0106] Then, multiple data segments are stored in corresponding target storage nodes respectively. When the stored target data is accessed later, the target storage node of each data segment can be dynamically adjusted according to the number of access requests and the storage balance point, thereby reducing the data storage cost.

[0107] In a multi-cloud environment, in order to meet storage needs while minimizing storage costs, this method can not only provide cloud users with an elastic, reliable and flexible multi-cloud storage service, but also effectively reduce the cost of multi-cloud storage for cloud users; by introducing a storage balance point and a dynamic sliding window, when a new access request arrives, the data storage in the multi-cloud environment is adjusted and scheduled from the perspective of reducing storage costs, so that cloud users can monitor the storage status of data in different cloud service providers in real time, and redistribute the stored resources on multiple cloud platforms according to the storage load.

[0108] Reference Figure 5 , shows a functional module diagram of a multi-cloud storage processing device provided in an embodiment of the present application, the device comprising: An acquisition module 100 is used to acquire target data to be stored and divide the target data into multiple data segments; The storage module 200 is used to execute the multi-cloud storage segmentation online algorithm strategy to determine the target storage nodes of the multiple data segments corresponding to the target data, including executing the allocation strategy and executing the allocation adjustment strategy; Among them, the allocation strategy is used to determine the target storage nodes of the multiple data segments corresponding to the target data; the allocation adjustment strategy is used to adjust the target storage nodes of the multiple data segments of the target data according to the storage balance point when there is any access request, and the storage balance point is used to characterize the balance relationship between the number of access requests for the target data and the storage cost.

[0109] Optionally, the acquisition module includes: An acquisition unit, used for acquiring target data to be stored; The division unit is used to divide the target data into multiple data segments according to the total amount of the target data, the current usage information of each storage node and the storage characteristics of the target data.

[0110] Optionally, the storage module includes a first storage unit, which is used to execute the allocation strategy, including: Obtain the current usage information of each storage node and the remaining storage capacity of each storage node; For any data segment of the target data, when the usage information of any storage node is characterized as being in an idle state and the remaining storage capacity of the storage node is greater than or equal to the data capacity of the data segment, the storage node is used as the target storage node of the data segment; When the usage information of any storage node is characterized as a saturated state, or the remaining storage capacity of the storage node is less than the data capacity of the data segment, the next storage node is traversed until the target storage node of the data segment is determined.

[0111] Optionally, the storage module includes a second storage unit, which is used to execute the allocation adjustment strategy, including: Acquire the remaining storage capacity of each storage node at the current moment in real time, and sort each storage node according to the size of the remaining storage capacity and add it to the storage node queue; When there is any access request, target storage nodes of multiple data segments of the target data are adjusted in the storage node queue according to the number of access requests for the target data and the storage balance point.

[0112] Optionally, the second storage unit is further used for: When the number of access requests for the target data is less than or equal to the storage balance point, any data segment of the target data is sequentially stored in the storage nodes in the storage node queue; Among them, when the remaining storage capacity of the current storage node in the storage node queue is less than or equal to the data capacity of the current data segment of the target data, the data to be migrated in the current storage node is determined according to the dynamic elastic window, and the data to be migrated is migrated to the storage node at the tail of the storage node queue, until the remaining storage capacity of the current storage node is greater than the data capacity of the current data segment, and the current storage node is used as the target storage node of the current data segment; When the number of access requests to the target data is greater than the storage balance point, the multi-cloud storage segmentation online algorithm strategy is re-executed on the target data.

[0113] Reference Figure 6 , shows a schematic diagram of a computer device provided in an embodiment of the present application, including: at least one processor 601, and a memory 602, wherein the memory 602 stores a computer program that can be run on the processor 601, wherein the processor 601 executes the multi-cloud storage processing method described in the embodiment when executing the computer program.

[0114] Reference Figure 7 , shows a schematic diagram of a non-volatile readable storage medium provided in an embodiment of the present application, wherein the non-volatile readable storage medium 700 stores a computer program 701, wherein when the computer program 701 is executed by a processor, the multi-cloud storage processing method described in the embodiment is executed.

[0115] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0116] It should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, devices or computer program products. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0117] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0121] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0122] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A multi-cloud storage processing method, characterized in that: The method comprises: Acquire target data to be stored, and divide the target data into multiple data segments; Execute the multi-cloud storage segmentation online algorithm strategy to determine the target storage nodes of the multiple data segments corresponding to the target data, including executing the allocation strategy and executing the allocation adjustment strategy; Among them, the allocation strategy is used to determine the target storage nodes of the multiple data segments corresponding to the target data; the allocation adjustment strategy is used to adjust the target storage nodes of the multiple data segments of the target data according to the storage balance point when there is any access request, and the storage balance point is used to characterize the balance relationship between the number of access requests for the target data and the storage cost.

2. The method according to claim 1, characterized in that Obtain target data to be stored, and divide the target data into multiple data segments, including: Obtain target data to be stored; The target data is divided into a plurality of data segments according to the total amount of the target data, current usage information of each storage node and storage characteristics of the target data.

3. The method according to claim 1, characterized in that Execute allocation strategies, including: Obtain the current usage information of each storage node and the remaining storage capacity of each storage node; For any data segment of the target data, when the usage information of any storage node is characterized as being in an idle state and the remaining storage capacity of the storage node is greater than or equal to the data capacity of the data segment, the storage node is used as the target storage node of the data segment; When the usage information of any storage node is characterized as a saturated state, or the remaining storage capacity of the storage node is less than the data capacity of the data segment, the next storage node is traversed until the target storage node of the data segment is determined.

4. The method according to claim 3, characterized in that Execute allocation adjustment strategies, including: Acquire the remaining storage capacity of each storage node at the current moment in real time, and sort each storage node according to the size of the remaining storage capacity and add it to the storage node queue; When there is any access request, target storage nodes of multiple data segments of the target data are adjusted in the storage node queue according to the number of access requests for the target data and the storage balance point.

5. The method according to claim 3, characterized in that: Adjusting target storage nodes of multiple data segments of the target data in the storage node queue according to the number of access requests for the target data and the storage balance point includes: When the number of access requests for the target data is less than or equal to the storage balance point, any data segment of the target data is sequentially stored in the storage nodes in the storage node queue; Among them, when the remaining storage capacity of the current storage node in the storage node queue is less than or equal to the data capacity of the current data segment of the target data, the data to be migrated in the current storage node is determined according to the dynamic elastic window, and the data to be migrated is migrated to the storage node at the tail of the storage node queue, until the remaining storage capacity of the current storage node is greater than the data capacity of the current data segment, and the current storage node is used as the target storage node of the current data segment; When the number of access requests to the target data is greater than the storage balance point, the multi-cloud storage segmentation online algorithm strategy is re-executed on the target data.

6. The method according to claim 3, characterized in that Execute allocation adjustment strategies, including: Calculating the storage balance point; The calculation formula of the storage balance point is: Wherein, Q is the target data; is the data capacity of the j-th data segment of the target data; is the storage unit price of storage node i; The storage duration of the target data; is the unit price of access request processing cost; n is the number of storage nodes; m is the number of data segments of the target data.

7. The method according to claim 5, characterized in that Execute allocation adjustment strategies, including: Calculating the size of the dynamic elastic window, where the size of the dynamic elastic window represents the amount of data stored per unit time; The calculation formula for the size of the dynamic elastic window is: in, is the size of the dynamic elastic window; T is the previous storage cycle; The number of access requests for the target data; is the data capacity of the j-th data segment of the target data; Cost per unit for access request processing; is the sensitivity coefficient.

8. A multi-cloud storage processing device, characterized in that: The device comprises: An acquisition module, used for acquiring target data to be stored and dividing the target data into multiple data segments; A storage module, used to execute a multi-cloud storage segmentation online algorithm strategy, determine the target storage nodes of the multiple data segments corresponding to the target data, including executing an allocation strategy and executing an allocation adjustment strategy; Among them, the allocation strategy is used to determine the target storage nodes of the multiple data segments corresponding to the target data; the allocation adjustment strategy is used to adjust the target storage nodes of the multiple data segments of the target data according to the storage balance point when there is any access request, and the storage balance point is used to characterize the balance relationship between the number of access requests for the target data and the storage cost.

9. A computer device, characterized in that: include: At least one processor and a memory, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the computer program, the multi-cloud storage processing method according to any one of claims 1 to 7 is executed.

10. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the multi-cloud storage processing method according to any one of claims 1 to 7 is executed.

Images