Hybrid data center data security storage scheduling method and device

Abstract

The invention relates to a hybrid data center data secure storage scheduling method and device.The method comprises the steps that distributed data center nodes of a plurality of data blocks and verification data blocks corresponding to a storage target file are determined, the number of online nodes is determined, and whether the number of the online nodes is smaller than the number of the data blocks of the plurality of data blocks or not is judged; if the number of the online nodes is smaller than the number of the data blocks, sending a data block request to a centralized data center to recover a target file, and sending the target file to a target user; if not, whether the distributed data center nodes are all online is judged, if yes, the target file is obtained and sent to the target user, and if not, the target file is recovered according to the multiple verification data blocks and a preset erasure code mechanism, and the target file is sent to the target user. Therefore, the problem that in a scene with green energy as a main source, scheduling distribution of data storage between a centralized data center and a distributed data center cannot be effectively carried out in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the technical field of data storage scheduling, and in particular to a hybrid data center data security storage scheduling method and device. Background Art

[0002] With the advancement of digitalization, the scale of data centers continues to expand, and their functions and structures are becoming more and more diverse. In order to achieve low-carbon goals, data centers have also begun to use green energy for power supply. However, green energy power supply is often not as stable as traditional thermal power. With the complexity of computing needs, it has become a difficulty for existing technologies to enable data centers to efficiently complete data storage and computing tasks while relying mainly on green energy power supply.

[0003] Centralized and distributed data center architectures each have their own advantages and disadvantages. Centralized data centers are more centralized, and data storage and maintenance are relatively simple, but they also face the risk of single point failure, that is, if a single point in a centralized data center fails, the entire system may fail; distributed data centers can use multiple copies, tolerate multiple node failures, and have features such as automatic fault tolerance and failover. They are more suitable for large-scale systems that require reliability, but they are more complex in terms of data storage and operation, and require coordination and management of resources in different locations, making operation and maintenance more difficult.

[0004] In addition, as more and more distributed data center nodes are powered by green energy (such as hydropower, photovoltaics, and wind power), the instability of energy supply is significantly higher compared to traditional electricity. In the case of energy supply fluctuations, the operation of distributed data center nodes may become unstable, leading to potential node offline situations. Considering other failure situations such as server load fluctuations, the scheduling of computing tasks will also become more complex and difficult.

[0005] In summary, in scenarios dominated by green energy, existing technologies cannot effectively schedule and allocate data storage between centralized data centers and distributed data centers, which needs to be solved urgently. Summary of the invention

[0006] The present application provides a hybrid data center data security storage scheduling method and device to solve the problem that in a scenario dominated by green energy, the existing technology cannot effectively schedule and allocate data storage between a centralized data center and a distributed data center.

[0007] The first aspect of the present application provides a hybrid data center data security storage scheduling method, comprising the following steps: obtaining a file retrieval request from a target user, determining at least one distributed data center node that stores multiple data blocks and multiple verification data blocks corresponding to the target file according to the file retrieval request, and determining the number of online nodes in the at least one distributed data center node, and judging whether the number of online nodes is less than the number of data blocks of the multiple data blocks; if the number of online nodes is less than the number of data blocks, sending a data block request to a preset centralized data center through a preset IPFS protocol, so that the centralized data center restores the target file according to the data block request, and sends the target file to the target user; if the number of online nodes is greater than or equal to the number of data blocks, judging whether the at least one distributed data center node is all online, wherein, if the at least one distributed data center node is all online, obtaining the target file to send to the target user, otherwise restoring the target file according to the multiple verification data blocks and a preset erasure code mechanism, and sending the target file to the target user.

[0008] Optionally, in one embodiment of the present application, before determining the at least one distributed data center node that stores the multiple data blocks and the multiple verification data blocks corresponding to the target file, it also includes: splitting the target file into the multiple data blocks, and processing the multiple data blocks through the erasure code mechanism to obtain the multiple verification data blocks, and backing up the multiple data blocks in the centralized data center; based on the supply status of each of the preset multiple green energies, determining the data storage ratio of the distributed data center nodes corresponding to each of the preset multiple green energies; storing the multiple data blocks and the multiple verification data blocks in the distributed data center nodes corresponding to the preset distributed data center according to the data storage ratio.

[0009] Optionally, in one embodiment of the present application, before splitting the target file into the multiple data blocks, it also includes: establishing multiple data nodes that adopt the multiple green energy sources, and constructing the distributed data center according to the multiple data nodes and the IPFS protocol, wherein the multiple green energy sources include wind energy, solar energy, hydropower and pumped storage.

[0010] Optionally, in one embodiment of the present application, sending a data block request to a preset centralized data center through a preset IPFS protocol so that the centralized data center restores the target file according to the data block request, includes: determining at least one offline data block based on the number of online nodes and the number of data blocks, and sending the data block request to the centralized data center according to the at least one offline data block and the IPFS protocol; in response to the data block request, obtaining at least one offline data block among the multiple data blocks pre-backed up by the centralized data center, and combining the data blocks stored in the online distributed data center nodes among the at least one distributed data center node to restore the target file.

[0011] The second aspect of the present application provides a hybrid data center data security storage scheduling device, including: a judgment module, used to obtain a file retrieval request from a target user, to determine at least one distributed data center node that stores multiple data blocks and multiple verification data blocks corresponding to the target file according to the file retrieval request, and to determine the number of online nodes in the at least one distributed data center node, and to determine whether the number of online nodes is less than the number of data blocks of the multiple data blocks; a first recovery module, used to send a data block request to a preset centralized data center through a preset IPFS protocol if the number of online nodes is less than the number of data blocks, so that the centralized data center recovers the target file according to the data block request, and sends the target file to the target user; a second recovery module, used to determine whether the at least one distributed data center node is all online if the number of online nodes is greater than or equal to the number of data blocks, wherein if the at least one distributed data center node is all online, the target file is obtained to be sent to the target user, otherwise the target file is recovered according to the multiple verification data blocks and a preset erasure code mechanism, and the target file is sent to the target user.

[0012] Optionally, in one embodiment of the present application, it also includes: a backup module, used to split the target file into the multiple data blocks before determining the at least one distributed data center node that stores the multiple data blocks and the multiple verification data blocks corresponding to the target file, and process the multiple data blocks through the erasure code mechanism to obtain the multiple verification data blocks, and back up the multiple data blocks in the centralized data center; a proportion division module, used to determine the data storage ratio of the distributed data center nodes corresponding to each of the preset multiple green energy sources based on the supply status of each of the preset multiple green energy sources; a storage module, used to store the multiple data blocks and the multiple verification data blocks in the distributed data center nodes corresponding to the preset distributed data center according to the data storage ratio.

[0013] Optionally, in one embodiment of the present application, it also includes: an establishment module, used to establish multiple data nodes that adopt the multiple green energy sources before splitting the target file into the multiple data blocks, and construct the distributed data center according to the multiple data nodes and the IPFS protocol, wherein the multiple green energy sources include wind energy, solar energy, hydropower and pumped storage.

[0014] Optionally, in one embodiment of the present application, the first recovery module includes: a determination unit, used to determine at least one offline data block based on the number of online nodes and the number of data blocks, and send the data block request to the centralized data center according to the at least one offline data block and the IPFS protocol; a combining unit, used to obtain, in response to the data block request, at least one offline data block among the multiple data blocks pre-backed up by the centralized data center, and combine with the data blocks stored in the online distributed data center nodes among the at least one distributed data center node to restore the target file.

[0015] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hybrid data center data security storage scheduling method as described in the above embodiment.

[0016] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above hybrid data center data security storage scheduling method.

[0017] The fifth aspect of the present application provides a computer program product, including a computer program, which is executed to implement the above-mentioned hybrid data center data security storage scheduling method.

[0018] Therefore, the embodiments of the present application have the following beneficial effects:

[0019] The embodiment of the present application can obtain the file retrieval request of the target user, so as to determine at least one distributed data center node storing multiple data blocks and multiple verification data blocks corresponding to the target file according to the file retrieval request, and determine the number of online nodes in at least one distributed data center node, and judge whether the number of online nodes is less than the number of data blocks of multiple data blocks; if the number of online nodes is less than the number of data blocks, a data block request is sent to a preset centralized data center through a preset IPFS protocol, so that the centralized data center recovers the target file according to the data block request, and sends the target file to the target user; if the number of online nodes is greater than or equal to the number of data blocks, it is judged whether at least one distributed data center node is online, wherein if at least one distributed data center node is online, the target file is obtained to send to the target user, otherwise the target file is recovered according to multiple verification data blocks and a preset erasure code mechanism, and the target file is sent to the target user. The present application supplies data center energy based on the distribution of green energy, combines the characteristics of the two data center architectures of "centralized" and "distributed", and makes full use of green energy while ensuring the reliable completion of data storage tasks. Thus, the problem that the existing technology cannot effectively perform the scheduling and allocation of data storage between centralized data centers and distributed data centers in the scenario dominated by green energy is solved.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a hybrid data center data security storage scheduling method provided according to an embodiment of the present application;

[0023] Figure 2 A schematic diagram of a hybrid data center framework provided for one embodiment of the present application;

[0024] Figure 3 A schematic diagram of the logical architecture of a hybrid data center data security storage scheduling method provided for one embodiment of the present application;

[0025] Figure 4 This is an example diagram of a hybrid data center data security storage scheduling device according to an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0027] Among them, 10-hybrid data center data security storage scheduling device, 100-judgment module, 200-first recovery module, 300-second recovery module, 501-memory, 502-processor, 503-communication interface. DETAILED DESCRIPTION

[0028] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0029] The hybrid data center data security storage scheduling method and device of the embodiment of the present application are described below with reference to the accompanying drawings. In view of the problems mentioned in the above background technology, the present application provides a hybrid data center data security storage scheduling method, in which a file retrieval request of a target user is obtained to determine at least one distributed data center node storing multiple data blocks and multiple verification data blocks corresponding to the target file according to the file retrieval request, and the number of online nodes in at least one distributed data center node is determined, and it is judged whether the number of online nodes is less than the number of data blocks of the multiple data blocks; if the number of online nodes is less than the number of data blocks, a data block request is sent to a preset centralized data center through a preset IPFS protocol, so that the centralized data center restores the target file according to the data block request, and sends the target file to the target user; if the number of online nodes is greater than or equal to the number of data blocks, it is judged whether at least one distributed data center node is online, wherein if at least one distributed data center node is online, the target file is obtained to be sent to the target user, otherwise the target file is restored according to multiple verification data blocks and a preset erasure code mechanism, and the target file is sent to the target user. This application supplies energy to data centers based on the distribution of green energy, combines the characteristics of "centralized" and "distributed" data center architectures, and fully utilizes green energy while ensuring that data storage tasks are completed reliably. This solves the problem that the existing technology cannot effectively schedule and allocate data storage between centralized data centers and distributed data centers in scenarios where green energy is the main source.

[0030] Specifically, Figure 1 A flowchart of a hybrid data center data security storage scheduling method provided in an embodiment of the present application.

[0031] like Figure 1 As shown, the hybrid data center data security storage scheduling method includes the following steps:

[0032] In step S101, a file retrieval request of a target user is obtained to determine at least one distributed data center node storing multiple data blocks and multiple verification data blocks corresponding to the target file according to the file retrieval request, and to determine the number of online nodes in the at least one distributed data center node, and to determine whether the number of online nodes is less than the number of data blocks of the multiple data blocks.

[0033] The embodiments of the present application can first obtain the user's file retrieval request, and according to the file retrieval request, find and determine the distributed data center nodes where the n data blocks and c verification data blocks corresponding to the target file N are located, and determine the number m of online nodes in these distributed data center nodes, and determine whether the number of online nodes is less than the number of data blocks of the plurality of data blocks, thereby providing reliable guidance and basis for the subsequent retrieval and recovery of files.

[0034] Optionally, in one embodiment of the present application, before determining at least one distributed data center node for storing multiple data blocks and multiple verification data blocks corresponding to the target file, it also includes: splitting the target file into multiple data blocks, and processing the multiple data blocks through an erasure code mechanism to obtain multiple verification data blocks, and backing up the multiple data blocks in a centralized data center; based on the supply status of each of the preset multiple green energies, determining the data storage ratio of the distributed data center nodes corresponding to each green energy; storing the multiple data blocks and the multiple verification data blocks in the distributed data center nodes corresponding to the preset distributed data center according to the data storage ratio.

[0035] In the actual implementation process, the existing distributed data center nodes face problems such as unstable green energy supply. The embodiments of the present application can build a hybrid data center framework by combining the centralized architecture and the distributed architecture. Among them, the embodiments of the present application can deploy distributed nodes near geographical locations with relatively abundant green energy, make full use of green energy for power supply, and complete data storage and computing tasks; at the same time, take advantage of the computing and storage advantages of the centralized data center, give full play to the stability of the power supply energy of the centralized data center, so as to ensure that when a node fails due to unstable power supply, the data storage and computing tasks can still be effectively completed.

[0036] Optionally, in one embodiment of the present application, before splitting the target file into multiple data blocks, it also includes: establishing multiple data nodes that use multiple green energy sources, and building a distributed data center based on the multiple data nodes and the IPFS protocol, wherein the multiple green energy sources include wind energy, solar energy, hydropower and pumped storage.

[0037] Specifically, Figure 2As shown, the hybrid data center framework includes a centralized data center, a distributed data center based on the IPFS system, the Internet, traditional energy, and green energy supply. In addition, the hybrid data center framework also includes a data recovery mechanism.

[0038] The main structure of the distributed data center is composed of multiple nodes that work with green energy; in terms of data storage, the embodiments of the present application can build a distributed data center based on the IPFS protocol, and each distributed node can process the access of file data in accordance with the IPFS protocol. In terms of energy supply, the embodiments of the present application can use wind energy, hydropower, solar energy, etc. as energy sources for power supply according to the energy distribution, so as to make full use of local renewable energy, reduce carbon emissions, and achieve green development of hybrid data centers. In addition, since green energy is affected by factors such as weather and time, the supply is not stable, and some distributed nodes may occasionally go offline. Therefore, the embodiments of the present application can further construct a data recovery mechanism based on the mechanism of the IPFS system to ensure that data recovery is performed to the greatest extent possible when a distributed node is offline.

[0039] It should be noted that the hybrid data center framework includes a central data center (i.e., centralized data center), the central data node is powered by stable traditional energy, and a data transmission link is established between the centralized data center and the distributed data center; the centralized data center stores some data blocks of the source files in the distributed data center. When too many nodes in the distributed data center are offline and data failure is serious, the centralized data center will send back part of the database to the distributed data center, so that the file data can be effectively restored.

[0040] Among them, the source file data block in the distributed data node refers to a part of the database of the source file in the distributed data center, and these data blocks are backed up in the centralized data center. In the data center structure improved based on IPFS, files are stored in the form of blocks, and there is only one copy of the source file in each distributed node; but the online state of the distributed node is unstable, and a certain amount of data redundancy is required to cope with data failure and facilitate data recovery. Therefore, the embodiment of the present application can improve the data recovery mechanism by backing up a certain number of source file data blocks in a centralized data center.

[0041] It should be noted that in order to reduce data storage occupancy, the centralized data center only stores part of the source file's data blocks instead of all of it. These data blocks can be used by distributed data centers for data recovery.

[0042] To summarize, in a distributed data center, when a file is uploaded, it will be split into several data blocks, and after being processed by the erasure coding mechanism, it will be distributed and stored in several distributed nodes according to a certain strategy. Another part of the data blocks of the file will be uploaded to the centralized data center via network transmission for data backup, thereby ensuring the integrity of data storage to the greatest extent when a distributed node is offline.

[0043] Therefore, the embodiments of the present application determine the online status of nodes and allocate data storage by combining the energy supply conditions of distributed nodes, thereby fully utilizing green energy while avoiding data failure caused by node offline.

[0044] In order to achieve stable operation of the hybrid data center, the embodiments of the present application can classify the storage of data according to the energy supply status to avoid data failure in a short period of time due to node offline. The centralized data center mainly plays the role of centralized storage and calculation of data, and is connected to each distributed data center node through a data transmission link. The centralized data center node can centrally receive the data uploaded by each distributed node and play a role in data backup; the distributed data center utilizes green energy based on the geographical location and stores data based on the IPFS protocol.

[0045] It should be noted that the embodiments of the present application mainly consider four green energies, namely, "wind energy", "solar energy", "hydro energy" and "pumped storage" as energy sources for distributed data centers.

[0046] During the dry season, the amount of hydropower generation is low, and the data center powered by this type of energy cannot operate normally, and pumped storage is also reduced accordingly. In this case, the proportion of distributed node data stored in hydropower and pumped storage power supply is set to 0; at this time, only the allocation ratio of wind power, photovoltaic power and the allocation ratio of centralized data centers are considered; when the distributed system can work normally, about 30% of the data blocks will be stored in the centralized data center nodes.

[0047] During the flood season, the amount of hydropower generated is relatively high, and a larger proportion of the data storage tasks of the distributed data center are undertaken by the hydropower supply nodes; the pumped storage power station will also supply power to the distributed nodes, but considering the cost, the allocation ratio is not set too large; the embodiment of the present application can be set according to general climatic conditions and power plant deployment, to set the hydropower supply distributed nodes to store more than 50% of the data blocks, and the pumped storage function distributed nodes to store about 10% of the data blocks; on this basis, the allocation ratio of wind power, photovoltaic power and the allocation ratio of the centralized data center are considered.

[0048] Therefore, the embodiments of the present application coordinate the centralized nodes according to the green energy supply situation, and collaboratively perform data storage and computing tasks through centralized data center nodes and distributed data center nodes, forming an efficient and highly reliable green data center development model, and proportionally divides the energy supply conditions for data storage, thereby ensuring the high reliability of data storage and reducing the consumption of traditional energy by centralized data centers.

[0049] In step S102, if the number of online nodes is less than the number of data blocks, a data block request is sent to a preset centralized data center through the preset IPFS protocol, so that the centralized data center recovers the target file according to the data block request and sends the target file to the target user.

[0050] In step S103, if the number of online nodes is greater than or equal to the number of data blocks, it is determined whether at least one distributed data center node is online. If at least one distributed data center node is online, the target file is obtained to be sent to the target user. Otherwise, the target file is restored according to multiple verification data blocks and a preset erasure code mechanism, and the target file is sent to the target user.

[0051] Furthermore, if the number of online nodes m is less than the number of data blocks n, the embodiment of the present application can send a data block request to the centralized data center through the IPFS system of the distributed data center and restore the file N; otherwise, the embodiment of the present application can find out whether the distributed nodes where the n data blocks storing the file N are located are all online. If they are all online, the search for file N is completed, and the file N is transmitted to the user to complete the response to the user's data request. Otherwise, the existing m data blocks are used to complete the recovery of the file N according to the erasure code mechanism, and the file N is transmitted to the user.

[0052] Those skilled in the art should understand that the erasure coding mechanism is a fault-tolerant technology widely used in distributed data storage systems. In the hybrid data center framework, the erasure coding mechanism is combined with the distributed data center structure to ensure that file data can still be accessed even if a distributed node fails or the network is interrupted. When a certain file data needs to be accessed and downloaded by a user, the distributed data center will search the internal nodes to find the complete file data; if the distributed nodes are offline and the file data blocks cannot be fully obtained, the distributed data center system will first perform data recovery internally; if too many distributed nodes are offline and the distributed data center system cannot perform data recovery independently, the centralized data center will provide a certain number of file data block backups, such as Figure 3 As shown, the user's file access request is completed.

[0053] Optionally, in one embodiment of the present application, a data block request is sent to a preset centralized data center through a preset IPFS protocol, so that the centralized data center restores the target file according to the data block request, including: determining at least one offline data block based on the number of online nodes and the number of data blocks, and sending a data block request to the centralized data center according to the at least one offline data block and the IPFS protocol; in response to the data block request, obtaining at least one offline data block among multiple data blocks pre-backed up by the centralized data center, and combining the data blocks stored in the online distributed data center nodes in at least one distributed data center node to restore the target file.

[0054] In the specific implementation process, for the missing at least (nm) data blocks (i.e., offline data blocks) that can restore the complete file N, the distributed data center sends a request to the centralized data center. The centralized data center transmits these data blocks to the distributed data center IPFS system, and performs data recovery based on the erasure code mechanism to obtain the complete file N, and transmits file N to the user to complete the response to the user's data request.

[0055] Therefore, the embodiments of the present application improve the recovery capability of data storage and reduce the possibility of external attacks when the distributed data center nodes are offline by integrating the IPFS protocol, erasure coding mechanism and hybrid data center architecture.

[0056] To summarize, the embodiments of the present application use a hybrid structure data center framework to supply energy to the data center based on the distribution of green energy, and combine the characteristics of both "centralized" and "distributed" data center architectures to fully utilize green energy while ensuring that data storage tasks are reliably completed; the hybrid structure data center framework proposed in the embodiments of the present application adopts a data storage architecture and data recovery method based on the IPFS protocol to ensure that data can be safely stored even when distributed nodes are offline; in addition, taking into account the energy supply conditions of distributed data center nodes, the embodiments of the present application use a hybrid data center data storage and scheduling strategy based on green energy to allocate data storage ratios of different nodes under different energy conditions, so that when green energy power supply is unstable, the data center can fully utilize green energy while reliably completing data storage scheduling tasks.

[0057] According to the hybrid data center data security storage scheduling method proposed in the embodiment of the present application, by obtaining the file retrieval request of the target user, at least one distributed data center node storing multiple data blocks and multiple verification data blocks corresponding to the target file is determined according to the file retrieval request, and the number of online nodes in at least one distributed data center node is determined, and it is judged whether the number of online nodes is less than the number of data blocks of multiple data blocks; if the number of online nodes is less than the number of data blocks, a data block request is sent to a preset centralized data center through a preset IPFS protocol, so that the centralized data center recovers the target file according to the data block request, and sends the target file to the target user; if the number of online nodes is greater than or equal to the number of data blocks, it is judged whether at least one distributed data center node is online, wherein if at least one distributed data center node is online, the target file is obtained to be sent to the target user, otherwise the target file is recovered according to multiple verification data blocks and a preset erasure code mechanism, and the target file is sent to the target user. The present application supplies data center energy based on the distribution of green energy, combines the characteristics of the "centralized" and "distributed" data center architectures, and makes full use of green energy while ensuring the reliable completion of data storage tasks.

[0058] Secondly, a hybrid data center data security storage scheduling device proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0059] Figure 4 It is a block diagram of a hybrid data center data security storage scheduling device according to an embodiment of the present application.

[0060] like Figure 4 As shown, the hybrid data center data security storage scheduling device 10 includes: a judgment module 100, a first recovery module 200 and a second recovery module 300.

[0061] Among them, the judgment module 100 is used to obtain a file retrieval request from a target user, to determine at least one distributed data center node storing multiple data blocks and multiple verification data blocks corresponding to the target file according to the file retrieval request, and to determine the number of online nodes in at least one distributed data center node, and to determine whether the number of online nodes is less than the number of data blocks of the multiple data blocks.

[0062] The first recovery module 200 is used to send a data block request to a preset centralized data center through a preset IPFS protocol if the number of online nodes is less than the number of data blocks, so that the centralized data center recovers the target file according to the data block request and sends the target file to the target user.

[0063] The second recovery module 300 is used to determine whether at least one distributed data center node is online if the number of online nodes is greater than or equal to the number of data blocks. If at least one distributed data center node is online, the target file is obtained to be sent to the target user. Otherwise, the target file is restored according to multiple verification data blocks and a preset erasure code mechanism, and the target file is sent to the target user.

[0064] Optionally, in one embodiment of the present application, the hybrid data center data security storage scheduling device 10 of the embodiment of the present application further includes: a backup module, a proportional division module and a storage module.

[0065] Among them, the backup module is used to split the target file into multiple data blocks before determining at least one distributed data center node that stores multiple data blocks and multiple verification data blocks corresponding to the target file, and process the multiple data blocks through an erasure code mechanism to obtain multiple verification data blocks, and back up the multiple data blocks in a centralized data center.

[0066] The proportion division module is used to determine the data storage proportion of the distributed data center nodes corresponding to each green energy based on the supply status of each green energy among the preset multiple green energy sources.

[0067] The storage module is used to store multiple data blocks and multiple verification data blocks in distributed data center nodes corresponding to preset distributed data centers according to the data storage ratio.

[0068] Optionally, in one embodiment of the present application, it also includes: an establishment module for establishing multiple data nodes that use multiple green energy sources before splitting the target file into multiple data blocks, and building a distributed data center based on the multiple data nodes and the IPFS protocol, wherein the multiple green energy sources include wind energy, solar energy, hydropower and pumped storage.

[0069] Optionally, in one embodiment of the present application, the first recovery module 200 includes: a determination unit and a combination unit.

[0070] Among them, the determination unit is used to determine at least one offline data block based on the number of online nodes and the number of data blocks, and send a data block request to the centralized data center according to the at least one offline data block and the IPFS protocol.

[0071] The combining unit is used to obtain at least one offline data block from a plurality of data blocks pre-backed up by the centralized data center in response to a data block request, and combine the data blocks stored in an online distributed data center node in at least one distributed data center node to restore the target file.

[0072] It should be noted that the aforementioned explanation of the embodiment of the hybrid data center data security storage scheduling method is also applicable to the hybrid data center data security storage scheduling device of this embodiment, and will not be repeated here.

[0073] According to the hybrid data center data security storage scheduling device proposed in the embodiment of the present application, it includes a judgment module, which is used to obtain the file retrieval request of the target user, so as to determine at least one distributed data center node storing multiple data blocks and multiple verification data blocks corresponding to the target file according to the file retrieval request, and determine the number of online nodes in at least one distributed data center node, and judge whether the number of online nodes is less than the number of data blocks of multiple data blocks; a first recovery module, which is used to send a data block request to a preset centralized data center through a preset IPFS protocol if the number of online nodes is less than the number of data blocks, so that the centralized data center recovers the target file according to the data block request, and sends the target file to the target user; a second recovery module, which is used to determine whether at least one distributed data center node is online if the number of online nodes is greater than or equal to the number of data blocks, wherein if at least one distributed data center node is online, the target file is obtained to be sent to the target user, otherwise the target file is recovered according to multiple verification data blocks and a preset erasure code mechanism, and the target file is sent to the target user. The present application supplies data center energy based on the distribution of green energy, combines the characteristics of the "centralized" and "distributed" data center architectures, and makes full use of green energy while ensuring the reliable completion of data storage tasks.

[0074] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0075] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0076] When the processor 502 executes the program, the hybrid data center data security storage scheduling method provided in the above embodiment is implemented.

[0077] Furthermore, the electronic device further comprises:

[0078] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0079] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0080] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0081] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0082] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0083] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0084] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above hybrid data center data security storage scheduling method.

[0085] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned hybrid data center data security storage scheduling.

[0086] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0087] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0088] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0089] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0090] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0091] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0092] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0093] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A hybrid data center data security storage scheduling method, characterized in that: The following steps are involved: Obtaining a file retrieval request from a target user, determining at least one distributed data center node storing a plurality of data blocks and a plurality of check data blocks corresponding to the target file according to the file retrieval request, determining the number of online nodes in the at least one distributed data center node, and determining whether the number of online nodes is less than the number of data blocks of the plurality of data blocks; If the number of online nodes is less than the number of data blocks, a data block request is sent to a preset centralized data center through a preset IPFS protocol, so that the centralized data center restores the target file according to the data block request and sends the target file to the target user; If the number of online nodes is greater than or equal to the number of data blocks, determine whether the at least one distributed data center node is all online, wherein, if the at least one distributed data center node is all online, obtain the target file to send to the target user, otherwise restore the target file according to the multiple verification data blocks and the preset erasure code mechanism, and send the target file to the target user.

2. The method according to claim 1, characterized in that Before determining the at least one distributed data center node storing the multiple data blocks and the multiple verification data blocks corresponding to the target file, the method further includes: Splitting the target file into the multiple data blocks, processing the multiple data blocks by using the erasure coding mechanism to obtain the multiple verification data blocks, and backing up the multiple data blocks in the centralized data center; Based on the supply status of each of the preset multiple green energy sources, determine the data storage ratio of the distributed data center nodes corresponding to each of the green energy sources; The plurality of data blocks and the plurality of verification data blocks are stored in distributed data center nodes corresponding to a preset distributed data center according to the data storage ratio.

3. The method according to claim 2, characterized in that Before splitting the target file into the multiple data blocks, the method further includes: Establish multiple data nodes that adopt the multiple green energy sources, and construct the distributed data center according to the multiple data nodes and the IPFS protocol, wherein the multiple green energy sources include wind energy, solar energy, hydropower and pumped storage.

4. The method according to claim 1, characterized in that: The sending of a data block request to a preset centralized data center through a preset IPFS protocol, so that the centralized data center restores the target file according to the data block request, includes: Based on the number of online nodes and the number of data blocks, determine at least one offline data block, and send the data block request to the centralized data center according to the at least one offline data block and the IPFS protocol; In response to the data block request, at least one offline data block among the multiple data blocks pre-backed up by the centralized data center is obtained, and combined with the data blocks stored in the online distributed data center nodes among the at least one distributed data center node to restore the target file.

5. A hybrid data center data security storage scheduling device, characterized in that: include: a judgment module, configured to obtain a file retrieval request from a target user, determine at least one distributed data center node storing a plurality of data blocks and a plurality of verification data blocks corresponding to the target file according to the file retrieval request, determine the number of online nodes in the at least one distributed data center node, and judge whether the number of online nodes is less than the number of data blocks of the plurality of data blocks; A first recovery module is used for sending a data block request to a preset centralized data center through a preset IPFS protocol if the number of online nodes is less than the number of data blocks, so that the centralized data center recovers the target file according to the data block request and sends the target file to the target user; The second recovery module is used to determine whether the at least one distributed data center node is all online if the number of online nodes is greater than or equal to the number of data blocks, wherein if the at least one distributed data center node is all online, the target file is obtained to send it to the target user; otherwise, the target file is restored according to the multiple verification data blocks and a preset erasure code mechanism, and the target file is sent to the target user.

6. The device according to claim 5, characterized in that Also includes: a backup module, configured to split the target file into the multiple data blocks before determining the at least one distributed data center node storing the multiple data blocks and the multiple check data blocks corresponding to the target file, process the multiple data blocks through the erasure coding mechanism to obtain the multiple check data blocks, and back up the multiple data blocks in the centralized data center; A proportion division module, used to determine the data storage proportion of the distributed data center nodes corresponding to each of the preset multiple green energy sources based on the supply status of each of the green energy sources; A storage module is used to store the multiple data blocks and the multiple verification data blocks in a distributed data center node corresponding to a preset distributed data center according to the data storage ratio.

7. The device according to claim 5, characterized in that The first recovery module comprises: A determination unit, configured to determine at least one offline data block based on the number of online nodes and the number of data blocks, and send the data block request to the centralized data center according to the at least one offline data block and the IPFS protocol; A combining unit is used to obtain, in response to the data block request, at least one offline data block among the multiple data blocks pre-backed up by the centralized data center, and combine the data blocks stored in the online distributed data center nodes among the at least one distributed data center node to restore the target file.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hybrid data center data security storage scheduling method as described in any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the hybrid data center data security storage scheduling method as described in any one of claims 1-4.

10. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement the hybrid data center data security storage scheduling method as described in any one of claims 1-4.

Images