Multi-cloud data processing method and device, electronic equipment and storage medium

Abstract

The invention relates to a multi-cloud data processing method and device, electronic equipment and a storage medium, and belongs to the technical field of data management, and the method comprises the steps: receiving feedback information which is sent by a first slave cloud and represents that the second storage space of the first slave cloud is insufficient; wherein the feedback information comprises a first data stream which fails to be stored in the first slave cloud; determining feature information of a first set byte segment of the first data stream; and when the first permission level represented by the feature information is less than or equal to a second permission level set by the first slave cloud, storing the first data stream into a first storage space of a matched second slave cloud.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of data management, and more specifically, to a multi-cloud data processing method, device, electronic device, and storage medium. Background Art

[0002] With the rapid development of Internet technology, the cloud, as a remote data center, can provide computing resources, data storage, database access and other IT services through the network. Users do not need to store or process large amounts of data on their own computers or local servers, but can transfer these tasks to the cloud.

[0003] At present, the storage space of the cloud is also limited. Different users can send data to the main cloud, and the main cloud will distribute the data to a slave cloud for storage. This requires the main cloud to schedule data every time, occupying more of the main cloud's memory. Summary of the invention

[0004] One purpose of the embodiments of the present disclosure is to provide a new technical solution for multi-cloud data processing.

[0005] According to a first aspect of the present disclosure, a multi-cloud data processing method is provided, wherein a master cloud and a slave cloud are in communication connection, a first storage space is set in the master cloud, and a second storage space is set in the slave cloud. The execution subject of the method is the master cloud, and the method includes:

[0006] Receiving feedback information sent by a first slave cloud indicating that a second storage space of the first slave cloud is insufficient; wherein the feedback information includes a first data stream that failed to be stored by the first slave cloud;

[0007] Determining characteristic information of a first set byte segment of the first data stream;

[0008] When the first permission level indicated by the characteristic information is less than or equal to the second permission level set by the first slave cloud, the first data stream is stored in a first storage space of a matching second slave cloud.

[0009] Optionally, after determining the characteristic information of the specific byte segment of the first data stream, the method further includes:

[0010] When the first permission level indicated by the characteristic information is greater than the second permission level set by the first slave cloud, the first data stream is temporarily stored in the second storage space.

[0011] Optionally, after temporarily storing the first data stream in the second storage space, the method further includes:

[0012] Determine a first check value and a feedback time of a second set byte segment of the first data stream;

[0013] Add the first character representing the feedback time to the second setting byte position of the second setting field, and update the first check value to the second check value; wherein the first setting byte position where the first check value is located is different from the second setting byte position.

[0014] Optionally, after storing the first data stream in the first storage space of the matching second slave cloud, the method further includes:

[0015] Determining a second check value of a second set byte segment of the first data stream;

[0016] Inputting the second verification value into a preset verification model to obtain a third verification value;

[0017] The first check value of the first data stream located in the second set byte segment is replaced by the third check value.

[0018] Optionally, the step of inputting the verification value into a preset verification model to obtain a third verification value includes:

[0019] Inputting the first identification value set by the first slave cloud and the second identification value set by the second slave cloud into a calculation algorithm of a preset verification model to obtain a combined value, and normalizing the combined value;

[0020] The normalized combined value and the first verification value are input into a reconstruction algorithm of the verification model to obtain a fourth verification value.

[0021] Optionally, the operation algorithm is expressed as:

[0022]

[0023] Among them, G goal is the combined value, G 1 is the first identification value, G 2 The second identification value.

[0024] According to a second aspect of the present disclosure, a multi-cloud data processing device is also provided, the device comprising:

[0025] A receiving module, configured to receive feedback information sent by a first slave cloud indicating that the second storage space of the first slave cloud is insufficient; wherein the feedback information includes a first data stream that fails to be stored by the first slave cloud;

[0026] A determination module, used to determine characteristic information of a specific byte segment of the first data stream;

[0027] A storage module is used to store the first data stream in a first storage space of a matching second slave cloud when the first permission level represented by the characteristic information is less than or equal to the second permission level set by the first slave cloud.

[0028] According to a third aspect of the present disclosure, an electronic device is also provided, including a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program to implement the method according to the first aspect of the present disclosure.

[0029] According to a fourth aspect of the present disclosure, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0030] According to a fifth aspect of the present disclosure, a computer program product is also provided. When the computer program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0031] One beneficial effect of the disclosed embodiments is that the multi-cloud data processing method provided by the present invention can, when the second storage space is insufficient, have the master cloud schedule the second slave cloud with an authority level not lower than that of the first slave cloud to store the first data stream. On the one hand, this reduces the occupancy rate of the master cloud memory, and on the other hand, storing the first data stream in the second slave cloud effectively improves data security.

[0032] Other features and advantages of the embodiments of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0034] Figure 1 is a schematic diagram of the composition structure of a multi-cloud data processing system to which a multi-cloud data processing method according to an embodiment can be applied;

[0035] Figure 2 is a flowchart of a multi-cloud data processing method according to an embodiment;

[0036] Figure 3 is a block diagram of a multi-cloud data processing device according to an embodiment;

[0037] Figure 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0038] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0039] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0040] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0041] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0042] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0043] <System Embodiment>

[0044] Figure 1 1 is a schematic diagram of the composition structure of a multi-cloud data processing system to which a multi-cloud data processing method according to an embodiment can be applied. Figure 1 As shown, the system includes a master cloud 10, a slave cloud 20 and a user's client 30, and the system can be applied to data storage scenarios.

[0045] The main cloud 10 is, for example, a server having functions of communication, data storage, and data processing, and a first storage space is set in the main cloud.

[0046] The slave cloud 20 is, for example, a server, and also has functions such as communication, data storage and data processing, and a second storage space is set in the slave cloud. The slave cloud 20 can communicate with the master cloud 10 through the network.

[0047] The client 30 of the user is, for example, a mobile phone, a tablet or a personal computer, etc., which is not limited here. The client 30 of the user can communicate with the designated slave cloud 20 .

[0048] As used in the embodiments of the present disclosure, the main cloud 10 memory is used to store a computer program, which is used to control the main cloud 10 processor to operate according to the multi-cloud data processing method of any embodiment. A technician can design a computer program according to the scheme of the embodiments of the present disclosure. How the computer program controls the processor to operate is well known in the art, so it will not be described in detail here.

[0049] <Method Example>

[0050] Figure 2 is a flowchart of a multi-cloud data processing method according to an embodiment. Figure 1 Zhongduo cloud data processing device 10.

[0051] like Figure 2 As shown, the multi-cloud data processing method of this embodiment may include the following steps S210 to S230:

[0052] Step S210, receiving feedback information sent by the first slave cloud indicating that the second storage space of the first slave cloud is insufficient; wherein the feedback information includes the first data stream that failed to be stored in the first slave cloud.

[0053] In some examples, the first slave cloud and the second slave cloud are two different slave clouds, and when the first slave cloud stores the first data stream uploaded by the user's client, the first slave cloud detects the second storage space of the first slave cloud. When it is detected that the first storage space of the first slave cloud is insufficient, feedback information indicating that the second storage space of the first slave cloud is insufficient can be fed back to the master cloud.

[0054] Step S220, determining characteristic information of a first set byte segment of the first data stream.

[0055] In some examples, the first set byte segment may be a specific byte segment such as a byte header or tail of the first data stream, which is not limited here. The characteristic information is, for example, user:owner123.

[0056] In some embodiments, since the number of storage spaces with higher permission levels is small, the data streams stored in the storage spaces with higher permission levels need to be stored again in the storage spaces with corresponding permission levels to ensure the reasonable use of the storage space in the cloud. After step S220, the method further includes the following step S221:

[0057] Step S221, when the first permission level indicated by the characteristic information is greater than the second permission level set by the first slave cloud, the first data stream is temporarily stored in the second storage space.

[0058] In some embodiments, in order to ensure that the stored data is complete, after step S221, the method further includes the following steps S222 and S223:

[0059] Step S222, determining a first check value and a feedback time of a second set byte segment of the first data stream.

[0060] In some examples, the second set byte segment can be a specific byte segment such as the byte header or tail of the first data stream, which is not limited here, and the first check value can be any character such as a number or letter, which is not limited here.

[0061] In some examples, the feedback time may be the time when the slave cloud first feeds back to the master cloud.

[0062] Step S223, adding the first character representing the feedback time to the second setting byte position of the second setting field, and updating the first check value to the second check value; wherein the first setting byte position where the first check value is located is different from the second setting byte position.

[0063] In some examples, the first character may be a single character or multiple characters. For example, if the feedback time is 12 o'clock, the first character is 12, which is not limited here.

[0064] In some examples, the second set byte segment of the first data stream is the first three digits of the byte header of the first data stream, and the first three digits are "005", where "5" represents the first check value. When the first character "12" is added to the second set byte position of the second set field, the first three digits of the byte header of the first data stream are updated to "125".

[0065] Step S230: When the first permission level indicated by the characteristic information is less than or equal to the second permission level set by the first slave cloud, the first data stream is stored in the first storage space of the matching second slave cloud.

[0066] In some examples, each slave cloud has a different permission level, and the permission level of each slave cloud can be set manually, which is not limited here. For example, when the first permission level is "3" and the second permission level is "2", the first permission level is greater than the second permission level.

[0067] In other words, when the second storage space is insufficient, the main cloud will schedule the second slave cloud with an authority level not lower than that of the first slave cloud to store the first data stream. On the one hand, this reduces the memory occupancy rate of the main cloud, and on the other hand, storing the first data stream in the second slave cloud effectively improves data security.

[0068] In some embodiments, in order to further improve the security of the data, after step S230, the method further includes the following steps S231 to S233:

[0069] Step S231, determining a second check value of a second set byte segment of the first data stream.

[0070] Step S232, input the second verification value into a preset verification model to obtain a third verification value.

[0071] In some embodiments, step S232 may include the following steps S2321 and S2322:

[0072] Step S2321, input the first identification value set by the first slave cloud and the second identification value set by the second slave cloud into the calculation algorithm of the preset verification model to obtain a combined value, and normalize the combined value.

[0073] In some embodiments, the expression of the operation algorithm is

[0074]

[0075] Among them, G goal is the combined value, G 1 is the first identification value, G 2 The second identification value.

[0076] Step S2322, input the normalized combined value and the first verification value into a reconstruction algorithm of the verification model to obtain a fourth verification value.

[0077] Among them, the reconstruction algorithm is an existing algorithm, which can be a fourth verification value obtained by calculating the normalized combined value and any value through an arbitrary operator. For example, the operator is "+" and the arbitrary value is "1", then the fourth verification value = combined value + 1.

[0078] Step S233: Replace the first check value of the first data stream located in the second set byte segment with the third check value.

[0079] In other words, by replacing the first verification value with the third verification value, the security of the data can be further improved.

[0080] <Equipment Example 1>

[0081] Figure 3 is a principle block diagram of a multi-cloud data processing device according to an embodiment. Figure 3 As shown, the multi-cloud data processing device 300 may include:

[0082] The receiving module 310 is configured to receive feedback information sent by the first slave cloud indicating that the second storage space of the first slave cloud is insufficient; wherein the feedback information includes the first data stream that failed to be stored in the first slave cloud;

[0083] A determination module 320, configured to determine characteristic information of a specific byte segment of the first data stream;

[0084] The storage module 330 is used to store the first data stream in the first storage space of the matching second slave cloud when the first permission level indicated by the characteristic information is less than or equal to the second permission level set by the first slave cloud.

[0085] Optionally, the multi-cloud data processing device 300 further includes a temporary storage module, which is used to temporarily store the first data stream in the second storage space when the first permission level indicated by the characteristic information is greater than the second permission level set by the first slave cloud.

[0086] Optionally, the multi-cloud data processing device 300 also includes a feedback module, which is used to determine a first check value and feedback time of a second set byte segment of the first data stream; add a first character representing the feedback time to a second set byte position of the second set field, and update the first check value to a second check value; wherein the first set byte position where the first check value is located is different from the second set byte position.

[0087] Optionally, the multi-cloud data processing device 300 also includes a verification module, which is used to determine a second verification value of a second set byte segment of the first data stream; input the second verification value into a preset verification model to obtain a third verification value; and replace the first verification value of the first data stream located in the second set byte segment with the third verification value.

[0088] Optionally, the verification module is also used to input a first identification value set from the cloud and a second identification value set from the cloud into an operation algorithm of a preset verification model to obtain a combined value, and normalize the combined value; input the normalized combined value and the first verification value into a reconstruction algorithm of the verification model to obtain a fourth verification value.

[0089] The multi-cloud data processing device 300 may be Figure 1 The main cloud in 10.

[0090] <Equipment Example 2>

[0091] Figure 4 is a schematic diagram of the hardware structure of an electronic device according to another embodiment.

[0092] like Figure 4As shown, the electronic device 400 includes a processor 410 and a memory 420, wherein the memory 420 is used to store an executable computer program, and the processor 410 is used to execute a method such as any of the above method embodiments under the control of the computer program.

[0093] The electronic device 400 may be Figure 1 The main cloud in 10.

[0094] Each module of the multi-cloud data processing device 300 may be implemented by the processor 410 in this embodiment executing a computer program stored in the memory 420 , or may be implemented by other structures, which is not limited here.

[0095] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0096] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0097] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0098] The computer program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0099] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0100] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0101] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0102] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0103] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A multi-cloud data processing method, It is characterized in that The master cloud and the slave cloud are in communication connection, the master cloud is provided with a first storage space, and the slave cloud is provided with a second storage space, the execution subject of the method is the master cloud, and the method comprises: Receiving feedback information sent by a first slave cloud indicating that a second storage space of the first slave cloud is insufficient; wherein the feedback information includes a first data stream that failed to be stored by the first slave cloud; Determining characteristic information of a first set byte segment of the first data stream; When the first permission level indicated by the characteristic information is less than or equal to the second permission level set by the first slave cloud, the first data stream is stored in a first storage space of a matching second slave cloud.

2. The method according to claim 1, It is characterized in that After determining the characteristic information of the specific byte segment of the first data stream, the method further includes: When the first permission level indicated by the characteristic information is greater than the second permission level set by the first slave cloud, the first data stream is temporarily stored in the second storage space.

3. The method according to claim 1, It is characterized in that After temporarily storing the first data stream in the second storage space, the method further includes: Determine a first check value and a feedback time of a second set byte segment of the first data stream; Add the first character representing the feedback time to the second setting byte position of the second setting field, and update the first check value to the second check value; wherein the first setting byte position where the first check value is located is different from the second setting byte position.

4. The method according to claim 3, It is characterized in that After storing the first data stream in the first storage space of the matching second slave cloud, the method further includes: Determining a second check value of a second set byte segment of the first data stream; Inputting the second verification value into a preset verification model to obtain a third verification value; The first check value of the first data stream located in the second set byte segment is replaced by the third check value.

5. The method according to claim 4, It is characterized in that The step of inputting the verification value into a preset verification model to obtain a third verification value comprises: Inputting the first identification value set by the first slave cloud and the second identification value set by the second slave cloud into a calculation algorithm of a preset verification model to obtain a combined value, and normalizing the combined value; The normalized combined value and the first verification value are input into a reconstruction algorithm of the verification model to obtain a fourth verification value.

6. The method according to claim 5, It is characterized in that The expression of the operation algorithm is: Among them, G goal is the combined value, G 1 is the first identification value, G 2 The second identification value.

7. A multi-cloud data processing device, It is characterized in that include: A receiving module, configured to receive feedback information sent by a first slave cloud indicating that the second storage space of the first slave cloud is insufficient; wherein the feedback information includes a first data stream that fails to be stored by the first slave cloud; A determination module, used to determine characteristic information of a specific byte segment of the first data stream; A storage module is used to store the first data stream in a first storage space of a matching second slave cloud when the first permission level represented by the characteristic information is less than or equal to the second permission level set by the first slave cloud.

8. An electronic device, It is characterized in that The method comprises a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, It is characterized in that The computer readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images