A method, device, equipment and storage medium for redundant data storage

Abstract

The present application provides a method, apparatus, device and storage medium for redundant data storage. The method includes: generating an initial storage statement for the target data under a sidecar service separated from the business service to which it belongs; converting the initial storage statement into a target storage statement under each storage node through the sidecar service; executing each of the target storage statements to redundantly store the target data. The present application can realize redundant data storage without the need for the business service to directly generate multiple storage statements under each storage node, and there is no need to change the execution logic of the business service, thereby ensuring the invariance of the business service logic, thereby simplifying the logical complexity of the business service when executing redundant data storage and improving the efficiency of redundant data storage.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically to a method, apparatus, device and storage medium for redundant data storage. Background Art

[0002] With the comprehensive development of business service functions, in order to ensure the high availability of various business data within the business service, multiple databases are usually configured for the business service to achieve redundant storage of the same business data in each database.

[0003] At present, when a business service stores the same business data in multiple databases configured under it, it is usually necessary to generate storage statements applicable to each database according to the database and table sharding rules of each database. Then, by sequentially executing each storage statement, the same business data can be sequentially stored in each database, thereby achieving data redundant storage.

[0004] However, since business services have different business logics for different databases, directly configuring multiple databases under a business service will increase the logical complexity of the business service itself, thereby increasing the time consumption of redundant data storage and reducing the efficiency of redundant data storage. Summary of the invention

[0005] The present application provides a method, device, equipment and storage medium for data redundant storage, which ensures the invariance of business service logic on the basis of realizing data redundant storage, thereby simplifying the logical complexity of business services when executing data redundant storage and improving the efficiency of data redundant storage.

[0006] In a first aspect, an embodiment of the present application provides a method for redundant data storage, the method comprising:

[0007] Generate the initial storage statement for the target data in the sidecar service separated from the business service to which it belongs;

[0008] Convert the initial storage statement into a target storage statement under each storage node through the sidecar service;

[0009] Execute each of the target storage statements to perform redundant storage on the target data.

[0010] In a second aspect, an embodiment of the present application provides a device for redundant data storage, the device comprising:

[0011] The initial statement generation module is used to generate the initial storage statement of the target data under the sidecar service separated from the business service;

[0012] A statement conversion module, used to convert the initial storage statement into a target storage statement under each storage node through the sidecar service;

[0013] The redundant storage module is used to execute each of the target storage statements and perform redundant storage on the target data.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0015] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method for redundant data storage provided in the first aspect of the present application.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the method for redundant data storage provided in the first aspect of the present application.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the method for redundant data storage provided in the first aspect of the present application is implemented.

[0018] The embodiments of the present application provide a method, apparatus, device and storage medium for redundant data storage, which pre-separates a corresponding sidecar service from a business service, first generates an initial storage statement for the target data in the business service under the sidecar service, then converts the initial storage statement into a target storage statement under each storage node through the sidecar service, and then executes each target storage statement to perform redundant storage of the target data. Redundant data storage can be achieved without the business service directly generating multiple storage statements under each storage node, and there is no need to change the execution logic of the business service, thereby ensuring the invariance of the business service logic, thereby simplifying the logical complexity of the business service when executing redundant data storage and improving the efficiency of redundant data storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flowchart of a method for redundant data storage shown in an embodiment of the present application;

[0021] Figure 2A schematic diagram of the principle of the data redundancy storage process shown in an embodiment of the present application;

[0022] Figure 3 A flowchart of another method for redundant data storage according to an embodiment of the present application;

[0023] Figure 4 A principle block diagram of a device for redundant data storage shown in an embodiment of the present application;

[0024] Figure 5 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0027] Considering that when the business service performs redundant data storage, it is necessary to directly generate different storage statements under multiple databases, which greatly increases the logical complexity of the business service itself, thereby increasing the time consumption of redundant data storage. Therefore, the embodiment of the present application designs a new redundant data storage solution. The corresponding sidecar service is pre-separated from the business service, and then, when redundantly storing the target data, an initial storage statement is first generated, and then the initial storage statement is converted into a multi-target storage statement under each storage node through the sidecar service, and then each target storage statement is executed to redundantly store the target data. There is no need for the business service to directly generate multiple storage statements under each storage node, and data redundant storage can be achieved, and there is no need to change the execution logic of the business service, ensuring the invariance of the business service logic, thereby simplifying the logical complexity of the business service when performing redundant data storage and improving the efficiency of redundant data storage.

[0028] Figure 1 This is a flow chart of a method for redundant data storage shown in an embodiment of the present application. Figure 1 , the method may specifically include the following steps:

[0029] S110, generating an initial storage statement for the target data under the sidecar service separated from the corresponding business service.

[0030] Among them, the target data refers to various business data that have redundant storage requirements when the business service performs various business functions. Considering the redundant storage of data, the business service is mainly concerned about whether the target data is stored successfully, and does not care about which database the target data is actually stored in. Moreover, judging whether the target data is stored successfully belongs to the logical function within the business service, and the actual storage operation of the target data to any specific database belongs to the control function of the business service. Therefore, in order to ensure that the execution logic of the business service does not need to be changed when the data is redundantly stored. The present application can separate the various functions of the control layer from the business service, while retaining the functions of the business logic layer. Then, the various functions of the separated control layer together constitute the sidecar service under the business service, so that the business service can execute the functions of the various logic layers, and the sidecar service implements the various control functions that are not related to the business logic within the business service.

[0031] As an optional implementation scheme in the present application, when there is a redundant storage demand for the target data, for the business service to which the target data belongs, the separated sidecar service will be used as the proxy storage node for the target data. Then, by directly analyzing the actual storage requirements of the sidecar service for the target data through the affiliated business service, an initial storage statement under the sidecar service can be directly generated. The initial storage statement can indicate the actual storage location of the target data in the proxy storage node represented by the sidecar service when the sidecar service is used as a proxy storage node. Furthermore, by sending the initial storage statement to the sidecar service, the function of whether the data at the logical level is successfully stored can be completed, and the actual redundant storage operation at the control level is subsequently performed by the sidecar service.

[0032] The initial storage statement in the present application may be a Structured Query Language (AQL) statement applicable to various types of databases, for example, a data insert statement.

[0033] It should be noted that the sidecar service in this application is deployed on the same machine as the business service to which the target data belongs. At this time, when there is a redundant storage demand for the target data, the business service to which the target data belongs will detect the storage instruction of the target data. Then, in response to the storage instruction, an initial storage statement for the target data under the sidecar service is directly generated by the business service to which it belongs. Then, Figure 2 As shown in the figure, after the business service generates the initial storage statement for the target data under the sidecar service, this initial storage statement will be forwarded to the sidecar service, which will perform the actual redundant storage operation at the control level based on the initial storage statement, without changing the execution logic of the business service, ensuring the invariance of the business service logic. In addition, the deployment of the sidecar service and the business service on the same machine can also reduce the forwarding delay of the storage statement, thereby ensuring the efficiency of data redundant storage.

[0034] S120, converting the initial storage statement into a target storage statement under each storage node through the sidecar service.

[0035] According to the characteristics of data redundancy storage, multiple storage nodes can be set to store various business data in business services. Among them, the storage nodes can be various databases, etc. Then, Figure 2 As shown, the business service and each storage node are connected through the sidecar service, so that the business service interacts with each storage node at the control level through the sidecar service, while the business service itself executes the business functions at the logic level.

[0036] In order to ensure the accuracy of redundant data storage, this application will configure the storage characteristics of each storage node in the sidecar service. Then, after obtaining the initial storage statement of the target data through the sidecar service, since the initial storage statement is suitable for the storage scenario when the sidecar service is used as a proxy storage node, and the sidecar service as a proxy storage node and each storage node may have different storage characteristics. Therefore, this application can convert an initial storage statement into a target storage statement applicable to each storage node by analyzing the differences in storage characteristics between the sidecar service as a proxy storage node and each storage node, and obtain multiple target storage statements, so that each storage node can store the same data.

[0037] S130, executing each target storage statement to perform redundant storage on the target data.

[0038] After obtaining the target storage statement under each storage node, by executing each target storage statement, the same target data can be stored in each storage node respectively, thereby completing the redundant storage of the target data.

[0039] It should be noted that in this application, the sidecar service responsible for the business functions at the control level is separated from the business service. Subsequently, by directly adding the storage features of the corresponding storage nodes configured in the sidecar service, the horizontal expansion of data redundant storage can be supported without changing the execution logic of the business service.

[0040] The technical solution provided in the embodiment of the present application pre-separates the corresponding sidecar service from the business service, first generates an initial storage statement for the target data in the business service under the sidecar service, and then converts the initial storage statement into a target storage statement under each storage node through the sidecar service, and then executes each target storage statement to perform redundant storage of the target data. Data redundant storage can be achieved without the business service directly generating multiple storage statements under each storage node, and there is no need to change the execution logic of the business service, thereby ensuring the invariance of the business service logic, thereby simplifying the logical complexity of the business service when executing data redundant storage and improving the efficiency of data redundant storage.

[0041] As an optional implementation scheme in this application, in order to ensure the accuracy of redundant data storage, this application provides a detailed explanation of the specific process of converting an initial storage statement into multiple target storage statements through the sidecar service.

[0042] Figure 3 This is a flow chart of another method for redundant data storage according to an embodiment of the present application. Figure 3 , the method may specifically include the following steps:

[0043] S310, generating an initial storage statement for the target data under the sidecar service separated from the corresponding business service.

[0044] S320, parsing the initial storage statement to obtain actual storage characteristics of the target data.

[0045] As an optional implementation scheme in the present application, after obtaining the initial storage statement of the target data through the sidecar service, it is first necessary to parse the initial storage statement to determine the actual storage characteristics of the target data under the proxy storage node represented by the sidecar service. For example, according to the storage characteristics of the proxy storage node, the actual storage location of the target data under the proxy storage node, etc.

[0046] It should be noted that the proxy storage node represented by the sidecar service pointed to in the initial storage statement in the present application can be one of the storage nodes used for redundant storage, so that when the initial storage statement is subsequently converted into the target storage statement under each storage node, the number of statement conversions is reduced, thereby improving the efficiency of redundant data storage.

[0047] S330: Generate a target storage statement for each storage node based on the actual storage characteristics and the data storage rules of each storage node pre-stored in the sidecar service.

[0048] In order to ensure the accuracy of redundant storage of data on each storage node, the present application can pre-store the data storage rules of each storage node in the sidecar service, such as the library and table sub-rules of each database serving as a storage node.

[0049] In this application, the actual storage characteristics of the target data obtained by parsing the initial storage statement and the data storage rules of each storage node pre-stored in the sidecar service are obtained. Then, the difference between the actual storage characteristics and the data storage rules of each storage node is determined to determine the actual storage location of the target data on each storage node. Furthermore, according to the actual storage location of the target data on each storage node, the target storage statement under each storage node is generated respectively, so that the same data can be stored on each storage node.

[0050] S340, executing each target storage statement in parallel to store the target data in each storage node.

[0051] After obtaining the target storage statement under each storage node, the sidecar service can execute each target storage statement in parallel on each storage node to store the target data in each storage node at the same time, thus realizing redundant storage of the target data. At this time, compared with sequentially executing multiple storage statements in the business service, it can reduce the time consumption of redundant data storage, thereby further improving the efficiency of redundant data storage.

[0052] The technical solution provided in the embodiment of the present application pre-separates the corresponding sidecar service from the business service, first generates an initial storage statement for the target data in the business service under the sidecar service, and then converts the initial storage statement into a target storage statement under each storage node through the sidecar service, and then executes each target storage statement to perform redundant storage of the target data. Data redundant storage can be achieved without the business service directly generating multiple storage statements under each storage node, and there is no need to change the execution logic of the business service, thereby ensuring the invariance of the business service logic, thereby simplifying the logical complexity of the business service when executing data redundant storage and improving the efficiency of data redundant storage.

[0053] Figure 4 The following is a block diagram of a data redundancy storage device according to an embodiment of the present application. Figure 4 As shown, the apparatus 400 may include:

[0054] An initial statement generation module 410 is used to generate an initial storage statement for target data under a sidecar service separated from the corresponding business service;

[0055] A statement conversion module 420, configured to convert the initial storage statement into a target storage statement under each storage node through the sidecar service;

[0056] The redundant storage module 430 is used to execute each of the target storage statements and perform redundant storage on the target data.

[0057] Furthermore, the statement conversion module 420 may be specifically used for:

[0058] Parsing the initial storage statement to obtain actual storage characteristics of the target data;

[0059] Based on the actual storage characteristics and the data storage rules of each storage node pre-stored in the sidecar service, a target storage statement is generated under each storage node.

[0060] Furthermore, the proxy storage node pointed to by the initial storage statement is one of the storage nodes used for redundant storage.

[0061] Furthermore, the redundant storage module 430 may be specifically used for:

[0062] The target storage statements are executed in parallel to store the target data in the storage nodes.

[0063] Furthermore, the initial statement generating module 410 may be specifically used for:

[0064] In response to the storage instruction of the target data, the initial storage statement is generated by the business service to which the target data belongs, and forwarded to the sidecar service.

[0065] Furthermore, the sidecar service is deployed on the same machine as the business service to which the target data belongs.

[0066] In an embodiment of the present application, the corresponding sidecar service is pre-separated from the business service, and an initial storage statement for the target data in the business service under the sidecar service is first generated. The initial storage statement is then converted into a target storage statement under each storage node through the sidecar service, and then each target storage statement is executed to perform redundant storage of the target data. Data redundant storage can be achieved without the business service directly generating multiple storage statements under each storage node, and there is no need to change the execution logic of the business service, thereby ensuring the invariance of the business service logic, thereby simplifying the logical complexity of the business service when executing data redundant storage and improving the efficiency of data redundant storage.

[0067] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 4 The device 400 shown can execute the method embodiment provided in the present application, and the aforementioned and other operations and / or functions of each module in the device 400 are respectively for implementing the corresponding processes in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0068] The above describes the device 400 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0069] Figure 5 It is a schematic block diagram of an electronic device 500 provided in an embodiment of the present application.

[0070] like Figure 5 As shown, the electronic device 500 may include:

[0071] The memory 510 and the processor 520, the memory 510 is used to store the computer program and transmit the program code to the processor 520. In other words, the processor 520 can call and run the computer program from the memory 510 to implement the method in the embodiment of the present application.

[0072] For example, the processor 520 may be configured to execute the above method embodiments according to instructions in the computer program.

[0073] In some embodiments of the present application, the processor 520 may include but is not limited to:

[0074] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0075] In some embodiments of the present application, the memory 510 includes but is not limited to:

[0076] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0077] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 510 and executed by the processor 520 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0078] like Figure 5 As shown, the electronic device may also include:

[0079] The transceiver 530 may be connected to the processor 520 or the memory 510 .

[0080] The processor 520 may control the transceiver 530 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, and the number of antennas may be one or more.

[0081] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0082] The embodiment of the present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the embodiment of the present application also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0083] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0084] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0085] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0086] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0087] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for redundant data storage, characterized in that: include: Generate an initial storage statement for the target data under the sidecar service separated from the business service, wherein the initial storage statement indicates the actual storage location of the target data in the proxy storage node represented by the sidecar service when the sidecar service is used as a proxy storage node; Convert the initial storage statement into a target storage statement under each storage node through the sidecar service; Execute each of the target storage statements to perform redundant storage on the target data.

2. The method according to claim 1, characterized in that The converting the initial storage statement into a target storage statement under each storage node by the sidecar service includes: Parsing the initial storage statement to obtain actual storage characteristics of the target data; Based on the actual storage characteristics and the data storage rules of each storage node pre-stored in the sidecar service, a target storage statement is generated under each storage node.

3. The method according to claim 1 or 2, characterized in that: The proxy storage node pointed to by the initial storage statement is one of the storage nodes used for redundant storage.

4. The method according to claim 1, characterized in that The executing each of the target storage statements to redundantly store the target data includes: The target storage statements are executed in parallel to store the target data in the storage nodes.

5. The method according to claim 1, characterized in that The initial storage statement for generating the target data in the sidecar service separated from the business service includes: In response to the storage instruction of the target data, the initial storage statement is generated by the business service to which the target data belongs, and forwarded to the sidecar service.

6. The method according to claim 1, characterized in that The sidecar service is deployed on the same machine as the business service to which the target data belongs.

7. A device for redundant data storage, characterized in that: include: An initial statement generation module, used to generate an initial storage statement for the target data under the sidecar service separated from the business service, wherein the initial storage statement indicates the actual storage location of the target data in the proxy storage node represented by the sidecar service when the sidecar service is used as a proxy storage node; A statement conversion module, used to convert the initial storage statement into a target storage statement under each storage node through the sidecar service; The redundant storage module is used to execute each of the target storage statements and perform redundant storage on the target data.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the data redundancy storage method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the data redundancy storage method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for redundant data storage according to any one of claims 1 to 6 is implemented.

Images