Electronic archive management method and device and storage medium

Through the collaborative work of dbproxy and management cluster, the problems of slow transmission rate and unreasonable storage of FTP methods are solved, efficient management and strong backup of electronic files are achieved, and data security is improved.

CN119106083APending Publication Date: 2024-12-10GUANGDONG ESHORE TECH
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202310686598.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-09
Publication Date
2024-12-10

AI Technical Summary

Technical Problem

In the management of financial data and archive file, the FTP method has a slow transmission rate, the traditional storage method is unreasonable, and effective backup cannot be achieved, resulting in complex management.

Method used

Get SQL statements through dbproxy in the data access layer, connect to the main gateway of the gateway layer, transmit it to the main database, and perform strong synchronization processing, synchronize electronic files to at least two slave databases, and use the management cluster monitoring and switching mechanism to achieve high availability and backup, unstructured data uses HBase to store metadata and HDFS to store file data.

Benefits of technology

It improves file transfer rate, realizes strong backup of electronic files, facilitates management and improves security, and meets the requirements of high availability and data security.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119106083A_ABST
    Figure CN119106083A_ABST
Patent Text Reader

Abstract

The invention discloses an electronic archive management method and device and a storage medium, and the method comprises the steps: obtaining an SQL statement used for storing an electronic archive through a dbproxy of a data access layer, connecting a main gateway of a gateway layer according to the SQL statement, and connecting the main gateway with a main database of a database layer to transmit the electronic archive to the main database. Compared with an existing FTP mode, the speed is increased; strong synchronization processing is carried out through the master database, the electronic archive is synchronized to the at least two slave databases, strong synchronization processing is introduced, the electronic archive is synchronized to the at least two slave databases, strong backup of the electronic archive is effectively achieved, management of the electronic archive is facilitated, and user experience is improved. And the management effect and the safety of the electronic archives are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of Internet, and in particular to an electronic archive management method, device and storage medium. Background Art

[0002] With the development of business, the number of financial data and archive files is large, and the data volume of a single file is small. The current FTP method used to achieve file writing transmission rate is very slow, and the traditional archive file storage method is also very unreasonable. The storage space cannot be shared and effective backup cannot be achieved, which makes the management and maintenance of archives very complicated. Summary of the invention

[0003] In view of this, in order to solve at least one of the above technical problems, an object of the present invention is to provide an electronic archive management method, device and storage medium to improve management effect.

[0004] An embodiment of the present invention provides an electronic archive management method, comprising:

[0005] Obtaining SQL statements for electronic archive storage through dbproxy of the data access layer, and connecting to the main gateway of the gateway layer according to the SQL statements;

[0006] Connecting to the main database of the database layer through the main gateway to transmit the electronic file to the main database;

[0007] The electronic archive is synchronized to at least two slave databases by performing strong synchronization processing through the master database.

[0008] Further, the connecting to the main gateway of the gateway layer according to the SQL statement includes:

[0009] Parsing the SQL statement and performing field splitting by the dbproxy;

[0010] The execution strategy is determined according to the field value of the field split and routed to the corresponding target partition for execution, and the target partition is connected to the main gateway of the gateway layer through the target partition.

[0011] Furthermore, the method further comprises:

[0012] Obtain the first status information of the main gateway of the gateway layer through the management cluster;

[0013] When the first status information is abnormal, a gateway switching instruction is sent through the management cluster to enable the standby gateway, so that the standby gateway is connected to the data access layer and the main database.

[0014] Further, the enabling of the standby gateway so that the standby gateway is connected to the data access layer and the primary database includes:

[0015] The backup gateway is enabled, and high-availability switching between the primary gateway and the backup gateway is achieved by configuring a virtual IP address and a KEEPALIVED mode, so that the switched backup gateway is connected to the data access layer and the primary database.

[0016] Furthermore, the method further comprises:

[0017] Acquire second status information of the main database of the database layer through the management cluster;

[0018] When the second status information is abnormal, the management cluster sends a database switching instruction, and one of the slave databases synchronized with the electronic file is used as the target slave database to provide services.

[0019] Furthermore, the sending of a database switching instruction through the management cluster to use one of the slave databases synchronized with the electronic archive as a target slave database to provide services includes:

[0020] Acquire, through the management cluster, the third state information of the slave database synchronized with the electronic archive, determine that the third state information is a normal candidate slave database, determine a target slave database from the candidate slave database, issue a database switching instruction through the management cluster to switch to the target slave database, and provide services;

[0021] or,

[0022] The third status information of the slave database synchronized with the electronic archive is obtained through the management cluster, the third status information is determined to be a normal candidate slave database, a target slave database is determined from the candidate slave database, a database switching instruction is issued through the management cluster to switch to the target slave database, and the fourth status information of the target slave database is obtained in real time or at preset intervals. When the fourth status information is abnormal, a new target slave database is determined from the candidate slave database, and a database switching instruction is issued through the management cluster to switch to the new target slave database to provide services.

[0023] Furthermore, after the electronic archive is transmitted to the master database, the process further includes:

[0024] When the electronic archive is unstructured data, metadata with indexes is stored in HBase, and file data is stored in HDFS; wherein the metadata includes at least one of the file name, file length, and file status of the file data.

[0025] The embodiment of the present invention further provides an electronic archive management device, comprising:

[0026] An acquisition module, used to acquire SQL statements for electronic archive storage through dbproxy of the data access layer, and connect to the main gateway of the gateway layer according to the SQL statements;

[0027] A connection module, used to connect to the main database of the database layer through the main gateway to transmit the electronic file to the main database;

[0028] The strong synchronization module is used to perform strong synchronization processing through the master database to synchronize the electronic file to at least two slave databases.

[0029] An embodiment of the present invention also provides an electronic archive management device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method.

[0030] An embodiment of the present invention also provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the method.

[0031] The beneficial effects of the present invention are:

[0032] The SQL statement for electronic archive storage is obtained through the dbproxy of the data access layer, and the main gateway of the gateway layer is connected according to the SQL statement. The main database of the database layer is connected through the main gateway to transmit the electronic archive to the main database, which improves the speed compared with the existing FTP method; the electronic archive is synchronized to at least two slave databases through the main database through strong synchronization processing. By introducing strong synchronization processing, the electronic archive is synchronized to at least two slave databases, which effectively realizes the strong backup of the electronic archive, facilitates the management of the electronic archive, and improves the management effect and security of the electronic archive.

[0033] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the steps of the electronic archive management method of the present invention;

[0035] Figure 2 It is a schematic diagram of the gateway layer, database layer and management cluster of a specific embodiment of the present invention;

[0036] Figure 3 It is a schematic diagram of field splitting according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0038] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0039] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0040] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides an electronic archive management method, comprising steps S100-S300:

[0041] S100, obtaining SQL statements for electronic archive storage through dbproxy of the data access layer, and connecting to the main gateway of the gateway layer according to the SQL statements.

[0042] Optionally, a distributed database middleware solution is formed through a unified data access layer, and the dbproxy of the data access layer is used to obtain the SQL statements of users accessing the data access layer, such as SQL statements for electronic archive storage. It should be noted that dbproxy is the core component of UDAL, which implements the Sever process of the mysa protocol. Front-end users can regard dbproxy as a database proxy and directly access it through the mysql client tool or command line. The back-end communicates with multiple mysal databases using the mysa native protocol, and can also communicate with most mainstream database servers using the dbc protocol. The core function of dbproxy is to shard libraries and tables and shield the application layer from the access difficulties caused by sharding libraries and tables.

[0043] Optionally, connecting to the main gateway of the gateway layer according to the SQL statement in step S100 includes steps S110-S130:

[0044] S110. Parse the SQL statement through dbproxy and perform field splitting.

[0045] Optionally, dbproxy is used to parse the SQL statement, understand the meaning of the SQL statement, and then split the SQL statement into fields according to the meaning.

[0046] S120, determine the execution strategy according to the field value of the field split and route it to the corresponding target partition for execution, and connect to the main gateway of the gateway layer through the target partition.

[0047] Optionally, the field value obtained by field splitting is matched with the preset strategy to determine the execution strategy, and then routed to the corresponding target partition for execution, such as Figure 3 As shown, transmission communication is performed through the target partition connected to the main gateway (Gateway(Main)) of the gateway layer.

[0048] S200, connecting to the main database of the database layer through the main gateway to transmit the electronic file to the main database.

[0049] In the embodiment of the present invention, the main database of the database layer (DB layer) is connected via the main gateway to transmit the electronic file to the main database (referred to as the main database) for storage.

[0050] S300, performing strong synchronization processing through the master database to synchronize the electronic archive to at least two slave databases.

[0051] In the embodiment of the present invention, two slave databases (referred to as slave databases) are used as an example for explanation, and other embodiments may include more than two slave databases. Specifically, a master database and two slave databases constitute a database node group (SET). In a database node group, the master database will perform strong synchronization processing, and synchronize the data received by the master database, such as electronic files, to the two slave databases in the database node group, realizing a data storage strategy of automatic offline storage in triplicate, and data storage is more reliable; and the deployment of two slave databases is very effective in eliminating instantaneous network jitters. If any slave database crashes, the master database can still be written.

[0052] In the embodiment of the present invention, a management cluster is configured to obtain the status information of each gateway in the gateway layer and the status information of each database in the database layer, monitor the gateway and the database, and send a switching instruction to switch when a fault / abnormality occurs. Optionally, the management cluster includes a keeper cluster (Scheduler) and a zookeeper cluster.

[0053] In the embodiment of the present invention, each database is configured with a heartbeat monitoring and information collection module (Agent) to obtain the status information of each database and ensure the robustness of the cluster; under the distributed architecture, several database node groups (SET) can provide a "logically unified, physically dispersed" distributed database instance, and the interaction between each database node group can be completed through the management cluster. Among them, the zookeeper cluster is used to receive the database status information collected by the gateway and the agent and upload it to the keeper cluster, provide SET routing, realize transparent reading and writing, and send the switching instructions of the keeper cluster to the gateway layer and the agent of the database layer. The zookeeper cluster implements the PAXOS algorithm internally to ensure the data consistency of the distributed system. The keeper cluster is used to monitor the cluster status of the zookeeper cluster and perform switching control of the zookeeper cluster when abnormal.

[0054] Optionally, in an embodiment of the present invention, the keeper cluster serves as the management and scheduling center of the cluster, mainly managing the normal operation of the SET. To ensure the high availability of the keeper cluster, three keeper clusters are deployed on three independent machines, and disaster recovery is completed by configuring the election mechanism of the zookeeper cluster to ensure that the scheduling cluster is fault-free and provide the creation, deletion, replacement and other tasks of the SET.

[0055] Optionally, the electronic archive management method of the embodiment of the present invention further includes steps S401-S402:

[0056] S401. Obtain first status information of a main gateway of a gateway layer through a management cluster.

[0057] S402: When the first status information is abnormal, a gateway switching instruction is issued through the management cluster to enable the standby gateway so that the standby gateway is connected to the data access layer and the main database.

[0058] Optionally, the first status information of the main gateway of the gateway layer is obtained through the zookeeper cluster, and the keeper cluster analyzes whether the first status information is abnormal. When it is abnormal, a gateway switching instruction is issued, which is sent to the main gateway and the backup gateway (Gateway (backup)) of the gateway layer through the zookeeper cluster, stops the main gateway and enables the backup gateway, connects the backup gateway to the data access layer and the main database, and realizes data transmission through the backup gateway.

[0059] Optionally, in S402, enabling the standby gateway to connect to the data access layer and the main database includes step S4021:

[0060] S4021. Enable the backup gateway, and implement high-availability switching between the primary gateway and the backup gateway by configuring the virtual IP address and the KEEPALIVED mode, so that the switched backup gateway is connected to the data access layer and the primary database.

[0061] Optionally, in an embodiment of the present invention, the main gateway and the backup gateway of the gateway layer can be configured with a virtual IP address (VIP) through keepalived. When the backup gateway needs to be enabled, high-availability switching between the main gateway and the backup gateway can be achieved through the configured virtual IP address (VIP) and the KEEPALIVED method, so that the switched backup gateway connects to the data access layer and the main database for data transmission.

[0062] In the embodiment of the present invention, from the maintainability level, an integrated management platform for configuration, publishing, monitoring, and operation and maintenance is provided, and complete status information monitoring and operation and maintenance analysis tools are provided to quickly locate and solve problems. Distributed databases that store complex structured data can be expanded in a Scale Out (horizontal) manner. By adopting a reasonable data slicing strategy for the data models of various applications, storage and performance bottlenecks can be broken. Reduce enterprise costs: The "de-IOE" model that uses cheap X86 + cheap storage + open source software and self-developed software can greatly save enterprise investment compared to commercial products, and can cope with peak traffic by realizing dynamic and smooth expansion and contraction of data access nodes.

[0063] Optionally, the electronic archive management method of the embodiment of the present invention further includes steps S411-S412:

[0064] S411. Obtain second status information of a main database of the database layer through the management cluster.

[0065] S412: When the second status information is abnormal, a database switching instruction is issued through the management cluster, and one of the slave databases synchronized with the electronic archive is used as the target slave database to provide services.

[0066] Optionally, the second state information of the main database of the database layer is obtained through the zookeeper cluster and the Agent. The keeper cluster analyzes whether the second state information is abnormal. If it is abnormal, a database switching instruction is issued, which is sent through the zookeeper cluster to the Agent of the main database of the database layer and one of the corresponding Agents in the slave databases synchronized with the electronic archives. The slave database at this time is recorded as the target slave database, the main database is disabled, and the target slave database is enabled to provide services.

[0067] Optionally, in S412, a database switching instruction is issued by the management cluster to use one of the slave databases synchronized with the electronic archive as the target slave database to provide services, including step S4121 or S4122:

[0068] S4121. Obtain the third state information of the slave database synchronized with the electronic archive through the management cluster, determine that the third state information is a normal candidate slave database, determine the target slave database from the candidate slave database, and issue a database switching instruction through the management cluster to switch to the target slave database to provide services.

[0069] In an embodiment of the present invention, the third state information of all slave databases with electronic archives synchronized at the database layer is obtained through the zookeeper cluster and the Agent. The keeper cluster analyzes the third state information and determines that the third state information is a normal candidate slave database. The target slave database is determined from the candidate slave database, for example, including but not limited to random selection. Then, the keeper cluster issues a database switching instruction, which is sent to the Agent of the master database and the Agent of the target slave database through the zookeeper cluster, deactivates the master database and enables the target slave database to provide services.

[0070] S4122. Obtain the third state information of the slave database synchronized with the electronic archives through the management cluster, determine that the third state information is a normal candidate slave database, determine the target slave database from the candidate slave database, issue a database switching instruction through the management cluster to switch to the target slave database, obtain the fourth state information of the target slave database in real time or at preset intervals, and when the fourth state information is abnormal, determine a new target slave database from the candidate slave database, issue a database switching instruction through the management cluster to switch to the new target slave database, and provide services.

[0071] Optionally, based on step S4121, after it is determined that the target slave database provides services, the target slave database is also monitored, and the management cluster obtains the fourth status information of the target slave database in real time or at preset intervals. When the fourth status information is abnormal, a new target slave database is determined from the candidate slave databases synchronized with electronic archives. Then, the keeper cluster issues a database switching instruction, which is sent to the Agent of the current target slave database and the Agent of the new target slave database through the zookeeper cluster, and the current target slave database is deactivated and the new target slave database provides services.

[0072] Optionally, the electronic archive management method of the embodiment of the present invention further includes step S420 after the electronic archive is transmitted to the master database:

[0073] S420. When the electronic archive is unstructured data, the metadata with index is stored in HBase, and the file data is stored in HDFS.

[0074] Optionally, for electronic archives of unstructured data, a self-developed distributed file system CTDFS component is used. The CTDFS component is self-developed based on the Hadoop ecosystem, providing users with a distributed file system product with reliable storage and efficient processing capabilities in scenarios of massive data, especially massive small file data. Specifically, in order to solve the impact of massive small file scenarios on HDFS, the CTDFS component stores indexed metadata through HBase and stores file data through HDFS, so that one HDFS stores multiple small files (data volume / size is less than a preset threshold). The metadata includes but is not limited to the file name of the file data, such as a small file name, an HDFS file name, a file length, and at least one of the file status. When the file is small and does not fill a data block, multiple files can be merged and stored in one data block, thereby meeting the requirements for massive file storage. It should be noted that when performing strong synchronization processing, the storage method in the slave database can be similar to the storage method in the above-mentioned master database, which will not be repeated.

[0075] Compared with the prior art, the electronic archive management method of the embodiment of the present invention:

[0076] 1) By adopting its own cloud technology platform, we build an offline cloud platform including access layer, gateway layer, database layer and management cluster to solve the multi-point backup problem of online and offline data of electronic archives, improve data security and meet the high availability requirements of accounting electronic archive data;

[0077] 2) Implement a cloud computing-based electronic archive storage and offline automatic triplicate solution. Through a distributed triplicate strong backup storage strategy for electronic archive structured data and unstructured data, one piece of data can store multiple backups. Unstructured small file data meets the online and offline data storage requirements of the electronic archive management method. Each set of online and offline data can adopt a triplicate strong backup strategy. Important data can be backed up offline in a different location to improve the security of archive data storage.

[0078] 3) The distributed database transforms the MYSQL database layer replication module and introduces enhanced strong synchronization, so that when the application transaction is submitted, the logs and files generated by the transaction are at least available on the master and slave databases. Therefore, if the master database crashes at this time, the slave database provides services, and its data is consistent with the master database, without any transaction data loss problem, high fault tolerance, and strong practicality.

[0079] 4) Based on MySQL and Hadoop open source protocols, a unified database access layer is used to form a distributed database middleware solution to solve the problem of transparent data access caused by distributed system database sub-libraries and sub-tables;

[0080] 5) Supports multiple split dimensions such as strings, numbers, dates, etc., and can smoothly expand capacity without interrupting business. Structured data supports SQL 92 standards in system development compatibility, is fully compatible with SQL protocols, syntax, and most SQL clients, and can achieve read-write separation with zero code changes. In terms of data security and reliability, it supports distributed architecture without single service points, stable services, and a complete data layer master-slave disaster recovery solution with automatic master-slave switching function, making data storage more reliable.

[0081] The embodiment of the present invention further provides an electronic archive management device, comprising:

[0082] The acquisition module is used to obtain the SQL statement for electronic archive storage through the dbproxy of the data access layer, and connect to the main gateway of the gateway layer according to the SQL statement;

[0083] A connection module, used to connect to the main database of the database layer through the main gateway to transmit the electronic file to the main database;

[0084] The strong synchronization module is used to perform strong synchronization processing through the master database to synchronize the electronic archives to at least two slave databases.

[0085] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments, and will not be repeated herein.

[0086] The embodiment of the present invention also provides another electronic archive management device, which includes a processor and a memory, wherein at least one instruction, at least one program, code set or instruction set is stored in the memory, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the electronic archive management method of the above embodiment. Optionally, the electronic archive management device includes but is not limited to a mobile phone, a tablet computer, a computer, and a car computer.

[0087] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments, and will not be repeated herein.

[0088] An embodiment of the present invention also provides a computer-readable storage medium, in which at least one instruction, at least one program, code set or instruction set is stored, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the electronic archive management method of the aforementioned embodiment.

[0089] The embodiment of the present invention also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the electronic archive management method of the above embodiment.

[0090] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0091] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0092] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present application can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0093] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0094] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An electronic archive management method, characterized in that: include: Obtaining SQL statements for electronic archive storage through dbproxy of the data access layer, and connecting to the main gateway of the gateway layer according to the SQL statements; Connecting to the main database of the database layer through the main gateway to transmit the electronic file to the main database; The electronic archive is synchronized to at least two slave databases by performing strong synchronization processing through the master database.

2. The electronic archive management method according to claim 1, characterized in that: The main gateway connected to the gateway layer according to the SQL statement includes: Parsing the SQL statement and performing field splitting by the dbproxy; The execution strategy is determined according to the field value of the field split and routed to the corresponding target partition for execution, and the target partition is connected to the main gateway of the gateway layer through the target partition.

3. The electronic archive management method according to any one of claims 1 to 2, characterized in that: The method further comprises: Obtain the first status information of the main gateway of the gateway layer through the management cluster; When the first status information is abnormal, a gateway switching instruction is sent through the management cluster to enable the standby gateway, so that the standby gateway is connected to the data access layer and the main database.

4. The electronic archive management method according to claim 3, characterized in that: The enabling of the standby gateway so that the standby gateway is connected to the data access layer and the primary database includes: The backup gateway is enabled, and high-availability switching between the primary gateway and the backup gateway is achieved by configuring a virtual IP address and a KEEPALIVED mode, so that the switched backup gateway is connected to the data access layer and the primary database.

5. The electronic archive management method according to any one of claims 1 to 2, characterized in that: The method further comprises: Acquire second status information of the main database of the database layer through the management cluster; When the second status information is abnormal, the management cluster sends a database switching instruction, and one of the slave databases synchronized with the electronic file is used as the target slave database to provide services.

6. The electronic archive management method according to claim 5, characterized in that: The sending of the database switching instruction by the management cluster to use one of the slave databases synchronized with the electronic archive as the target slave database to provide services includes: Acquire, through the management cluster, the third state information of the slave database synchronized with the electronic archive, determine that the third state information is a normal candidate slave database, determine a target slave database from the candidate slave database, issue a database switching instruction through the management cluster to switch to the target slave database, and provide services; or, The third status information of the slave database synchronized with the electronic archive is obtained through the management cluster, the third status information is determined to be a normal candidate slave database, a target slave database is determined from the candidate slave database, a database switching instruction is issued through the management cluster to switch to the target slave database, and the fourth status information of the target slave database is obtained in real time or at preset intervals. When the fourth status information is abnormal, a new target slave database is determined from the candidate slave database, and a database switching instruction is issued through the management cluster to switch to the new target slave database to provide services.

7. The electronic archive management method according to claim 1, characterized in that: After the electronic archive is transmitted to the master database, the method further includes: When the electronic archive is unstructured data, metadata with indexes is stored in HBase, and file data is stored in HDFS; wherein the metadata includes at least one of the file name, file length, and file status of the file data.

8. An electronic archive management device, characterized in that: include: An acquisition module, used to acquire SQL statements for electronic archive storage through dbproxy of the data access layer, and connect to the main gateway of the gateway layer according to the SQL statements; A connection module, used to connect to the main database of the database layer through the main gateway to transmit the electronic file to the main database; The strong synchronization module is used to perform strong synchronization processing through the master database to synchronize the electronic file to at least two slave databases.

9. An electronic file management device, characterized in that: The electronic archive management device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1-7.

Citation Information

Patent Citations

  • Systems and methods for database proxy request switching

    CN103403707A

  • Solution system and method for disaster recovery across Internet data centers (IDCs) based on open-source database

    CN106407042A

  • Distributed database system and implementation method thereof

    CN106570074A

  • Cloud computing-based ocean archive database parallelization construction method and device

    CN110716897A

  • Data processing method and device and storage medium

    CN112866256A