A data management method and device, electronic equipment and storage medium

Abstract

The application provides a data management method and device, electronic equipment and storage medium, and is applied to an OceanBase system. The OceanBase system comprises a master cluster, a backup cluster and object storage backup. The master cluster comprises a master replica cluster and two slave replica clusters. The backup cluster is an independent cluster. The master replica cluster obtains a data access request and processes target data. The processing result is synchronized to the slave replica cluster and the backup cluster. A database snapshot of the master replica cluster is synchronized to the object storage at a fixed time. If the master replica cluster fails, a new master replica cluster is selected from the slave replica cluster through a majority protocol. After data processing, the data is synchronized to multiple sets of clusters and object storage in real time. The method can resist various risks such as server downtime and computer room failure, guarantees data persistence and losslessness and consistency of multiple replica data, and improves data access reliability in a high concurrency scenario.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and in particular to a data management method, apparatus, electronic device, and storage medium. Background Technology

[0002] Most mainstream database clusters currently adopt a deployment architecture of a single master cluster with a small number of replicas. They only deploy replicas in the local data center to achieve basic hardware failure protection. Once a data center failure occurs or data is accidentally deleted by human error, data loss and long-term business interruption are very likely to occur.

[0003] Meanwhile, existing cluster failover relies on manual intervention, which is time-consuming. The multi-replica synchronization link is simple and cannot balance data consistency and business continuity across multiple data centers and regions. Overall disaster recovery capabilities are weak and cannot meet the data security operation requirements of highly reliable services. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a data management method, apparatus, electronic device, and storage medium to improve data storage stability and business processing stability.

[0005] According to one aspect of the present invention, a data management method is provided, applied to an OceanBase system, the OceanBase system including a primary cluster, a backup cluster, and object storage backup, the primary cluster including a primary replica cluster and two secondary replica clusters, the backup cluster being an independent cluster separate from the primary cluster, the method comprising: Data access requests are obtained through the primary replica cluster, and the target data is processed based on the data access requests; The processing results of the target data are synchronized to the two replica clusters and the backup cluster; The database snapshots of the primary replica cluster are periodically synchronized to the object storage. In the event of a failure of the primary replica cluster, a new primary replica cluster is selected from the secondary replica clusters via a majority vote protocol to handle data access requests.

[0006] According to another aspect of the present invention, a data management apparatus is provided, applied to an OceanBase system, the OceanBase system including a primary cluster, a backup cluster, and object storage backup, the primary cluster including a primary replica cluster and two secondary replica clusters, the backup cluster being an independent cluster separate from the primary cluster, the apparatus comprising: The acquisition module is used to acquire data access requests through the primary replica cluster and process the target data based on the data access requests. The synchronization module is used to synchronize the processing results of the target data to the two slave replica clusters and the backup cluster; and periodically synchronize the database snapshot of the primary replica cluster to the object storage. The selection module is used to select a new primary replica cluster from the secondary replica clusters via a majority protocol in the event of a failure of the primary replica cluster, so that data access requests can be processed by the new primary replica cluster.

[0007] According to another aspect of the present invention, an electronic device is provided, comprising: Processor; and Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform any of the data management methods described above.

[0008] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to perform any of the data management methods described above.

[0009] The present invention provides one or more technical solutions that construct a multi-level data redundancy system combining local multi-slave replica clusters, off-site backup clusters, and offline object storage snapshots. After business data is processed, it is synchronized in real time to multiple sets of slave replica clusters and off-site backup clusters. At the same time, the full snapshot of the master cluster is archived to object storage on a regular basis. This system can comprehensively resist various risks such as server downtime, data center failures, and accidental deletion or tampering, and fully ensure that data is persistent and does not lose data and that the data of multiple replicas is consistent. When the master replica cluster fails, the system automatically elects a new master cluster from the slave replica clusters to take over the business requests based on the majority protocol. The entire failover process is automated and requires no manual intervention, which greatly shortens the business interruption time and covers different levels of failure scenarios such as data center level, city level, and logical corruption. The entire data synchronization, snapshot archiving, and failure election process is executed automatically, which reduces the investment in manual operation and maintenance and the risk of misoperation. The failover process is completely transparent to the front-end business, effectively ensuring the continuous and stable operation of online business and improving the reliability of data access and user experience in high-concurrency scenarios. Attached Figure Description

[0010] Further details, features, and advantages of the invention are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a schematic diagram of the architecture of the OceanBase system provided by the present invention; Figure 2 A flowchart illustrating the data management method provided by the present invention; Figure 3A schematic diagram of the data management device provided by the present invention; Figure 4 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present invention is shown. Detailed Implementation

[0011] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the invention. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the invention.

[0012] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0013] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0014] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0015] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0016] With the digital transformation of enterprises, distributed databases such as OceanBase are widely used. However, industry-standard practices for high availability construction and operation and maintenance typically have limitations: The disaster recovery architecture is simplistic: it often adopts a "single data center with multiple replicas" or "dual-active deployment within the same city" approach, lacking cross-region disaster recovery capabilities and unable to cope with city-level disasters.

[0017] Fault recovery relies on manual intervention: When database nodes or networks fail, manual intervention (location, master switch, recovery) is often triggered by monitoring alarms. The RTO (Recovery Time Objective) is difficult to meet the minute-level or even second-level requirements of financial-grade business.

[0018] The access layer has a single point of failure risk: the proxy layer or database connection pool lacks automated high availability design, and a single point of failure often leads to a large-scale service interruption, and the recovery process involves complex configuration modifications.

[0019] Lack of stability verification: The lack of stress testing and compatibility verification based on real production traffic before changes are deployed leads to "deploying with defects" and a lack of automated loss prevention measures.

[0020] In summary, existing technologies mainly suffer from weak off-site disaster recovery capabilities, low automation in fault handling, high latency in access layer switching, and a lack of systematic verification and loss mitigation mechanisms, making it difficult to guarantee the continuity of core business and data security in extreme scenarios.

[0021] Based on this, the present invention provides a data management method, apparatus, electronic device, and storage medium. The data management method provided by the present invention can be applied to any electronic device with data management functions, such as a server, computer, or mobile terminal. The following describes the solution of the present invention with reference to the accompanying drawings: Figure 1This is a schematic diagram of the OceanBase system architecture provided by the present invention. The architecture can include a primary cluster 101, a backup cluster 102, and object storage 103. The primary cluster is built based on a three-replica mechanism. Specifically, it can be deployed across three physically distributed data centers based on the Paxos protocol, with each data center corresponding to one replica. For example, the three data centers can be distributed in different cities or different server rooms. Each data center can contain an OBProxy (access proxy node) and an Observer (data worker node). The OBProxy is the database access entry point and traffic relay station, and does not store business data. The OBServer is the underlying storage and computing core server, used to store data and execute specific data processing tasks. The Paxos protocol is a distributed consensus protocol used to synchronize data and state among multiple nodes, ensuring data consistency and service availability even when some nodes fail. The three replicas of the primary cluster can be divided into a primary replica cluster and two secondary replica clusters. When all three data centers are working normally, data access requests are processed by the primary replica cluster by default, and the data processing results are synchronized to the two secondary replica clusters. The aforementioned data access requests can include data read requests and data write requests, etc. The specific selection of the primary replica cluster and the secondary replica cluster can be determined according to the actual application scenario, and this invention does not impose specific limitations on this.

[0022] The backup cluster is a cluster independent of the primary cluster. It can be located in a different data center, city, etc., and can be of any type. For example,... Figure 1 As shown, the backup cluster can be an independent OceanBase read-only cluster, which can also contain OBProxy and Observer. Data in the primary cluster can be synchronized to the backup cluster in real time or periodically, such as synchronizing primary cluster data every 10 minutes.

[0023] Each cluster (primary replica cluster, secondary replica cluster, and standby cluster) can have the same architecture, such as including an application layer, access proxy layer, compute layer, transaction scheduling layer, and storage replication layer. The application layer is usually the business application server, the access proxy layer is OBProxy, the compute layer corresponds to OBServer, and includes the SQL engine and route resolution within OBServer. The transaction scheduling layer includes the transaction manager, 2PC (Two-Phase Commit) coordination, Paxos consistency algorithm, and automatic scheduling of faulty transactions. The storage replication layer is used for data sharding (Tablet), three replicas, redo logs, and data center (Zone) isolation.

[0024] Object Storage (BOS) operates independently of the primary and standby clusters, serving as offline archive backup storage. The primary cluster can periodically synchronize complete database snapshots, Redo logs, and other data to object storage, such as synchronizing full and incremental data daily. A database snapshot is a complete image of the database at a specific point in time, containing all data from all tables and shards. Redo logs are operation logs recorded during database insert, delete, and update operations, essentially acting as a database operation notebook. These logs only record data changes, not queries. If the primary and standby clusters are contaminated with corrupted data (e.g., due to accidental deletion or virus tampering), historical archive backups can be read from BOS to restore the intact data.

[0025] Based on the above OceanBase cluster architecture, a three-tiered backup network of online hot standby, nearline warm standby, and offline cold standby can be achieved. Correspondingly, based on this OceanBase cluster architecture, this invention also provides a data management method, such as... Figure 2 As shown, Figure 2 A flowchart illustrating the data management method provided by the present invention may include the following steps: S201. Obtain data access requests through the primary replica cluster, and process the target data based on the data access requests; S202. Synchronize the processing results of the target data to the two replica clusters and the backup cluster; S203. Periodically synchronize the database snapshots of the primary replica cluster to the object storage; S204. In the event of a failure of the primary replica cluster, a new primary replica cluster is selected from the secondary replica clusters through a majority vote protocol, so that data access requests can be processed by the new primary replica cluster.

[0026] In this embodiment of the invention, a multi-level data redundancy system is constructed, combining local multi-slave replica clusters, off-site backup clusters, and offline object storage snapshots. After business data is processed, it is synchronized to multiple sets of slave replica clusters and off-site backup clusters in real time. At the same time, the full snapshot of the primary cluster is archived to object storage on a regular basis. This system can comprehensively resist various risks such as server downtime, data center failures, and accidental deletion or tampering, and fully ensure data persistence and consistency across multiple replicas. When the primary replica cluster fails, the system automatically elects a new primary cluster from the slave replica clusters to take over the business requests based on the majority protocol. The entire failover process is automated and requires no manual intervention, significantly shortening the business interruption time. It covers different levels of failure scenarios, such as data center-level, city-level, and logical corruption. The entire data synchronization, snapshot archiving, and failure election process is executed automatically, reducing manual maintenance input and the risk of misoperation. The failover process is completely transparent to the front-end business, effectively ensuring the continuous and stable operation of online business and improving the reliability of data access and user experience in high-concurrency scenarios.

[0027] The majority rule means that a consensus is required for the replication mechanism to be valid. In other words, in the three replicas of the primary cluster, as long as two data centers are functioning normally, decisions can be made, a primary node can be elected, and data can be read. If any one of the data centers fails completely, goes down, or loses network access, the remaining two healthy data centers will automatically follow the majority rule (i.e., the number of surviving nodes equals 2), reaching a majority. Without manual intervention, they will automatically vote to elect one of the replicas as the new primary cluster to take over the business. Because the three sets of data are synchronized in real time, as long as two data centers remain healthy, all written data has been persisted to disk, ensuring no data is lost, achieving an RPO (Recovery Point Objective) of 0.

[0028] In one possible implementation, when the primary cluster's logic fails, data access requests to the primary cluster can be quickly switched to the backup cluster. Specifically, the proxy entry point can be switched; for example, the service connection address can be switched from the proxy address of the primary cluster to the proxy entry point of the backup cluster. The proxy is a database proxy layer responsible for receiving application connections, routing, and load balancing. When the primary cluster proxy fails, it needs to quickly switch to the backup proxy. The upper-layer application connects to the backup cluster's proxy, and all data access traffic is forwarded to the remote backup cluster OBServer, completing service takeover and thus avoiding service interruption.

[0029] In one possible embodiment, a cluster management module can be set up to monitor multiple preset indicator data for each worker node. These preset indicators can reflect the operational status of the worker nodes and may include CPU, I / O, and connection count. For each preset indicator, a threshold value can be preset. This threshold value is used to determine whether the corresponding indicator data is within the normal range or whether it is in a high-load state. For example, if the CPU utilization rate of the indicator data is higher than the corresponding CPU indicator threshold, it can be determined that the corresponding worker node is in a high-load state. In this invention, a multi-dimensional resource scoring algorithm can be used to determine that a worker node is in a high-load state when one or more indicator data exceed the corresponding indicator threshold.

[0030] For work nodes under high load, data in those work nodes can be scheduled to work nodes with low load. Low load nodes are usually work nodes whose metric data is lower than the corresponding metric threshold. For example, work nodes can be sorted in order of metric data from low to high, and the work node with the highest ranking in the sequence of work nodes corresponding to various metrics can be selected as the low load node. Data in high load nodes can then be scheduled to the low load node until the metric data of the high load node is no higher than the corresponding metric threshold.

[0031] In a worker node, the data of a large table is usually divided into many small pieces according to rules. Each small piece is called a data shard (Tablet, also known as a partition). The above rules can be set based on data size, business identifier, or data timestamp. Correspondingly, when scheduling data from a high-load node to a low-load node, the scheduling can be carried out on a data shard basis.

[0032] In one possible implementation, a fault management service can be set up to monitor the health status of nodes in real time. This can include monitoring the health status of worker nodes and proxy nodes. For example, health status monitoring can include heartbeat detection and disk probing. Heartbeat detection refers to all OBServer nodes in the cluster periodically sending short probe messages (heartbeat packets) to each other at millisecond / second intervals to prove that they are operating normally. If a node can receive a heartbeat packet, it indicates that the node is healthy (operating normally). Disk probing refers to the system continuously monitoring the read / write status of the server's local disk, specifically identifying false liveness faults such as IO Hang (disk freeze): the machine itself is not powered off, the network is working, and heartbeats can still be sent, but the disk is damaged / fully loaded and unable to read or write data. Heartbeat detection cannot detect this problem; disk probing must be performed separately.

[0033] If a worker node's health status is abnormal, isolation is automatically triggered. Specifically, the abnormal worker node's status can be marked as isolated. The access layer OBProxy will no longer forward any new data access requests to the isolated worker node, and the data cluster will no longer synchronize logs with it, completely cutting off traffic and preventing continuous business downtime. Simultaneously, for transactions in transit (transactions that have started but not yet been committed) within the abnormal worker node, they can be routed to the new primary worker node elected from the remaining replicas. This new primary worker node will execute the in-transit transactions, reading the complete redo logs and automatically continuing processing and rolling back any unfinished transactions. Transaction execution possesses atomicity and durability; that is, unfinished transactions will either be fully rolled back or fully committed, preventing situations where payment is successful but orders are not generated. Durability means that as long as the client receives a successful commit, the data is permanently stored on at least two healthy OBServers, ensuring no data loss even if the failed OBServer is completely destroyed.

[0034] In this invention, the access layer employs a domain name resolution and VIP (Virtual IP) migration architecture for routing data access requests. Specifically, the data access request sent by the client typically includes domain name information. A domain name is a textual identifier in the Domain Name System (DNS). The client can access the database through the domain name. Actual computer processing relies on the VIP identifier to route the data access request to a specific proxy node. The proxy node then uses load balancing to route the data access request to a specific worker node for actual processing. The database typically stores the mapping between domain names and VIPs through a DNS (Domain Name System) server. DNS resolution of the domain name yields the proxy's VIP, accessing the VIP, and thus routing the data access request to the corresponding proxy node.

[0035] The VIP will bind the Proxy nodes of the three replicas contained in the primary cluster. For example, it can bind the primary replica Proxy node (primary Proxy node) and two secondary replica Proxy nodes (secondary Proxy nodes). Specifically, the VIP can bind the primary replica Proxy instance and two secondary replica Proxy instances. When the primary replica worker node is running normally, the primary Proxy node will be accessed through this VIP by default.

[0036] As described above, the fault management service monitors the health status of nodes in real time. When the primary proxy node's heartbeat is lost, the high-availability component (such as Keepalived) migrates the VIP to a backup proxy node (the secondary proxy node) within milliseconds. This can be achieved by removing the primary proxy from the corresponding VIP binding relationship, allowing the secondary proxy node to handle data access requests and synchronously modify DNS resolution records. If the VIP migration fails, a DNS record update is automatically triggered. This mechanism ensures that upper-layer services can restore connections without modifying the connection string or restarting.

[0037] This invention provides a three-in-one distributed database multi-level disaster recovery architecture: innovatively combining the Paxos strong consistency protocol (three-center, multi-replica across different locations), physically isolated backup clusters, and low-cost object storage (BOS) backup. This architecture breaks through the limitations of traditional single disaster recovery modes, achieving full-scenario data protection from hardware failures and logical errors to catastrophic destruction.

[0038] Intelligent fault self-healing mechanism based on transaction state awareness: Unlike traditional monitoring based on process survival, this invention delves into the transaction scheduling layer of the database kernel. When a node fails, it not only isolates resources but also automatically migrates and recovers affected transactions in transit, achieving zero-interference fault awareness at the business layer.

[0039] A lossless access method based on VIP + domain name dual protection is proposed: A complementary high-availability logic for the access layer is proposed, prioritizing VIP migration for second-level intranet switching, supplemented by domain name DNS switching to handle VIP resource exhaustion or cross-network segment failure scenarios. This combined strategy completely eliminates the single point of failure vulnerability in the database proxy layer.

[0040] Based on the same inventive concept, this invention also provides a data management device applied to an OceanBase system. The OceanBase system includes a primary cluster, a backup cluster, and object storage backup. The primary cluster includes a primary replica cluster and two secondary replica clusters. The backup cluster is an independent cluster separate from the primary cluster. Figure 3 As shown, the device 300 may include: The acquisition module 301 is used to acquire data access requests through the primary replica cluster and process target data based on the data access requests; Synchronization module 302 is used to synchronize the processing results of the target data to the two slave replica clusters and the backup cluster; and periodically synchronize the database snapshot of the primary replica cluster to the object storage; Selection module 303 is used to select a new primary replica cluster from the secondary replica clusters via a majority protocol in the event of a failure of the primary replica cluster, so as to process data access requests through the new primary replica cluster.

[0041] In one possible embodiment, the primary replica cluster includes multiple worker nodes, and the device further includes: a scheduling module, used to collect multiple preset indicator data of each worker node in real time through the cluster management module. Based on the preset indicator data of each working node and the indicator thresholds set for each preset indicator, high-load nodes are identified. The data in the high-load node is sharded and scheduled to the low-load node, which is a working node whose preset indicator data is lower than the corresponding indicator threshold. The scheduling module is used to monitor the health status of each working node in real time through the fault management service; If the health status of a worker node indicates that the worker node is abnormal, isolate the abnormal worker node; Reschedule in-transit transactions on the abnormal worker node to be executed on a normal worker node in the replica cluster; The primary replica cluster includes a primary proxy node, the secondary replica cluster includes a secondary proxy node, and the device further includes: a forwarding module, used to parse the domain name information contained in the data access request when a data access request is received; Based on the domain name information, the data access request is forwarded to the corresponding VIP, which is bound to the main Proxy node and the secondary Proxy node; The data access request is sent to the main proxy node corresponding to the VIP for processing. The deletion module is used to delete the main proxy from the corresponding VIP binding relationship when the health status of the main proxy node is abnormal, so that the secondary proxy node can be used to process data access requests. The step of removing the primary proxy from the corresponding VIP binding relationship so that the secondary proxy node can handle data access requests includes: In the event of a failure of the primary cluster, the service connection address will be switched from the Proxy entry point of the primary cluster to the Proxy entry point of the backup cluster.

[0042] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this invention comply with relevant laws and regulations and do not violate public order and good morals.

[0043] An exemplary embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of the present invention.

[0044] An exemplary embodiment of the present invention also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0045] An exemplary embodiment of the present invention also provides a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform a method according to an embodiment of the present invention.

[0046] refer to Figure 4 The present invention will now be described in the form of a structural block diagram of an electronic device 400 that can serve as a server or client of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0047] like Figure 4 As shown, the electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the electronic device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0048] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, output unit 407, storage unit 408, and communication unit 409. Input unit 406 can be any type of device capable of inputting information to electronic device 400. Input unit 406 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 407 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 408 may include, but is not limited to, disks and optical discs. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0049] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above. For example, in some embodiments, any of the data management methods described above can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 can be configured to perform any of the data management methods described above by any other suitable means (e.g., by means of firmware).

[0050] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0051] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0052] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0053] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0054] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0055] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A data management method, characterized in that, The method is applied to the OceanBase system, which includes a primary cluster, a backup cluster, and object storage backup. The primary cluster includes a primary replica cluster and two secondary replica clusters. The backup cluster is an independent cluster separate from the primary cluster. Data access requests are obtained through the primary replica cluster, and the target data is processed based on the data access requests; The processing results of the target data are synchronized to the two replica clusters and the backup cluster; The database snapshots of the primary replica cluster are periodically synchronized to the object storage. In the event of a failure of the primary replica cluster, a new primary replica cluster is selected from the secondary replica clusters via a majority vote protocol to handle data access requests.

2. The method according to claim 1, characterized in that, The primary replica cluster contains multiple worker nodes, and the method further includes: The cluster management module collects multiple preset indicator data from each of the aforementioned working nodes in real time. Based on the preset indicator data of each working node and the indicator thresholds set for each preset indicator, high-load nodes are identified. Data from the high-load nodes is sharded and scheduled to low-load nodes, where the low-load nodes are working nodes whose preset indicator data is lower than the corresponding indicator threshold.

3. The method according to claim 2, characterized in that, The method further includes: The health status of each working node is monitored in real time through the fault management service. If the health status of a worker node indicates that the worker node is abnormal, isolate the abnormal worker node; The in-transit transactions on the abnormal worker node are scheduled to be executed on normal worker nodes in the replica cluster.

4. The method according to claim 3, characterized in that, The primary replica cluster includes a primary proxy node, the secondary replica cluster includes secondary proxy nodes, and the method further includes: Upon receiving a data access request, the domain name information contained in the data access request is parsed; Based on the domain name information, the data access request is forwarded to the corresponding VIP, which is bound to the main Proxy node and the secondary Proxy node; The data access request is sent to the main proxy node corresponding to the VIP for processing.

5. The method according to claim 4, characterized in that, The method further includes: If the primary proxy node is in an abnormal health state, the primary proxy will be removed from the corresponding VIP binding relationship so that the secondary proxy node can be used to handle data access requests.

6. The method according to claim 1, characterized in that, The method further includes: In the event of a failure of the primary cluster, the service connection address will be switched from the Proxy entry point of the primary cluster to the Proxy entry point of the backup cluster.

7. A data management device, characterized in that, The device is applied to the OceanBase system, which includes a primary cluster, a backup cluster, and object storage backup. The primary cluster includes a primary replica cluster and two secondary replica clusters. The backup cluster is an independent cluster separate from the primary cluster. The device includes: The acquisition module is used to acquire data access requests through the primary replica cluster and process the target data based on the data access requests. The synchronization module is used to synchronize the processing results of the target data to the two slave replica clusters and the backup cluster; and periodically synchronize the database snapshot of the primary replica cluster to the object storage. The selection module is used to select a new primary replica cluster from the secondary replica clusters via a majority protocol in the event of a failure of the primary replica cluster, so that data access requests can be processed by the new primary replica cluster.

8. The apparatus according to claim 7, characterized in that, The primary replica cluster contains multiple worker nodes, and the device further includes a scheduling module, used to collect multiple preset indicator data of each worker node in real time through the cluster management module. Based on the preset indicator data of each working node and the indicator thresholds set for each preset indicator, high-load nodes are identified. The data in the high-load node is sharded and scheduled to the low-load node, which is a working node whose preset indicator data is lower than the corresponding indicator threshold. The scheduling module is used to monitor the health status of each working node in real time through the fault management service; If the health status of a worker node indicates that the worker node is abnormal, isolate the abnormal worker node; Reschedule in-transit transactions on the abnormal worker node to be executed on a normal worker node in the replica cluster; The primary replica cluster includes a primary proxy node, the secondary replica cluster includes a secondary proxy node, and the device further includes: a forwarding module, used to parse the domain name information contained in the data access request when a data access request is received; Based on the domain name information, the data access request is forwarded to the corresponding VIP, which is bound to the main Proxy node and the secondary Proxy node; The data access request is sent to the main proxy node corresponding to the VIP for processing. The deletion module is used to delete the main proxy from the corresponding VIP binding relationship when the health status of the main proxy node is abnormal, so that the secondary proxy node can be used to process data access requests. The step of removing the primary proxy from the corresponding VIP binding relationship so that the secondary proxy node can handle data access requests includes: In the event of a failure of the primary cluster, the service connection address will be switched from the Proxy entry point of the primary cluster to the Proxy entry point of the backup cluster.

9. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-6.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

Images