Data processing method, system and apparatus

Abstract

A data processing method, system and apparatus. The data processing method is applied to a cloud storage server, and comprises: receiving a database operation request submitted for a target database (S202); when it is determined on the basis of the database operation request that the target database is abnormal, reading, from a local data file, program data recorded in a backup cycle, and reading, from a storage node, disk metadata recorded in the backup cycle (S204); on the basis of the program data, starting up a database service program associated with the target database, and loading the disk metadata into a memory of a management control node (S206); and on the basis of a program startup result and a metadata loading result, executing a database operation task corresponding to the database operation request (S208).

Description

Data processing method, system and device

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on December 22, 2023, with application number 202311785700.1 and application name “Data Processing Method, System and Device,” the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The embodiments of this specification relate to the field of database technology, and in particular to data processing methods, systems, and devices. Background Art

[0003] With the development of computer technology, cloud storage services are being applied in more and more scenarios, and the providers of cloud storage services are gradually expanding in order to provide users with stable services. As the scale gradually increases, the number of storage machines inevitably increases. For large-scale storage machines, cloud storage service providers need to monitor not only frequently occurring events, but also accidental events that are less likely but still exist. For example, database instance failures can easily cause problems with the core management nodes, thereby affecting the IO path. In the existing technology, for such accidental events that are less likely but still exist, most of them use manual monitoring to implement event management. That is, after such events occur, human intervention is used to resolve the failure. However, human intervention is slow to detect, which seriously affects the normal operation of the service after the event occurs. In addition, the time of event occurrence is uncertain, making it difficult to resolve the failure quickly and promptly. Therefore, an effective solution is urgently needed to solve the above problems.

[0004] Summary of the Invention

[0005] In view of this, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing system, a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program to address technical deficiencies in the prior art.

[0006] According to a first aspect of an embodiment of this specification, a data processing method is provided, which is applied to a cloud storage service client, including:

[0007] receiving a database operation request submitted to a target database;

[0008] When it is determined that the target database is abnormal according to the database operation request, reading the program data recorded in the backup period from the local data file, and reading the disk metadata recorded in the backup period from the storage node;

[0009] Starting a database service program associated with the target database according to the program data, and loading the disk metadata into the memory of the management and control node;

[0010] The database operation task corresponding to the database operation request is executed according to the program startup result and the metadata loading result.

[0011] According to a second aspect of an embodiment of this specification, a data processing system is provided, including a cloud storage service client, a storage node, and a management and control node. The system includes:

[0012] The cloud storage service client is configured to receive a database operation request submitted to a target database; if the target database is determined to be abnormal according to the database operation request, read the program data recorded in the backup cycle from the local data file, and send a metadata acquisition request to the storage node;

[0013] The storage node is configured to send the disk metadata recorded during the backup period to the cloud storage service client in response to the metadata acquisition request;

[0014] The cloud storage service end is also used to start the database service program associated with the target database according to the program data, and load the disk metadata into the management node memory of the management node; and execute the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0015] According to a third aspect of an embodiment of this specification, a data processing device is provided, which is applied to a cloud storage service client, including:

[0016] A receiving module is configured to receive a database operation request submitted to a target database;

[0017] a reading module configured to read program data recorded in a backup cycle from a local data file and disk metadata recorded in a storage node when determining that the target database is abnormal according to the database operation request;

[0018] a loading module configured to start a database service program associated with the target database according to the program data, and to load the disk metadata into a memory of a management and control node;

[0019] The execution module is configured to execute the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0020] According to a fourth aspect of the embodiments of this specification, a computing device is provided, including:

[0021] memory and processor;

[0022] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned data processing method are implemented.

[0023] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned data processing method are implemented.

[0024] According to a sixth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data processing method.

[0025] The data processing method provided in this embodiment is applied to a cloud storage service end. To protect against database anomalies that are less likely but still exist, and to prevent database operation requests submitted by computing nodes from being unable to execute, program data and disk metadata can be backed up during the normal operation of the service. When a database operation request submitted to a target database is received, and the target database is abnormal at this time, the program data recorded in the backup cycle can be directly read from the local data file, and the disk metadata recorded in the backup cycle can be read from the storage node. After obtaining the basic data that can support the normal execution of the database operation request, the database service program associated with the target database can be started based on the program data, and the disk metadata can be loaded into the memory of the control node, so that the database service program can run normally. At the same time, the disk metadata associated with the database operation request can be read from the memory of the control node. Subsequently, the database operation task corresponding to the database operation request can be executed based on the sequence startup result and the metadata loading result. This ensures that the execution of database operation tasks will not be affected by database anomalies at any time, solves the impact of metadata unavailability on the IO link during database anomalies, and thus achieves the purpose of non-intervention automatic recovery, effectively reducing the impact on database operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] FIG1 is a schematic diagram of a data processing method provided by one embodiment of this specification;

[0027] FIG2 is a flow chart of a data processing method provided by one embodiment of this specification;

[0028] FIG3 is a schematic diagram of the structure of a cloud storage service in a data processing method provided in one embodiment of this specification;

[0029] FIG4 is a schematic diagram of the structure of a cloud storage service in another data processing method provided in one embodiment of this specification;

[0030] FIG5 is a flowchart of a data processing method according to an embodiment of the present disclosure;

[0031] FIG6 is a schematic diagram of the structure of a data processing system provided by one embodiment of this specification;

[0032] FIG7 is a schematic structural diagram of a data processing device provided by one embodiment of this specification;

[0033] FIG8 is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0034] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0035] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0036] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0037] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0038] First, the terms involved in one or more embodiments of this specification are explained.

[0039] VM (Virtual Machine): A virtual machine is a server that creates multiple virtual instances that can run an OS.

[0040] TDC: Block storage front-end. The block storage front-end TDC runs on compute nodes. As the entry point for block storage control and data flows, TDC provides complete block storage services.

[0041] In this specification, a data processing method is provided. This specification also relates to a data processing system, a data processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0042] In actual applications, database instance failures, such as the inability to connect the master and standby database networks, or failures in the master and standby database physical devices, will cause problems in the core management and control nodes, which in turn will affect the IO path. Specifically, in the cloud storage management and control environment in the prior art, the persistence and backup of management metadata are mainly implemented in high-availability database instances. For example, in the disk opening process, the block storage front-end TDC needs to first go to the management and control node RiverMaster to obtain the storage cluster information and disk metadata where the disk is located, and then complete the subsequent disk opening process. However, when the database is unavailable, the disk metadata bureau in the management and control node will become unavailable, causing the disk opening failure. Therefore, there is an urgent need for an effective solution to solve the above problems.

[0043] Referring to the schematic diagram shown in FIG1 , the data processing method provided in this embodiment is applied to a cloud storage service end. To protect against database anomalies that are less likely but still exist, and to prevent database operation requests submitted by computing nodes from being unable to execute, program data and disk metadata can be backed up during the normal operation phase of the service. When a database operation request submitted to a target database is received, and the target database is abnormal at this time, the program data recorded in the backup cycle can be directly read from the local static data file, and the disk metadata recorded in the backup cycle can be read from the storage node. After obtaining the basic data that can support the normal execution of the database operation request, the database service program associated with the target database can be started based on the program data, and the disk metadata can be loaded into the memory of the control node, so that the database service program can run normally. At the same time, the disk metadata associated with the database operation request can be read from the memory of the control node. Subsequently, the database operation task corresponding to the database operation request can be executed based on the sequence startup result and the metadata loading result. This ensures that the execution of database operation tasks will not be affected by database anomalies at any time, and solves the impact of metadata unavailability on the IO link during database anomalies, thereby achieving the purpose of non-intervention automatic recovery and effectively reducing the impact on database operations.

[0044] 2 , which shows a flow chart of a data processing method according to an embodiment of this specification. The method is applied to a cloud storage service end and specifically includes the following steps:

[0045] Step S202: receiving a database operation request submitted to a target database.

[0046] The data processing method provided in this embodiment is applied to the cloud storage service end, wherein the cloud storage service end specifically refers to the service end corresponding to the service provider providing the cloud storage service. Accordingly, the template database specifically refers to the relational database that needs to be operated according to the database operation request submitted by the computing node in the current scenario. And the target database is managed by the control node of the cloud storage service end, which is used to perform database operation processing on the target database that meets the operation requirements of the computing node, such as opening processing, releasing disk processing, changing data storage location processing, etc. Correspondingly, the database operation request specifically refers to the operation request submitted by the computing node to the target database when the cloud storage service is running, such as opening operation request, releasing disk operation request, changing data storage location operation request, etc. This embodiment does not make any limitation here.

[0047] Based on this, when a database operation request submitted to a target database is received, it indicates that the computing node needs to operate on the target database at this time. In order to be able to protect against database abnormal events that are less likely but still exist, and to avoid the failure to execute the database operation request submitted by the computing node, the program data and disk metadata can be backed up during the normal operation of the service. In this embodiment, the specific implementation method is as follows:

[0048] When the database service corresponding to the target database is in a running state, determine the target disk corresponding to the target database; read the disk metadata corresponding to the target disk, load the disk metadata into the management and control node memory, and back up to the storage node; determine the database service program corresponding to the database service, and read the global program data corresponding to the database service program; determine the local program data for starting the database service program in the global program data as program data, and load it into the local data file.

[0049] Specifically, the database service refers to a stable, reliable, and elastically scalable online database service provided by the cloud storage service. Through this service, users of the cloud storage service can perform any required operations on the target database. Correspondingly, the running status refers to the state in which the database service is providing services normally. Correspondingly, the target disk refers to the physical storage device corresponding to the target database, including but not limited to solid-state drives, mechanical hard drives, etc. Correspondingly, the disk metadata refers to the system data that describes the characteristics of the disk, which is used to record the location of the disk, creation time, disk space, the cluster to which the disk belongs, and other information. Since the disk is the physical storage space of the database, the database operation request for the target database will be completed by the disk, and the disk metadata is required to execute the request.

[0050] Accordingly, the term "control node memory" specifically refers to the runtime memory corresponding to the control node. Accordingly, the term "storage node" specifically refers to the node used for persistent data storage and for backing up disk metadata to prevent disk metadata loss. Accordingly, the term "database service program" specifically refers to the program on which the database service operates. This program is the front-end representation of the database service, meaning that all functions provided by the database service can be implemented through operations within the database service program. Accordingly, the term "global program data" specifically refers to all program data required by the database service program during runtime, including but not limited to data required for program startup, data required to implement each function within the program, and data required to support normal operation of these functions. Accordingly, the term "local program data" specifically refers to the data necessary for the database service program to start up. This data can support the startup and operation of some basic functions of the database service program. However, data required for some more complex functions within the program does not constitute local program data. In other words, global program data supports the database service program in providing complete and efficient database services, while local program data supports the database service program in providing some basic database services. Accordingly, the term "local data file" specifically refers to a static data file used for persistent storage of program data. This file is deployed on the server side and is not affected by database failures.

[0051] Based on this, in order to support the ability to restore the database at any time when a database anomaly occurs and to support database operations on the target database, the program data and disk metadata required for the operation of the database service can be backed up regularly. That is to say, when the database service corresponding to the target database is in operation, the target disk corresponding to the target database can be determined first; then the disk metadata corresponding to the target disk can be read, and the disk metadata can be backed up by loading the disk metadata into the memory of the management node and backing it up to the storage node. At the same time, for the database service program, the database service program corresponding to the database service can also be determined, and the global program data corresponding to the database service program can be read. On this basis, in order to avoid backing up redundant data while ensuring that the database service program provides basic functions, the local program data for starting the database service program can be determined in the global program data as program data, and loaded into the local data file to complete the backup of the program data. In order to support the normal operation of the database service based on the backed-up program data and disk metadata in the event of a database failure, the normal execution of the database operation task can be achieved.

[0052] In practice, when backing up program data and disk metadata, to reduce resource consumption, backups can be performed according to a set backup cycle, such as storing program data in a local data file every hour. Alternatively, backups can be performed based on data freshness, such as triggering a disk metadata backup whenever the disk metadata changes. This ensures that program data and disk metadata backups are available to handle database anomalies. Furthermore, during the disk metadata backup phase, to support external data query services while eliminating database query operations, it is added to the control node's memory, improving service response time. Furthermore, this ensures that metadata query services remain available even when the database service is unavailable, supporting normal service operation. In practice, since the data required to start the database service program is modified infrequently and the online production environment is typically not modified after deployment, the backup cycle can be set as needed to ensure data integrity and accuracy.

[0053] In summary, in order to support database operations on computing nodes in the event of database anomalies, program data and disk metadata can be backed up in advance according to the backup cycle while the database service is running, so that the backup data can be called to continue to provide stable database services when the database anomalies occur.

[0054] On this basis, considering that the program data and disk metadata are backed up when the service is running, and the backup result may be inconsistent with the disk metadata after the change, for example, a new disk opening operation occurs at the backup time node, which will result in an incomplete backup. Therefore, in order to ensure that the disk metadata in the memory of the control node is the latest and complete disk metadata, an identifier comparison can be performed after the backup is completed; in this embodiment, the specific implementation method is as follows:

[0055] Read the initial disk metadata corresponding to the target disk, determine the metadata identifier corresponding to the initial disk metadata, and determine the backup metadata identifier corresponding to the backup disk metadata in the management and control node memory; if the backup metadata identifier is different from the metadata identifier, determine the disk metadata in the initial disk metadata based on the backup metadata identifier and the metadata identifier, and execute the step of loading the disk metadata into the management and control node memory.

[0056] Specifically, the initial disk metadata refers to the disk metadata currently read. Accordingly, the metadata identifier refers to the unique identifier corresponding to the initial disk metadata. In actual applications, disk metadata consists of multiple metadata items, each of which corresponds to a metadata identifier. Metadata identifiers are set incrementally. Therefore, the metadata identifier of the initial disk metadata is the identifier corresponding to the last metadata item in the initial disk metadata. Accordingly, the backup metadata identifier is the identifier corresponding to the last metadata item in the disk metadata stored in the control node's memory.

[0057] Based on this, when performing a disk metadata backup operation, it is considered that the disk metadata backed up in the memory of the control node may not be the latest disk metadata. For example, if the disk metadata changes at the backup time node, the backed-up disk metadata will not be the latest version of the disk metadata. Therefore, the initial disk metadata corresponding to the target disk can be read first, and the metadata identifier corresponding to the initial disk metadata can be determined, as well as the backup metadata identifier corresponding to the backup disk metadata in the memory of the control node. By comparing the metadata identifier and the backup metadata identifier, it can be determined whether the two are completely synchronized. If the backup metadata identifier is different from the metadata identifier, it means that the ID value corresponding to the metadata identifier is greater than the ID value corresponding to the backup metadata identifier, which means that the backup of all disk metadata has not been completed at this time. Therefore, the disk metadata can be determined in the initial disk metadata based on the backup metadata identifier and the metadata identifier, that is, the disk metadata that needs to be backed up can be determined. Then, step S204 is executed to ensure that the data operation request is executed normally at the abnormal node of the database.

[0058] In actual applications, during the startup phase of the database service program, a thread can be added to periodically load disk metadata into memory. Each time the disk metadata is loaded into the control node's memory, a check can be performed to ensure that all disk metadata has been updated. This ensures that the disk metadata in the control node's memory always maintains the latest version of the disk metadata to address database anomalies. Furthermore, during the database service program's runtime, a thread can be used to periodically retrieve specified data from the database and back it up locally. For example, program data can be backed up to a local static data file every hour to ensure the program can operate normally during database anomalies.

[0059] This embodiment uses the disk opening process as an example to illustrate the data processing method. Referring to the schematic diagram shown in FIG3 (a), the processing of the disk opening process requires the block storage front-end TDC of the computing node to obtain the storage cluster information and disk metadata of the disk from the control node. In order to support the normal operation of the disk opening process at any time, the necessary data and disk metadata required for the operation of the database service program can be backed up at the database service running node. Specifically, the escape preparation module can load the full disk metadata corresponding to the database into the thread in the cache during the startup phase of the database service program, and check whether all the disk metadata have been backed up after each backup is completed. If not, continue to select the remaining disk metadata for backup until the disk metadata of the database is consistent with the ID of the disk metadata in the memory, and the backup of the disk metadata is completed. At the same time, the startup data necessary for the database service program can be obtained every hour, such as the basic information Azone_id and Azone_name in the Azone table, Cluster_id and Azone_id in the Cluster_info table, the number of cluster disks (device_count), the cluster disk capacity (total_size), and the cluster disk sales capacity (sale_size). This data is stored in local static data files. In the event of a database exception, disk startup processing can be continued based on the backed-up disk metadata and startup data.

[0060] In summary, by comparing metadata identifiers after each disk metadata backup process, the backup results can be tested, and the disk metadata backup can be synchronized based on the test results to support database anomalies at any stage without affecting subsequent normal database operations.

[0061] Step S204 : when it is determined that the target database is abnormal according to the database operation request, program data recorded in the backup period is read from the local data file, and disk metadata recorded in the backup period is read from the storage node.

[0062] Specifically, based on the above-mentioned receipt of the database operation request, if it is determined that the target database is abnormal when performing the corresponding operation on the target database according to the database operation request, the database operation cannot be continued at the current moment. For example, when performing the opening operation, the network connection of the database is interrupted, or the physical device of the primary and standby databases fails, resulting in the database operation being unable to continue through the existing primary and standby database mode. Therefore, in order to avoid affecting the operation of the database service, the program data required for starting the database service program can be read from the local data file where the program data has been backed up, and the disk metadata recorded in the backup cycle can be read in the storage node, so that the program data and disk metadata recorded in the backup cycle can continue to support the operation of the database service, thereby supporting the normal execution of the database operation request.

[0063] Among them, the target database abnormality specifically refers to the target database being unable to work based on the database service at the current time node, such as the target database network connection being interrupted, or the physical device of the primary and standby databases failing, which in turn causes the database operation to be unable to complete the corresponding operation based on the primary data and the standby database. Correspondingly, the backup cycle specifically refers to the cycle for backing up program data and disk metadata. For example, if the program data is backed up once every hour, the backup cycle is one hour, and if the disk metadata is backed up once each time the disk metadata changes, the backup cycle is each time the disk metadata changes. Correspondingly, the storage node specifically refers to the underlying data storage node that persists the disk metadata. It should be noted that the relevant description of the program data and disk metadata can be found in the description in the above embodiment, and this embodiment will not be elaborated on here.

[0064] Furthermore, considering that the database service program may be in the startup phase or the running phase when providing services to the computing nodes, the target database may fail in different phases. In order to accurately detect the failure of the target database in any state, different methods can be used. In this embodiment, the specific implementation method is as follows:

[0065] In response to the database operation request, the program phase corresponding to the database service program is detected; when the database service program is in the program startup phase, the target database is connected according to a set time interval and a set number of detections. If the connection fails, the target database is determined to be abnormal, and the step of reading the program data recorded in the backup cycle in the local data file is executed; when the database service program is in the program running phase, the connection pool corresponding to the target database is determined, and the threads in the connection pool are detected. If the detection fails, the target database is determined to be abnormal, and the step of reading the program data recorded in the backup cycle in the local data file is executed.

[0066] Specifically, the program startup phase specifically refers to the phase when the database service program should be in the startup state. Correspondingly, the program running phase specifically refers to the phase when the database service program should be in the running state. Correspondingly, setting the time interval specifically refers to setting the time interval according to actual needs, such as 0.1-0.5s, etc. Correspondingly, setting the number of detections specifically refers to the number of consecutive connectivity detections set according to actual needs, such as 5 times, or 10 times, etc. Correspondingly, the connection pool specifically refers to the thread pool composed of threads associated with the target database. Correspondingly, detecting the thread is to detect whether the thread can connect to the database normally.

[0067] Based on this, after receiving a database operation request submitted to a database, in order to ensure that the target database can support the database operation, the target database can be checked for faults. At this time, the program stage corresponding to the database service program can be checked in response to the database operation request; if the database service program is in the program startup stage, it means that the database service program has not yet started, but has been started. At this time, the database connection pool has not been fully started and cannot provide services. Therefore, the target database can be connected according to the set time interval and the set number of tests. If the connection fails, it means that the target database cannot be connected and cannot support the database operation. Therefore, it is determined that the target database is abnormal, and the step of reading the program data recorded in the backup cycle in the local data file can be executed.

[0068] When the database service program is in the program running stage, it means that the database service program should be in working state. At this time, the connection pool corresponding to the target database can be determined, and the threads in the connection pool can be tested. If the test fails, it means that the connection pool cannot connect to the database, and then the target database is determined to be abnormal. Then, the step of reading the program data recorded in the backup cycle in the local data file can be executed, so that the database operation request can be continued based on the backed up program data and disk metadata.

[0069] In other words, during the normal operation of the target database, database availability can be checked. If the database experiences network connectivity issues, downtime on both the primary and backup nodes, or issues with the database storage media, this indicates a database anomaly, requiring backup data to ensure normal operation. In practical applications, during the startup phase of the database service program, a periodic thread can be added to regularly check database availability. This check is performed as follows: During the database service program startup phase, since the database connection pool has not yet been activated, a direct connection to the database can be performed. If the database connection is unavailable or the database ping fails, a continuous check can be performed n times at a set interval. If no connection is found, this indicates a database anomaly and triggers an abort process, requiring database operations to be completed based on backup data. During the database service program's runtime phase, the health of the database connection pool can be checked, specifically whether threads in the connection pool are connecting to the database normally. If this fails m times in a row, this indicates a database anomaly and triggers an abort process, requiring database operations to be completed based on backup data.

[0070] Continuing with the previous example, before the disk opening process, to ensure normal disk opening, the database can be checked. Referring to the schematic diagram shown in Figure 3 (b), when the database service program is in the startup phase, the escape detection module can directly connect to the database. If the database connection fails or the database ping fails, it can perform 10 consecutive checks at intervals of 0.1-0.5 seconds. If no connection is made to the database, it indicates that the database is abnormal, triggering the escape handling module to operate. This means that the disk opening process can continue to proceed normally based on the backed-up disk metadata and necessary startup data. When the database service program is in the running phase, it can check whether the threads in the database connection pool are connected to the database. If the database connection fails 10 times in a row, it indicates that the database is abnormal, triggering the escape handling module to operate. This means that the disk opening process can continue to proceed normally based on the backed-up disk metadata and necessary startup data. Furthermore, if the database is detected to have recovered, the automatic recovery module can trigger an automatic recovery process to restore the database connection to a normal state.

[0071] Furthermore, if a database anomaly is detected and the process is restarted, it indicates that the database service program needs to be restarted and disk metadata needs to be provided. Therefore, the necessary startup data for the database service program can be read from the local static data file. At the same time, disk metadata can also be read from the underlying storage node to support the compute node's block storage front-end TDC to obtain disk metadata and proceed with the subsequent disk opening process.

[0072] In summary, by performing different detection methods on the target database at different program stages, the accuracy of detecting database anomalies can be ensured, so that the subsequent detection results can trigger the escape mechanism, thereby supporting the database to complete database operations in the event of anomalies and ensuring the stability of database services.

[0073] Step S206: starting a database service program associated with the target database according to the program data, and loading the disk metadata into the memory of the management and control node.

[0074] Specifically, in the case of the aforementioned determination of a database anomaly, after obtaining the program data and disk metadata recorded during the backup cycle, further, in order to support the normal operation of the database service and ensure that the database operation results meet the requirements, the database service program associated with the target database can be started based on the program data, and the disk metadata can be loaded into the control node memory. This allows the corresponding database operation to be completed based on the started database service program and the disk metadata loaded into the control node memory. The relevant description of the database service program and the control node memory can be found in the description of the above embodiment, and this embodiment will not be elaborated on here.

[0075] Furthermore, after obtaining the program data and disk metadata, they can be loaded into the memory to complete the startup of the program and the execution of the database operation request. In this embodiment, the specific implementation is as follows:

[0076] The program data is loaded into the server memory, and the database service program associated with the target database is started according to the loading result; the control node associated with the database service program is determined, and the disk metadata is loaded into the control node memory of the control node.

[0077] Specifically, server-side memory refers to the memory used to load data when running a database service program. Therefore, during program startup, program data can be loaded into server-side memory. Based on the program data loaded into server-side memory, the database service program associated with the target database can be started, enabling the database service program to provide basic functionality. Simultaneously, the control node associated with the database service program can be determined, and disk metadata can be loaded into the control node memory of the control node, providing the database service program with the metadata foundation for performing database operations. Subsequently, the database service program and disk metadata can be combined to complete corresponding database operations.

[0078] Furthermore, considering that the storage nodes and the control nodes may store data in different formats, the disk metadata in the storage nodes needs to be converted when loading the disk metadata into the control node memory. In this embodiment, the specific implementation is as follows:

[0079] Acquire a data conversion relationship between the storage node and the management and control node; convert the disk metadata into target disk metadata according to the data conversion relationship; and load the target disk metadata into the management and control node memory of the management and control node.

[0080] Specifically, a data conversion relationship refers to a conversion relationship that converts the data format stored in a storage node into the data format stored in the corresponding control node. Accordingly, the target disk metadata is the disk metadata converted according to the data conversion relationship and is in the same storage format as the data on the control node.

[0081] Based on this, after reading the disk metadata recorded in the backup cycle from the storage node, further, considering that the storage node and the control node may have different data storage formats, the data conversion relationship between the storage node and the control node can be obtained; then the disk metadata is converted into the target disk metadata according to the data conversion relationship; after the disk metadata is converted into the storage format of the corresponding control node, the target disk metadata is loaded into the control node memory of the control node, and the corresponding database operations can continue.

[0082] In summary, after reading the disk metadata from the storage node, in order to ensure that it can be used normally after being loaded into the management node memory, the disk metadata can be converted into target disk metadata corresponding to the management node data storage format according to the data conversion relationship, thereby ensuring that subsequent database operation tasks can be executed.

[0083] In addition, after the database service program is started and the disk metadata is loaded, an integrity check can be performed to ensure that the disk metadata loaded into the management node memory is complete, thereby ensuring that the database operation request can be executed normally. In this embodiment, the specific implementation method is as follows:

[0084] The first disk information corresponding to the target database is read in the program data, and the second disk information is determined according to the disk metadata loaded in the memory of the control node; the first disk information and the second disk information are compared; if it is determined according to the comparison result that the disk metadata loaded in the memory of the control node is incomplete, supplementary disk metadata is read in the storage node according to the first disk information and the second disk information; the supplementary disk metadata is loaded into the memory of the control node, and the steps of executing the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result are executed.

[0085] Specifically, the first disk information refers to the disk number information of the disk metadata corresponding to the target database; correspondingly, the second disk information refers to the disk number information of the disk metadata loaded in the control node; if the first disk information and the second disk information are inconsistent, it means that the disk metadata in the control node's memory is incomplete. Correspondingly, the supplementary disk metadata refers to the disk metadata read from the storage node to supplement the disk metadata already loaded in the control node, ensuring that after the control node loads the supplementary disk metadata, the disk metadata it loads is consistent with the disk metadata corresponding to the target database, thereby ensuring that database operation tasks can be executed normally.

[0086] Based on this, after entering the operation stage, in order to ensure the normal operation of the service and thus complete the database operation task corresponding to the database operation request, the integrity of the disk metadata can also be checked; that is, the first disk information corresponding to the target database can be read in the program data, and the second disk information can be determined based on the disk metadata loaded in the control node memory; at this time, the first disk information and the second disk information will be compared; if it is determined according to the comparison result that the first disk information and the second disk information are inconsistent, it means that the number of disks of the target database is greater than the disk data corresponding to the disk metadata in the control node memory, and then it means that the disk metadata loaded in the control node memory is incomplete. Therefore, in order to ensure the integrity of the disk metadata in the control node memory, supplementary disk metadata can be read in the storage node based on the first disk information and the second disk information; then the supplementary disk metadata is loaded into the control node memory, and step S208 is executed.

[0087] During specific implementation, when it is determined that the target database is abnormal, in order to ensure that the database service can continue to provide services, the disk metadata can be restored from the underlying storage node to the memory during the operation phase, thereby ensuring that the existing IO service is not interrupted. That is to say, in the startup phase, if it is detected that the database is unavailable and the escape process is triggered, the program data can be loaded into the memory from the local static data file, and then the database service program can be started according to the program data loaded into the memory, and the database service program can be made to work normally. After entering the operation phase, it can be determined whether the disk metadata in the memory of the control node is complete, such as by comparing the number of disks in the cluster table in the program data with the number of disks in the memory. If it is less than the cluster disk data, the disk metadata is incomplete. At this time, all the disk metadata can be obtained by polling from the underlying storage node, and based on the data conversion relationship between the storage node and the control node, the disk metadata can be restored to the disk metadata required by the control node, thereby supporting the normal operation of the database service.

[0088] Continuing with the previous example, referring to the schematic diagram shown in FIG4 , the necessary startup data required to start the database service program can be first read from a local static data file, loaded into memory, and then started according to the necessary startup program in memory. During the program execution phase, the completeness of the disk metadata in memory is checked. If incomplete, the underlying storage nodes can be polled to obtain all the disk metadata. All the disk metadata read from the storage nodes can then be restored to the disk metadata of the corresponding control node according to the preset service logic. For example, the key in the RiverMaster device table of the control node is device_key, which is a two-part cluster_id + device_id, while the key in the device table of the underlying storage is a one-part volume_id. At this point, after reading the disk metadata stored in the underlying storage node, it can be converted into the disk metadata of the corresponding storage node and then written to the memory of the control node. This allows the block storage front-end TDC of the compute node to obtain the disk metadata and proceed with the subsequent disk opening process.

[0089] In summary, before performing database operations, the restored disk metadata will be checked for integrity to ensure that database operations are performed only when all disk metadata has been restored, to avoid the operation results not meeting the requirements and affecting the operation of the front-end service.

[0090] Step S208: executing the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0091] Specifically, after the database service program is started according to the program data, it indicates that the database service program can now operate normally. After the disk metadata is loaded into the memory of the control node, it further indicates that the database service corresponding to the target database can provide the corresponding database operation service. Therefore, based on the program startup results and the metadata loading results, the database operation task corresponding to the database operation request can be executed to complete the database operation processing for the target database. Among them, the database operation task specifically refers to a task that performs any type of operation on the target database, including but not limited to the operation task of opening the disk, the operation task of releasing the disk, the operation task of changing the data storage location, etc.

[0092] Continuing with the above example, if the database operation program can run normally and the disk metadata in the memory of the control node is complete, the disk metadata in the memory of the control node can be sent to the block storage front-end TDC. The block storage front-end TDC can then use the database service program and the obtained disk metadata to open storage space that meets the usage requirements in Disk 1 in the target storage cluster where the target disk is located according to the disk opening requirements. This storage space corresponds to the physical storage device - Disk 1, and the storage space size is 1T to facilitate downstream use.

[0093] Furthermore, after the database operation task is completed, in order to ensure that the cloud storage service client can provide complete database services, the mode can be switched; in this embodiment, the specific implementation method is as follows:

[0094] Detect whether the target database is in disaster recovery mode; if so, switch the disaster recovery mode to baseline mode, and run the target database service associated with the target database according to the mode switching result; if not, rebuild the target connection pool for the target database, and run the target database service associated with the target database according to the connection pool reconstruction result.

[0095] Specifically, disaster recovery mode refers to a model in which the target database continues to support database operations even under abnormal conditions. Accordingly, baseline mode refers to the mode in which the target database operates normally. It should be noted that disaster recovery mode may provide partial functionality, while baseline mode provides full functionality. For a description of the target database services, please refer to the description of the database services above, and this embodiment does not impose any limitations thereon.

[0096] Based on this, when the database returns to normal, in order to provide complete database services, you can check whether the target database is in disaster recovery mode. If so, it means that the target database cannot provide complete database services at the current stage. Therefore, you can switch the disaster recovery mode to baseline mode and run the target database services associated with the target database based on the mode switch result to provide complete database services. If not, it means that the target database cannot provide database services at the current stage. At this time, you can rebuild the target connection pool for the target database and run the target database services associated with the target database based on the connection pool reconstruction result to provide complete database services.

[0097] In other words, if the target database is currently in disaster recovery mode (i.e., the control node restarts after a database exception), it can be switched from disaster recovery mode to normal mode to provide stable and complete database services. If it is not in disaster recovery mode, the target database connection pool can be re-established, and stable and complete database services can be provided based on the connection pool establishment results.

[0098] In summary, by switching the mode of the target database, the target database can be switched to the baseline mode after returning to normal, thereby ensuring that the database service can provide complete services.

[0099] In addition, considering that a database failure may occur in the running state of the database service program, it is not necessary to restart the database service program at this time. It is only necessary to load the disk metadata into the memory to support the database service program to continue to execute the database operation task. In this embodiment, the specific implementation method is as follows:

[0100] If the target database is determined to be abnormal based on the database operation request and the database service program is operating normally, the storage node reads the disaster recovery disk metadata recorded during the backup cycle; the disaster recovery disk metadata is loaded into the memory of the control node; and the database operation task corresponding to the database operation request is executed based on the running database service program and the metadata loading result. The description of the disaster recovery disk metadata can be found in the description of the disk metadata in the above embodiment, and is not further elaborated in this embodiment.

[0101] Based on this, if the target database is abnormal based on a database operation request and the database service program is functioning normally, this indicates that the target database is abnormal but the database service program has not been restarted, which means that the database service program does not need to be restarted. To support the execution of the database operation request, only the disk metadata can be restored. In other words, the disaster recovery disk metadata recorded during the backup cycle can be read from the storage node; this metadata can then be loaded into the memory of the control node. At this point, the database operation task corresponding to the database operation request can be executed directly based on the running database service program and the metadata loading results.

[0102] In summary, the data processing method provided in this embodiment can achieve the degradation of the cloud storage management link and the backup of metadata, and at the same time solve the impact on the IO link when the metadata is unavailable, ensuring that the IO link can still operate normally when the original metadata is unavailable.

[0103] The data processing method provided in this embodiment is applied to a cloud storage service end. To protect against database anomalies that are less likely but still exist, and to prevent database operation requests submitted by computing nodes from being unable to execute, program data and disk metadata can be backed up during the normal operation of the service. When a database operation request submitted to a target database is received, and the target database is abnormal at this time, the program data recorded in the backup cycle can be directly read from the local data file, and the disk metadata recorded in the backup cycle can be read from the storage node. After obtaining the basic data that can support the normal execution of the database operation request, the database service program associated with the target database can be started based on the program data, and the disk metadata can be loaded into the memory of the control node, so that the database service program can run normally. At the same time, the disk metadata associated with the database operation request can be read from the memory of the control node. Subsequently, the database operation task corresponding to the database operation request can be executed based on the sequence startup result and the metadata loading result. This ensures that the execution of database operation tasks will not be affected by database anomalies at any time, solves the impact of metadata unavailability on the IO link during database anomalies, and thus achieves the purpose of non-intervention automatic recovery, effectively reducing the impact on database operations.

[0104] The following further illustrates the data processing method provided in this specification using the application of the data processing method in a cloud storage service opening scenario, with reference to FIG5 . FIG5 shows a flowchart of a data processing method provided in one embodiment of this specification, specifically including the following steps.

[0105] Step S502 : When the database service corresponding to the target database is in a running state, determine the target disk corresponding to the target database.

[0106] Step S504 , reading the initial disk metadata corresponding to the target disk, determining the metadata identifier corresponding to the initial disk metadata, and determining the backup metadata identifier corresponding to the backup disk metadata in the memory of the control node.

[0107] Step S506: If the backup metadata identifier is different from the metadata identifier, the disk metadata is determined in the initial disk metadata according to the backup metadata identifier and the metadata identifier, and the disk metadata is loaded into the management node memory and backed up to the storage node.

[0108] Step S508: Determine the database service program corresponding to the database service, and read the global program data corresponding to the database service program.

[0109] Step S510 : determining local program data for starting a database service program in the global program data as program data, and loading the data into a local data file.

[0110] Step S512: receiving a database operation request submitted to the target database.

[0111] Step S514 , when it is determined that the target database is abnormal according to the database operation request, the program data recorded in the backup period is read from the local data file, and the disk metadata recorded in the backup period is read from the storage node.

[0112] Specifically, in response to a database operation request, the program phase corresponding to the database service program is detected; when the database service program is in the program startup phase, the target database is connected according to the set time interval and the set number of detections. If the connection fails, it is determined that the target database is abnormal, and the step of reading the program data recorded in the backup cycle in the local data file is executed.

[0113] When the database service program is in the program running stage, the connection pool corresponding to the target database is determined, and the threads in the connection pool are detected. If the detection fails, it is determined that the target database is abnormal, and the step of reading the program data recorded in the backup cycle in the local data file is executed.

[0114] Furthermore, if a database anomaly is detected and the process is restarted, it indicates that the database service program needs to be restarted and disk metadata needs to be provided. Therefore, the necessary startup data for the database service program can be read from the local static data file. At the same time, disk metadata can also be read from the underlying storage node to support the compute node's block storage front-end TDC to obtain disk metadata and proceed with the subsequent disk opening process.

[0115] Step S516: Load the program data into the server memory, and start the database service program associated with the target database according to the loading result.

[0116] Step S518: determine the management and control node associated with the database service program, and obtain the data conversion relationship between the storage node and the management and control node.

[0117] Step S520 : converting the disk metadata into target disk metadata according to the data conversion relationship, and loading the target disk metadata into the management and control node memory of the management and control node.

[0118] Step S522: Read the first disk information corresponding to the target database in the program data, and determine the second disk information according to the disk metadata loaded in the memory of the management and control node.

[0119] Step S524, compare the first disk information and the second disk information, and if it is determined based on the comparison result that the disk metadata loaded in the memory of the management and control node is incomplete, read the supplementary disk metadata in the storage node based on the first disk information and the second disk information.

[0120] Step S526: Load the supplementary disk metadata into the management node memory, and execute the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0121] Furthermore, it is possible to detect whether the target database is in disaster recovery mode; if so, switch the disaster recovery mode to baseline mode, and run the target database service associated with the target database according to the mode switching result; if not, rebuild the target connection pool for the target database, and run the target database service associated with the target database according to the connection pool reconstruction result.

[0122] In summary, in order to be able to protect against database abnormal events that are less likely but still exist, and to avoid the inability to execute database operation requests submitted by computing nodes, the program data and disk metadata can be backed up during the normal operation of the service. When a database operation request submitted to the target database is received, and the target database is abnormal at this time, the program data recorded in the backup cycle can be directly read from the local data file, and the disk metadata recorded in the backup cycle can be read in the storage node to obtain the basic data that can support the normal execution of the database operation request. The database service program associated with the target database can be started according to the program data, and the disk metadata can be loaded into the memory of the control node at the same time, so that the database service program can run normally. At the same time, the disk metadata associated with the database operation request can be read from the memory of the control node. Subsequently, the database operation task corresponding to the database operation request can be executed according to the sequence startup result and the metadata loading result. It is achieved that at any time, the execution of the database operation task will not be affected by the abnormality of the database, and the impact of the unavailability of metadata on the IO link when the database is abnormal is solved, thereby achieving the purpose of automatic recovery without intervention, and effectively reducing the impact on database operations.

[0123] Corresponding to the above method embodiments, this specification also provides a data processing system embodiment. Figure 6 shows a schematic diagram of the structure of a data processing system provided in one embodiment of this specification. As shown in Figure 6, the data processing system 600 includes a cloud storage service end 610, a storage node 620, and a control node 630. The system includes:

[0124] The cloud storage service client 610 is configured to receive a database operation request submitted to a target database; if the target database is determined to be abnormal according to the database operation request, read the program data recorded in the backup cycle from the local data file, and send a metadata acquisition request to the storage node;

[0125] The storage node 620 is configured to send the disk metadata recorded in the backup period to the cloud storage service client in response to the metadata acquisition request;

[0126] The cloud storage service end 610 is also used to start the database service program associated with the target database according to the program data, and load the disk metadata into the management node memory of the management node 630; and execute the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0127] In an optional embodiment, before the step of receiving a database operation request submitted to a target database is executed, it also includes: when the database service corresponding to the target database is in a running state, determining the target disk corresponding to the target database; reading the disk metadata corresponding to the target disk, loading the disk metadata into the management and control node memory, and backing up to the storage node; determining the database service program corresponding to the database service, and reading the global program data corresponding to the database service program; determining the local program data for starting the database service program in the global program data as program data, and loading it into the local data file.

[0128] In an optional embodiment, the reading of the disk metadata corresponding to the target disk and the loading of the disk metadata into the management and control node memory include: reading the initial disk metadata corresponding to the target disk, determining the metadata identifier corresponding to the initial disk metadata, and determining the backup metadata identifier corresponding to the backup disk metadata in the management and control node memory; when the backup metadata identifier is different from the metadata identifier, determining the disk metadata in the initial disk metadata based on the backup metadata identifier and the metadata identifier, and executing the step of loading the disk metadata into the management and control node memory.

[0129] In an optional embodiment, when it is determined that the target database is abnormal according to the database operation request, the program data recorded in the backup cycle is read in the local data file, including: detecting the program stage corresponding to the database service program in response to the database operation request; when the database service program is in the program startup stage, connecting to the target database according to a set time interval and a set number of detections, if the connection fails, determining that the target database is abnormal, and executing the step of reading the program data recorded in the backup cycle in the local data file; when the database service program is in the program running stage, determining the connection pool corresponding to the target database, and detecting the threads in the connection pool, if the detection fails, determining that the target database is abnormal, and executing the step of reading the program data recorded in the backup cycle in the local data file.

[0130] In an optional embodiment, before executing the database operation task step corresponding to the database operation request according to the program startup result and the metadata loading result, it also includes: reading the first disk information corresponding to the target database in the program data, and determining the second disk information according to the disk metadata loaded in the control node memory; comparing the first disk information and the second disk information; when it is determined according to the comparison result that the disk metadata loaded in the control node memory is incomplete, reading supplementary disk metadata in the storage node according to the first disk information and the second disk information; loading the supplementary disk metadata into the control node memory, and executing the database operation task step corresponding to the database operation request according to the program startup result and the metadata loading result.

[0131] In an optional embodiment, starting the database service program associated with the target database based on the program data, and loading the disk metadata into the management and control node memory, includes: loading the program data into the server memory, and starting the database service program associated with the target database based on the loading result; determining the management and control node associated with the database service program, and loading the disk metadata into the management and control node memory of the management and control node.

[0132] In an optional embodiment, loading the disk metadata into the control node memory of the control node includes: obtaining a data conversion relationship between the storage node and the control node; converting the disk metadata into target disk metadata according to the data conversion relationship; and loading the target disk metadata into the control node memory of the control node.

[0133] In an optional embodiment, after the step of receiving a database operation request submitted to a target database is executed, it also includes: determining that the target database is abnormal based on the database operation request and the database service program is normal, reading the disaster recovery disk metadata recorded in the backup cycle in the storage node; loading the disaster recovery disk metadata into the management node memory; and executing the database operation task corresponding to the database operation request based on the running status of the database service program and the metadata loading result.

[0134] In an optional embodiment, after executing the database operation task step corresponding to the database operation request according to the program startup result and the metadata loading result, it also includes: detecting whether the target database is in disaster recovery mode; if so, switching the disaster recovery mode to the baseline mode, and running the target database service associated with the target database according to the mode switching result; if not, rebuilding the target connection pool for the target database, and running the target database service associated with the target database according to the connection pool reconstruction result.

[0135] The data processing system provided in this embodiment ensures that the execution of database operation tasks will not be affected by database anomalies at any time, and solves the impact of metadata unavailability on IO links during database anomalies, thereby achieving the purpose of non-intervention automatic recovery and effectively reducing the impact on database operations.

[0136] The above is a schematic diagram of a data processing system according to this embodiment. It should be noted that the technical solution of the data processing system and the technical solution of the above-mentioned data processing method are based on the same concept. For details not described in detail in the technical solution of the data processing system, please refer to the description of the technical solution of the above-mentioned data processing method.

[0137] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment. FIG7 shows a schematic diagram of the structure of a data processing device provided in one embodiment of this specification. As shown in FIG7, the device includes:

[0138] The receiving module 702 is configured to receive a database operation request submitted to a target database;

[0139] The reading module 704 is configured to read the program data recorded in the backup cycle from the local data file and read the disk metadata recorded in the backup cycle from the storage node when determining that the target database is abnormal according to the database operation request;

[0140] A loading module 706 is configured to start a database service program associated with the target database according to the program data, and load the disk metadata into the memory of the management and control node;

[0141] The execution module 708 is configured to execute the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0142] In an optional embodiment, the device further includes:

[0143] The backup module is configured to, when the database service corresponding to the target database is in a running state, determine the target disk corresponding to the target database; read the disk metadata corresponding to the target disk, load the disk metadata into the management and control node memory, and back up to the storage node; determine the database service program corresponding to the database service, and read the global program data corresponding to the database service program; determine the local program data for starting the database service program in the global program data as program data, and load it into the local data file.

[0144] In an optional embodiment, the backup module is further configured to:

[0145] Read the initial disk metadata corresponding to the target disk, determine the metadata identifier corresponding to the initial disk metadata, and determine the backup metadata identifier corresponding to the backup disk metadata in the management and control node memory; if the backup metadata identifier is different from the metadata identifier, determine the disk metadata in the initial disk metadata based on the backup metadata identifier and the metadata identifier, and execute the step of loading the disk metadata into the management and control node memory.

[0146] In an optional embodiment, the reading module 704 is further configured to:

[0147] In response to the database operation request, the program phase corresponding to the database service program is detected; when the database service program is in the program startup phase, the target database is connected according to a set time interval and a set number of detections. If the connection fails, the target database is determined to be abnormal, and the step of reading the program data recorded in the backup cycle in the local data file is executed; when the database service program is in the program running phase, the connection pool corresponding to the target database is determined, and the threads in the connection pool are detected. If the detection fails, the target database is determined to be abnormal, and the step of reading the program data recorded in the backup cycle in the local data file is executed.

[0148] In an optional embodiment, the device further includes:

[0149] A comparison module is configured to read the first disk information corresponding to the target database in the program data, and determine the second disk information based on the disk metadata loaded in the memory of the control node; compare the first disk information with the second disk information; if it is determined based on the comparison result that the disk metadata loaded in the memory of the control node is incomplete, read supplementary disk metadata in the storage node based on the first disk information and the second disk information; load the supplementary disk metadata into the memory of the control node, and execute the steps of executing the database operation task corresponding to the database operation request based on the program startup result and the metadata loading result.

[0150] In an optional embodiment, the loading module 706 is further configured to:

[0151] The program data is loaded into the server memory, and the database service program associated with the target database is started according to the loading result; the control node associated with the database service program is determined, and the disk metadata is loaded into the control node memory of the control node.

[0152] In an optional embodiment, the loading module 706 is further configured to:

[0153] Acquire a data conversion relationship between the storage node and the management and control node; convert the disk metadata into target disk metadata according to the data conversion relationship; and load the target disk metadata into the management and control node memory of the management and control node.

[0154] In an optional embodiment, the device further includes:

[0155] The disaster recovery module is configured to determine that the target database is abnormal based on the database operation request and that the database service program is normal, read the disaster recovery disk metadata recorded in the backup cycle in the storage node; load the disaster recovery disk metadata into the management node memory; and execute the database operation task corresponding to the database operation request based on the database service program in the running state and the metadata loading result.

[0156] In an optional embodiment, the device further includes:

[0157] The detection mode module is configured to detect whether the target database is in disaster recovery mode; if so, switch the disaster recovery mode to the baseline mode, and run the target database service associated with the target database according to the mode switching result; if not, rebuild the target connection pool for the target database, and run the target database service associated with the target database according to the connection pool reconstruction result.

[0158] The data processing device provided in this embodiment ensures that the execution of database operation tasks will not be affected by database anomalies at any time, and solves the impact of metadata unavailability on IO links during database anomalies, thereby achieving the purpose of non-intervention automatic recovery and effectively reducing the impact on database operations.

[0159] The above is a schematic diagram of a data processing device according to this embodiment. It should be noted that the technical solution of the data processing device and the technical solution of the above-mentioned data processing method are based on the same concept. For details not described in detail in the technical solution of the data processing device, please refer to the description of the technical solution of the above-mentioned data processing method.

[0160] Figure 8 shows a block diagram of a computing device 800 according to one embodiment of this specification. Components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0161] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0162] In one embodiment of the present specification, the aforementioned components of computing device 800 and other components not shown in FIG8 may also be connected to each other, for example, via a bus. It should be understood that the computing device structure block diagram shown in FIG8 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0163] Computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 800 may also be a mobile or stationary server.

[0164] The processor 820 is configured to execute the following computer-executable instructions, which implement the steps of the above-mentioned data processing method when executed by the processor.

[0165] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned data processing method are of the same concept. For details not described in detail in the technical scheme of the computing device, please refer to the description of the technical scheme of the above-mentioned data processing method.

[0166] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which implement the steps of the above-mentioned data processing method when executed by a processor.

[0167] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned data processing method are based on the same concept. For details not described in detail in the technical scheme of the storage medium, please refer to the description of the technical scheme of the above-mentioned data processing method.

[0168] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data processing method.

[0169] The above is an illustrative solution of a computer program of this embodiment. It should be noted that the technical solution of the computer program and the technical solution of the above-mentioned data processing method are of the same concept. For details not described in detail in the technical solution of the computer program, please refer to the description of the technical solution of the above-mentioned data processing method.

[0170] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0171] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0172] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0173] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0174] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, applied to a cloud storage service end, comprising: receiving a database operation request submitted to a target database; When it is determined that the target database is abnormal according to the database operation request, program data recorded in the backup cycle is read from a local data file, and disk metadata recorded in the backup cycle is read from a storage node; Starting a database service program associated with the target database according to the program data, and loading the disk metadata into the memory of the management and control node; The database operation task corresponding to the database operation request is executed according to the program startup result and the metadata loading result.

2. The data processing method according to claim 1, before the step of receiving a database operation request submitted to a target database is performed, further comprising: When the database service corresponding to the target database is in a running state, determining the target disk corresponding to the target database; Read the disk metadata corresponding to the target disk, load the disk metadata into the management and control node memory, and back up to the storage node; Determine a database service program corresponding to the database service, and read global program data corresponding to the database service program; The local program data for starting the database service program is determined in the global program data as program data, and loaded into a local data file.

3. The data processing method according to claim 2, wherein the step of reading the disk metadata corresponding to the target disk and loading the disk metadata into the memory of the management and control node comprises: Reading the initial disk metadata corresponding to the target disk, determining the metadata identifier corresponding to the initial disk metadata, and determining the backup metadata identifier corresponding to the backup disk metadata in the memory of the control node; In the case where the backup metadata identifier is different from the metadata identifier, the disk metadata is determined in the initial disk metadata according to the backup metadata identifier and the metadata identifier, and the step of loading the disk metadata into the management and control node memory is performed.

4. The data processing method according to any one of claims 1 to 3, wherein when it is determined that the target database is abnormal according to the database operation request, reading the program data recorded in the backup cycle in the local data file comprises: detecting a program phase corresponding to the database service program in response to the database operation request; When the database service program is in the program startup phase, the target database is connected according to the set time interval and the set detection times. If the connection fails, it is determined that the target database is abnormal, and the following steps are executed: The step of reading the program data recorded in the backup cycle in the local data file; When the database service program is in the program running stage, the connection pool corresponding to the target database is determined, and the threads in the connection pool are detected. If the detection fails, it is determined that the target database is abnormal, and the step of reading the program data recorded in the backup cycle in the local data file is executed.

5. The data processing method according to any one of claims 1 to 4, before executing the database operation task step corresponding to the database operation request according to the program startup result and the metadata loading result, further comprising: Reading first disk information corresponding to the target database in the program data, and determining second disk information according to disk metadata loaded in the memory of the control node; comparing the first disk information and the second disk information; When it is determined according to the comparison result that the disk metadata loaded in the memory of the control node is incomplete, supplementary disk metadata is read in the storage node according to the first disk information and the second disk information; The supplementary disk metadata is loaded into the memory of the management and control node, and the step of executing the database operation task corresponding to the database operation request is performed according to the program startup result and the metadata loading result.

6. The data processing method according to any one of claims 1 to 5, wherein starting the database service program associated with the target database according to the program data and loading the disk metadata into the memory of the management and control node comprises: Loading the program data into the server memory, and starting the database service program associated with the target database according to the loading result; A control node associated with the database service program is determined, and the disk metadata is loaded into a control node memory of the control node.

7. The data processing method according to claim 6, wherein the step of loading the disk metadata into the control node memory of the control node comprises: Acquire the data conversion relationship between the storage node and the management and control node; Converting the disk metadata into target disk metadata according to the data conversion relationship; The target disk metadata is loaded into the management and control node memory of the management and control node.

8. The data processing method according to any one of claims 1 to 7, after the step of receiving a database operation request submitted to a target database is executed, further comprising: When it is determined that the target database is abnormal according to the database operation request and the database service program is normal, reading the disaster recovery disk metadata recorded in the backup cycle in the storage node; Loading the disaster recovery disk metadata into the management and control node memory; The database operation task corresponding to the database operation request is executed according to the database service program in the running state and the metadata loading result.

9. The data processing method according to any one of claims 1 to 8, after the step of executing the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result is executed, it also includes: Detecting whether the target database is in disaster recovery mode; If yes, switch the disaster recovery mode to the baseline mode, and run the target database service associated with the target database according to the mode switching result; If not, rebuild the target connection pool for the target database, and run the target database service associated with the target database according to the connection pool reconstruction result.

10. A data processing system, comprising a cloud storage service client, a storage node and a management and control node, the system comprising: The cloud storage service client is used to receive a database operation request submitted to a target database; When it is determined according to the database operation request that the target database is abnormal, reading the program data recorded in the backup period in the local data file, and sending a metadata acquisition request to the storage node; The storage node is used to send the disk metadata recorded in the backup period to the cloud storage service client in response to the metadata acquisition request; The cloud storage service end is also used to start the database service program associated with the target database according to the program data, and load the disk metadata into the management and control node memory of the management and control node; and execute the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

11. A data processing device, applied to a cloud storage service end, comprising: A receiving module is configured to receive a database operation request submitted to a target database; a reading module configured to read program data recorded in a backup cycle from a local data file and read disk metadata recorded in a storage node when determining that the target database is abnormal according to the database operation request; A loading module, configured to start a database service program associated with the target database according to the program data, and load the disk metadata into a management and control node memory; The execution module is configured to execute the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

12. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 9 are implemented.

13. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9.

14. A computer program, wherein: When the computer program is executed in a computer, the computer is caused to execute the steps of the method according to any one of claims 1 to 9.

Images