Data processing method, system and device

Abstract

The embodiment of the invention provides a data processing method, system and device, and the data processing method is applied to a cloud storage server and comprises the steps: receiving a database operation request submitted for a target database; when it is determined that the target database is abnormal according to the database operation request, program data recorded in a backup cycle is read in a local data file, and disk metadata recorded in the backup cycle is read in a storage node; starting a database service program associated with the target database according to the program data, and loading the disk metadata to a management and control node memory; and executing a database operation task corresponding to the database operation request according to a program starting result and a metadata loading result.

Description

Technical Field

[0001] The embodiments of this specification relate to the technical field of databases, and particularly to data processing methods, systems, and devices. Background Art

[0002] With the development of computer technology, cloud storage services are applied in more and more scenarios, and the providers of cloud storage services are gradually increasing in scale in order to provide stable services for users. Along with the gradual increase in scale, it is inevitably faced with an increase in the number of storage machines. For a large number of storage machines, the providers of cloud storage services not only need to monitor frequently occurring events, but also need to monitor occasional events with a relatively low probability but still existing, such as database instance failures, etc., which can easily lead to problems in the control core node, and then affect the IO path. In the prior art, for such occasional events with a relatively low probability but still existing, most use the method of manual monitoring to implement event management. That is to say, after such an event occurs, the fault is solved by manual intervention. However, the method of manual intervention is belated, resulting in serious impact on the normal operation of the service after the event occurs; and the event occurrence time is uncertain, making it difficult to quickly and timely eliminate the fault. Therefore, an effective solution is urgently needed to solve the above problems. Summary of the Invention

[0003] In view of this, the embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing system, a data processing device, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects existing in the prior art.

[0004] According to the first aspect of the embodiments of this specification, a data processing method is provided, which is applied to a cloud storage server and includes:

[0005] Receiving a database operation request submitted for a target database;

[0006] When it is determined that the target database is abnormal according to the database operation request, reading the program data recorded in the backup period in the local data file, and reading the disk metadata recorded in the backup period in the storage node;

[0007] Starting the database service program associated with the target database according to the program data, and loading the disk metadata into the memory of the control node;

[0008] Executing the database operation task corresponding to the database operation request according to the program start result and the metadata loading result.

[0009] According to the second aspect of the embodiments of this specification, a data processing system is provided, including a cloud storage server, a storage node, and a management and control node. The system includes:

[0010] The cloud storage server is configured to receive a database operation request submitted for a target database; when it is determined that the target database is abnormal according to the database operation request, read the program data recorded in the backup period from a local data file, and send a metadata acquisition request to the storage node;

[0011] The storage node is configured to, in response to the metadata acquisition request, send the disk metadata recorded in the backup period to the cloud storage server;

[0012] The cloud storage server is further configured to start the database service program associated with the target database according to the program data, and load the disk metadata into the memory of the management and control node of the management and control node; execute the database operation task corresponding to the database operation request according to the program start result and the metadata loading result.

[0013] According to the third aspect of the embodiments of this specification, a data processing device is provided, which is applied to a cloud storage server and includes:

[0014] A receiving module, configured to receive a database operation request submitted for a target database;

[0015] A reading module, configured to, when it is determined that the target database is abnormal according to the database operation request, read the program data recorded in the backup period from a local data file, and read the disk metadata recorded in the backup period from the storage node;

[0016] A loading module, configured to start the database service program associated with the target database according to the program data, and load the disk metadata into the memory of the management and control node;

[0017] An execution module, configured to execute the database operation task corresponding to the database operation request according to the program start result and the metadata loading result.

[0018] According to the fourth aspect of the embodiments of this specification, a computing device is provided, including:

[0019] A memory and a processor;

[0020] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above data processing method are implemented.

[0021] According to a fifth aspect of the embodiments of the present specification, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above data processing method.

[0022] According to a sixth aspect of the embodiments of the present specification, there is provided a computer program which, when executed on a computer, causes the computer to execute the steps of the above data processing method.

[0023] The data processing method provided in this embodiment is applied to a cloud storage server. In order to protect against database exception events that are less likely but still exist and avoid the inability to execute database operation requests submitted by computing nodes, program data and disk metadata can be backed up during the normal operation phase of the service. When a database operation request for a target database is received and the target database is abnormal at this time, the program data recorded in the backup cycle can be directly read from the local data file, and the disk metadata recorded in the backup cycle can be read from the storage node, so as to obtain the basic data that can support the normal execution of the database operation request. Then, the database service program associated with the target database can be started according to the program data, and at the same time, the disk metadata can be loaded into the memory of the management node, so that the database service program can run normally and can read the disk metadata associated with the database operation request from the memory of the management node. Subsequently, the database operation task corresponding to the database operation request can be executed according to the program start result and the metadata loading result. At any time, the execution of the database operation task will not be affected by the database exception, and the impact of unavailable metadata on the IO link during database exceptions can be solved, so as to achieve the purpose of automatic recovery without intervention and effectively reduce the impact on database operations. Description of the Drawings

[0024] Figure 1 is a schematic diagram of a data processing method provided by an embodiment of the present specification;

[0025] Figure 2 is a flowchart of a data processing method provided by an embodiment of the present specification;

[0026] Figure 3 is a schematic structural diagram of a cloud storage service in a data processing method provided by an embodiment of the present specification;

[0027] Figure 4 is a schematic structural diagram of a cloud storage service in another data processing method provided by an embodiment of the present specification;

[0028] Figure 5 is a flowchart of the processing process of a data processing method provided by an embodiment of the present specification;

[0029] Figure 6 It is a schematic structural diagram of a data processing system provided by an embodiment of this specification;

[0030] Figure 7 It is a schematic structural diagram of a data processing device provided by an embodiment of this specification;

[0031] Figure 8 It is a structural block diagram of a computing device provided by an embodiment of this specification. Detailed implementation manners

[0032] In the following description, many specific details are set forth in order to provide a thorough understanding of this specification. However, this specification can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the connotation of this specification. Therefore, this specification is not limited by the specific implementations disclosed below.

[0033] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more of the associated listed items.

[0034] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein can be interpreted as "when" or "while" or "in response to determining".

[0035] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for the user to choose to authorize or refuse.

[0036] First, the noun terms involved in one or more embodiments of this specification are explained.

[0037] VM (Virtual Machine): Virtual machine, multiple instances that can run an OS virtualized on a single server.

[0038] TDC: Front-end of block storage. The front-end of block storage, TDC, runs on compute nodes. As the entry point for the control flow and data flow of block storage, TDC provides complete block storage services.

[0039] In this specification, a data processing method is provided. This specification also relates to a data processing system, a data processing device, a computing device, and a computer-readable storage medium, which will be described in detail one by one in the following embodiments.

[0040] In practical applications, failures of database instances, such as the inability to connect the primary and standby databases over the network, or failures of the physical devices of the primary and standby databases, will cause problems in the control core node, which in turn affects the IO path. Specifically, in the cloud storage control environment of the prior art, the persistence and backup of control metadata are mainly implemented in a highly available database instance. For example, in the disk opening process, the front-end of block storage, TDC, needs to first obtain the storage cluster information where the disk is located and the metadata of the disk from the control node RiverMaster, and then complete the subsequent disk opening process. However, when the database is unavailable, the disk metadata in the control node becomes unavailable, resulting in the problem of failed disk opening. Therefore, an effective solution is urgently needed to solve the above problems.

[0041] See Figure 1As shown in the schematic diagram, the data processing method provided in this embodiment is applied to a cloud storage server. In order to protect against database exception events with a relatively low probability but still existing and avoid the inability to execute database operation requests submitted by computing nodes, during the normal operation stage of the service, program data and disk metadata can be backed up. When a database operation request for a target database is received and the target database is abnormal at this time, the program data recorded in the backup period can be directly read from the local static data file, and the disk metadata recorded in the backup period can be read from the storage node. After obtaining the basic data that can support the normal execution of the database operation request, the database service program associated with the target database can be started according to the program data, and at the same time, the disk metadata can be loaded into the memory of the management and control node, so that the database service program can run normally and can read the disk metadata associated with the database operation request from the memory of the management and control node. Subsequently, the database operation task corresponding to the database operation request can be executed according to the program start result and the metadata loading result. At any time, the execution of the database operation task will not be affected by the abnormality of the database, and the impact of unavailable metadata on the IO link during database abnormality can be solved, so as to achieve the purpose of automatic recovery without intervention and effectively reduce the impact on database operations.

[0042] See Figure 2 , Figure 2 Fig. shows a flowchart of a data processing method according to an embodiment of the present specification. This method is applied to a cloud storage server and specifically includes the following steps:

[0043] Step S202: Receive a database operation request submitted for a target database.

[0044] The data processing method provided in this embodiment is applied to a cloud storage server, where the cloud storage server specifically refers to the server corresponding to the service provider that provides cloud storage services. Correspondingly, the template database specifically refers to a relational database that needs to operate according to the database operation requests submitted by computing nodes in the current scenario. And the target database is managed by the management and control node of the cloud storage server and is used to perform database operation processing that meets the operation requirements of computing nodes for the target database, such as opening a disk, releasing a disk, changing the data storage location, etc. Correspondingly, the database operation request specifically refers to an operation request submitted by a computing node for the target database during the operation state of the cloud storage service, such as an opening disk operation request, a releasing disk operation request, a changing data storage location operation request, etc. This embodiment does not make any limitations here.

[0045] Based on this, when a database operation request submitted for the target database is received, it indicates that the computing node needs to operate on the target database at this time. In order to protect against database exception events that are less likely but still exist and avoid the inability to execute the database operation request submitted by the computing node, the program data and disk metadata can be backed up during the normal operation stage of the service; in this embodiment, the specific implementation method is as follows:

[0046] When the database service corresponding to the target database is in a running state, determine the target disk corresponding to the target database; read the disk metadata corresponding to the target disk, load the disk metadata into the memory of the management and control node, and back it up to the storage node; determine the database service program corresponding to the database service, and read the global program data corresponding to the database service program; in the global program data, determine the local program data that starts the database service program as the program data, and load it into the local data file.

[0047] Specifically, the database service specifically refers to a stable, reliable, and elastically scalable online database service provided by the cloud storage server. Through this service, the user of the cloud storage service can perform operations on the target database for any requirements. Correspondingly, the running state specifically refers to the state where the database service provides services normally. Correspondingly, the target disk specifically refers to the physical storage device corresponding to the target database, including but not limited to solid-state drives, mechanical hard drives, etc.; correspondingly, the disk metadata specifically refers to the system data that describes the characteristics of the disk and is used to record information such as the location of the disk, creation time, disk space, and the cluster to which the disk belongs. Since the disk is the physical storage space of the database, the database operation request for the target database will be attached to the disk to complete, and the execution of this request requires disk metadata.

[0048] Correspondingly, the control node memory specifically refers to the operating memory corresponding to the control node. Correspondingly, the storage node specifically refers to the node used for persistent data storage, which is used to back up disk metadata to prevent the loss of disk metadata. Correspondingly, the database service program specifically refers to the program on which the database service runs. This program is the carrier presented by the database service at the front end, that is, all functions that the database service can provide can be implemented through operations in the database service program. Correspondingly, the global program data specifically refers to all the program data required when the database service program runs, including but not limited to the data required when the program starts, the data required for each function implementation in the program, and the data required to support the normal operation of the function, etc. Correspondingly, the local program data specifically refers to the data necessary when the database service program starts. This data can support the database service program to start and run some basic functions, but the data required for some relatively complex functions involved in this program does not belong to the local program data. That is to say, the global program data can support the database service program to provide a complete and efficient database service, while the local program data supports the database service program to provide a database service with some basic functions. Correspondingly, the local data file specifically refers to the static data file used for persistent storage of program data. This file is deployed on the server side and is not affected by database failures.

[0049] Based on this, in order to support the recovery of the database at any time in case of database exceptions and support database operations on the target database, the program data and disk metadata required for the operation of the database service can be backed up regularly. That is to say, when the database service corresponding to the target database is in the running state, the target disk corresponding to the target database can be determined first; then the disk metadata corresponding to the target disk is read, and the disk metadata is backed up by loading it into the control node memory and backing it up to the storage node. At the same time, for the database service program, the database service program corresponding to the database service can be determined, and the global program data corresponding to the database service program can be read. On this basis, in order to avoid backing up redundant data while ensuring that the database service program provides basic functions, the local program data for starting the database service program can be determined from the global program data as the program data and loaded into the local data file to complete the backup of the program data, so as to support the normal operation of the database service according to the backed-up program data and disk metadata in case of database failures, thereby realizing the normal execution of database operation tasks.

[0050] In practical applications, when backing up program data and disk metadata, in order to reduce the resources consumed by the backup operation, the backup can be completed according to the set backup period. For example, the program data can be stored in a local data file every hour. Or the backup can be completed according to the degree of data update. For example, when the disk metadata changes, a backup of the disk metadata is triggered, so as to ensure that the backup of the program data and disk metadata can support database anomalies. In addition, during the backup stage of the disk metadata, in order to support the external data query service while avoiding the operation of querying the database, adding it to the memory of the control node can improve the response time of the service. At the same time, it can ensure that the service of querying metadata can still be provided in the state where the database service is unavailable, thus supporting the normal operation of the service. During specific implementation, since the necessary data required for the startup of the database service program is modified less frequently, and basically no changes will be made in the online production environment after the environment deployment is completed, the backup period time can be set as needed to ensure the integrity and correctness of the data.

[0051] In summary, to support database operations on the compute nodes even in the case of database anomalies, the program data and disk metadata can be backed up in advance according to the backup period while the database service is running, so as to support the stable database service by calling the backup data in case of database anomalies.

[0052] On this basis, considering that the program data and disk metadata are backed up while the service is running, and the backup result may be inconsistent with the changed disk metadata. For example, a new disk opening operation occurs at the backup time node, which will result in an incomplete backup. Therefore, in order to ensure that the disk metadata in the control node memory is the latest and complete disk metadata, the identity comparison can be performed after the backup. In this embodiment, the specific implementation method is as follows:

[0053] Read the initial disk metadata corresponding to the target disk, determine the metadata identifier corresponding to the initial disk metadata, and determine the backup metadata identifier corresponding to the backup disk metadata in the control node memory; in the case where the backup metadata identifier is different from the metadata identifier, determine the disk metadata from the initial disk metadata according to the backup metadata identifier and the metadata identifier, and execute the step of loading the disk metadata into the control node memory.

[0054] Specifically, the initial disk metadata specifically refers to the disk metadata read at the current moment. Correspondingly, the metadata identifier specifically refers to the unique identifier corresponding to the initial disk metadata. In practical applications, the disk metadata consists of multiple pieces of metadata, and each piece of metadata corresponds to a metadata identifier, and the metadata identifiers are set in an increasing manner. Therefore, the metadata identifier of the initial disk metadata is the identifier corresponding to the last piece of metadata in the initial disk metadata. Correspondingly, the backup metadata identifier is the identifier corresponding to the last piece of metadata in the disk metadata stored in the memory of the control node.

[0055] Based on this, when performing the backup operation of the disk metadata, considering that the disk metadata backed up to the memory of the control node may not be the latest disk metadata. For example, if there is a change in the disk metadata at the backup time node, the backed-up disk metadata will not be the latest version of the disk metadata at this time. Therefore, the initial disk metadata corresponding to the target disk can be read first, and the metadata identifier corresponding to the initial disk metadata can be determined, as well as the backup metadata identifier corresponding to the backup disk metadata in the memory of the control node. By comparing the metadata identifier and the backup metadata identifier, it can be determined whether the two are completely synchronized. If the backup metadata identifier is different from the metadata identifier, it means that the ID value corresponding to the metadata identifier is greater than the ID value corresponding to the backup metadata identifier, which further indicates that not all disk metadata has been backed up at this time. Therefore, the disk metadata can be determined in the initial disk metadata according to the backup metadata identifier and the metadata identifier, that is, the disk metadata that needs to be backed up can be determined, and then step S204 can be executed to ensure that the data operation request is normally executed at the database abnormal node.

[0056] In practical applications, during the startup phase of the database service program, a thread for periodically loading disk metadata into the memory can be added. And each time the disk metadata is loaded into the memory of the control node, it can be checked whether all the disk metadata has been updated to ensure that the disk metadata in the memory of the control node can always maintain the latest version of the disk metadata to cope with the occurrence of database anomalies. In addition, during the running phase of the database service program, a thread for periodically obtaining specified data from the database and backing it up locally can be adopted. For example, every hour, the program data is backed up to the local static data file, so that the program can run normally during the database abnormal phase.

[0057] This embodiment illustrates the data processing method by taking the opening process as an example. Refer to Figure 3As shown in the schematic diagram in (a), for the processing of the opening process, the block storage front-end TDC of the computing node needs to obtain the storage cluster information where the disk is located and the disk metadata at the management and control node. In order to support the normal progress of the opening process at any time, the necessary data required for the operation of the database service program and the disk metadata can be backed up at the database service running node. Specifically, the escape preparation module can, during the startup phase of the database service program, load the full amount of disk metadata corresponding to the database into the thread in the cache, and check whether all the disk metadata has been backed up after each backup. If not, continue to select the remaining disk metadata for backup until the IDs of the disk metadata in the database and the disk metadata in the memory are the same, then the backup of the disk metadata is completed. At the same time, the necessary startup data of the database service program can be obtained every hour, such as the basic information Azone_id and Azone_name of the Azone table, Cluster_id and Azone_id in the Cluster_info table, the number of disks device_count in the cluster, the disk capacity total_size of the cluster, the sold disk capacity sale_size of the cluster, etc., and this part of the data is stored in the local static data file, so as to support the opening process based on the backed-up disk metadata and the necessary startup data in case of database exception.

[0058] In summary, by comparing the metadata identifiers after each backup process of the disk metadata is completed, the detection of the backup result can be achieved, and the backup synchronization of the disk metadata can be performed according to the detection result, so as to support that in case of database exception at any stage, the subsequent normal database operations will not be affected.

[0059] Step S204, when it is determined that the target database is abnormal according to the database operation request, read the program data recorded in the backup period in the local data file and read the disk metadata recorded in the backup period in the storage node.

[0060] Specifically, based on the above receipt of the database operation request, if an operation is performed on the target database according to the database operation request and the target database is determined to be abnormal, it will cause the database operation to be unable to continue at the current moment. For example, when performing an opening operation, a network connection interruption occurs in the database, or a physical device failure occurs in the primary and standby databases, resulting in the inability to continue the database operation through the existing primary and standby database modes. Therefore, in order to avoid affecting the operation of the database service, the program data required for starting the database service program can be read from the local data file that has backed up the program data, and the disk metadata recorded during the backup cycle can be read from the storage node, so as to realize that the program data and disk metadata recorded during the backup cycle can continue to support the operation of the database service, thereby supporting the normal execution of the database operation request.

[0061] Among them, the abnormality of the target database specifically means that the target database cannot work based on the database service at the current time node. For example, a network connection interruption occurs in the target database, or a physical device failure occurs in the primary and standby databases, which further causes the database operation to be unable to complete the corresponding operation based on the primary data and the standby database. Correspondingly, the backup cycle specifically refers to the cycle for backing up the program data and disk metadata. For example, if the program data is backed up every hour, the backup cycle is one hour; if the disk metadata is backed up each time the disk metadata changes, the backup cycle is the change cycle of the disk metadata each time. Correspondingly, the storage node specifically refers to the underlying data storage node for persisting the disk metadata. It should be noted that the relevant descriptions of the program data and disk metadata can be found in the description of the above embodiments, and this embodiment will not elaborate too much here.

[0062] Furthermore, considering that when providing services to the computing node, the database service program may be in the startup phase or the running phase, and the target database may fail in different phases. In order to accurately detect the faults of the target database in any state, different methods can be adopted; in this embodiment, the specific implementation method is as follows:

[0063] Respond to the database operation request to detect the program phase corresponding to the database service program; when the database service program is in the program startup phase, connect to the target database at a set time interval and a set number of detection times. If the connection fails, determine that the target database is abnormal and execute the step of reading the program data recorded in the backup cycle from the local data file; when the database service program is in the program running phase, determine the connection pool corresponding to the target database and detect the threads in the connection pool. If the detection fails, determine that the target database is abnormal and execute the step of reading the program data recorded in the backup cycle from the local data file.

[0064] Specifically, the program startup phase specifically refers to the phase in which the database service program should be in the startup state. Accordingly, the program running phase specifically refers to the phase in which the database service program should be in the running state. Accordingly, setting the time interval specifically refers to setting the time interval according to actual needs, such as 0.1-0.5s, etc. Accordingly, setting the number of detections specifically refers to the number of continuous connectivity detections set according to actual needs, such as 5 times, or 10 times, etc.; accordingly, the connection pool specifically refers to the thread pool composed of threads associated with the target database. Accordingly, detecting the thread is to detect whether the thread can connect to the database normally.

[0065] Based on this, after receiving a database operation request submitted to the database, in order to ensure that the target database can support the database operation, the target database can be detected whether there is a fault. At this time, the program stage corresponding to the database service program can be detected in response to the database operation request; when the database service program is in the program startup stage, it means that the database service program has not yet worked, but has been started. At this time, the connection pool of the database has not been fully started and cannot provide services. Therefore, the target database can be connected according to the set time interval and the set number of detections. If the connection fails, it means that the target database cannot be connected and cannot support database operations. Therefore, it is determined that the target database is abnormal, and the steps of reading the program data recorded in the backup cycle in the local data file can be executed.

[0066] When the database service program is in the program running stage, it means that the database service program should be in working state at this time. Then, the connection pool corresponding to the target database can be determined, and the threads in the connection pool can be tested. If the test fails, it means that the connection pool cannot connect to the database, and then the target database is determined to be abnormal. Then, the step of reading the program data recorded in the backup cycle in the local data file can be executed, so that the database operation request can be continued based on the backed-up program data and disk metadata.

[0067] That is to say, during the normal operation of the target database, it is possible to detect whether the database is available. If the database network is not accessible, the database master and backup nodes are down, or there is a problem with the database storage medium, it will indicate that the database is abnormal, and the backup data needs to support the normal operation of the database. In actual applications, during the startup phase of the database service program, a periodic thread can be added to regularly detect whether the database is available. The detection method is as follows: During the startup phase of the database service program, since the database connection pool has not yet been started, you can directly connect to the database for detection. If the database cannot be connected or the database ping cannot be reached, you can set a time interval to continuously detect n times. If you cannot connect at all, it means that the database is abnormal, and the escape process is triggered, that is, it is necessary to complete the relevant operations of the database based on the backup data. During the operation phase of the database service program, you can only detect whether the database connection pool is healthy, that is, detect whether the threads in the connection pool are normally connected to the database. If it is unhealthy for m consecutive times, it means that the database is abnormal, and the escape process is triggered, that is, it is necessary to complete the relevant operations of the database based on the backup data.

[0068] Using the above example, before the opening process, in order to ensure that the opening can proceed normally, the database can be tested; see Figure 3 In the schematic diagram shown in (b), when the database service program is in the program startup stage, the escape monitoring module can directly connect to the database. If the database cannot be connected or the database ping cannot be reached at this time, it can be detected 10 times continuously at intervals of 0.1-0.5s. If it is not connected to the database, it means that the database is abnormal at this time, and the escape processing module is triggered to work, that is, it can continue to support the normal opening processing operation based on the backup disk metadata and necessary startup data in the future. When the database service program is in the program running stage, it can be detected whether the thread in the database connection pool is connected to the database. If the database cannot be connected for 10 consecutive times, it means that the database is abnormal at this time, and the escape processing module is triggered to work, that is, it can continue to support the normal opening processing operation based on the backup disk metadata and necessary startup data in the future. In addition, if it is detected that the database has been restored, the automatic recovery module can trigger the automatic recovery process to restore to the normal database connection state.

[0069] Furthermore, if the database is determined to be abnormal and the process is restarted, it means that the database service program needs to be restarted and disk metadata needs to be provided. Therefore, the necessary startup data for starting the database service program can be read from the local static data file, and the disk metadata can also be read from the underlying storage node to support the block storage front-end TDC of the computing node to obtain the disk metadata and then proceed with the subsequent disk opening process.

[0070] In summary, by performing different detection methods on the target database at different program stages, the accuracy of detecting database anomalies can be ensured, so that the subsequent detection results can trigger the escape mechanism, thereby supporting the database to complete database operations in abnormal situations and ensuring the stability of database services.

[0071] Step S206: starting a database service program associated with the target database according to the program data, and loading the disk metadata into the memory of the management and control node.

[0072] Specifically, in the above-mentioned determination that the database is abnormal, after obtaining the program data and disk metadata recorded in the backup period, further, in order to support the normal operation of the database service and support the database operation results to meet the requirements, the database service program associated with the target database can be started according to the program data, and the disk metadata can be loaded into the management node memory at the same time, so that when performing database operations, the corresponding database operations can be completed according to the started database service program and the disk metadata loaded in the management node memory. Among them, the relevant descriptions of the database service program and the management node memory can be referred to the description in the above embodiment, and this embodiment will not be repeated here.

[0073] Furthermore, after obtaining the program data and disk metadata, they can be loaded into the memory to complete the startup of the program and the execution of the database operation request; in this embodiment, the specific implementation method is as follows:

[0074] The program data is loaded into the server memory, and the database service program associated with the target database is started according to the loading result; the control node associated with the database service program is determined, and the disk metadata is loaded into the control node memory of the control node.

[0075] Specifically, the server memory specifically refers to the memory that loads data when running the database service program. Based on this, during the program startup phase, the program data can be first loaded into the server memory, and the database service program associated with the target database can be started according to the program data loaded into the server memory, so that the database service program can provide basic functions; at the same time, the control node associated with the database service program can be determined, and the disk metadata can be loaded into the control node memory of the control node, so as to provide the database service program with the metadata basis for performing database operations, so that the corresponding database operations can be completed in combination with the started database service program and the disk metadata.

[0076] Furthermore, considering that the storage node and the control node may store data in different forms, when loading the disk metadata in the storage node into the control node memory, the disk metadata needs to be converted; in this embodiment, the specific implementation method is as follows:

[0077] Acquire the data conversion relationship between the storage node and the management and control node; convert the disk metadata into target disk metadata according to the data conversion relationship; and load the target disk metadata into the management and control node memory of the management and control node.

[0078] Specifically, the data conversion relationship specifically refers to a conversion relationship that can convert the data format stored in the storage node into the data format stored in the corresponding control node. Correspondingly, the target disk metadata is the disk metadata obtained after converting the disk metadata according to the data conversion relationship, and is the same as the data storage format of the control node.

[0079] Based on this, after reading the disk metadata recorded in the backup cycle from the storage node, further, considering that the storage node and the control node may have different data storage formats, the data conversion relationship between the storage node and the control node can be obtained; then the disk metadata is converted into the target disk metadata according to the data conversion relationship; after the disk metadata is converted into the storage format of the corresponding control node, the target disk metadata is loaded into the control node memory of the control node, and the corresponding database operation can continue.

[0080] In summary, after reading the disk metadata from the storage node, in order to ensure that it can be used normally after being loaded into the management and control node memory, the disk metadata can be converted into the target disk metadata of the corresponding management and control node data storage format according to the data conversion relationship, thereby ensuring that the database operation tasks can be performed subsequently.

[0081] In addition, after the database service program is started and the disk metadata is loaded, in order to ensure that the disk metadata loaded into the memory of the control node is complete disk metadata, thereby ensuring that the database operation request can be executed normally, an integrity check can also be performed; in this embodiment, the specific implementation method is as follows:

[0082] The first disk information corresponding to the target database is read in the program data, and the second disk information is determined according to the disk metadata loaded in the memory of the control node; the first disk information and the second disk information are compared; when it is determined according to the comparison result that the disk metadata loaded in the memory of the control node is incomplete, supplementary disk metadata is read in the storage node according to the first disk information and the second disk information; the supplementary disk metadata is loaded into the memory of the control node, and the steps of executing the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result are performed.

[0083] Specifically, the first disk information specifically refers to the disk number information of the disk metadata corresponding to the target database; correspondingly, the second disk information specifically refers to the disk number information of the disk metadata loaded in the control node; if the first disk information and the second disk information are inconsistent, it means that the disk metadata in the memory of the control node is incomplete. Correspondingly, the supplementary disk metadata specifically refers to the disk metadata read from the storage node to supplement the disk metadata already loaded in the control node, ensuring that after the control node loads the supplementary disk metadata, the disk metadata it loads is consistent with the disk metadata corresponding to the target database, thereby ensuring that the database operation task can be performed normally.

[0084] Based on this, after entering the operation stage, in order to ensure the normal operation of the service and complete the database operation task corresponding to the database operation request, the integrity of the disk metadata can also be checked; that is, the first disk information corresponding to the target database can be read in the program data, and the second disk information can be determined according to the disk metadata loaded in the memory of the control node; at this time, the first disk information and the second disk information will be compared; if it is determined according to the comparison result that the first disk information and the second disk information are inconsistent, it means that the number of disks of the target database is greater than the disk data corresponding to the disk metadata in the memory of the control node, and then it means that the disk metadata loaded in the memory of the control node is incomplete. Therefore, in order to ensure the integrity of the disk metadata in the memory of the control node, supplementary disk metadata can be read in the storage node according to the first disk information and the second disk information; then the supplementary disk metadata is loaded into the memory of the control node, and step S208 is executed.

[0085] In specific implementation, when it is determined that the target database is abnormal, in order to ensure that the database service can continue to provide services, the disk metadata can be restored from the underlying storage node to the memory during the operation phase, thereby ensuring that the existing IO service is not interrupted. That is to say, in the startup phase, if it is detected that the database is unavailable and the escape process is triggered, the program data can be loaded into the memory from the local static data file, and then the database service program can be started according to the program data loaded into the memory, and the database service program can be made to work normally at this time. And after entering the operation phase, it can be determined whether the disk metadata in the memory of the control node is complete, such as by comparing the number of disks in the cluster table in the program data with the number of disks in the memory. If it is less than the cluster disk data, the disk metadata is incomplete. At this time, all the disk metadata can be obtained by polling from the underlying storage node, and according to the data conversion relationship between the storage node and the control node, the disk metadata can be restored to the disk metadata required by the control node, thereby supporting the normal operation of the database service.

[0086] Using the above example, see Figure 4As shown in the schematic diagram, the necessary startup data required to start the database service program can be read from the local static data file first, and the necessary startup data can be loaded into the memory, and the database service program can be started according to the necessary startup program in the memory. During the program running stage, the disk metadata in the memory is checked to see if it is complete. If it is incomplete, all disk metadata can be obtained by polling the underlying storage node, and then all disk metadata read from the storage node can be restored to the disk metadata of the corresponding control node according to the preset service logic. For example, the Key in the RiverMaster device table of the control node is device_key, which is a two-segment cluster_id+device_id, while the Key of the device table of the underlying storage is a one-segment volume_id. At this time, after reading the disk metadata stored in the underlying storage node, it can be converted into the disk metadata of the corresponding storage node, and then written into the memory of the control node, so that the block storage front-end TDC of the computing node can obtain the disk metadata and then carry out the subsequent disk opening process.

[0087] In summary, before performing database operations, the restored disk metadata will be checked for integrity, thereby ensuring that database operations are performed when all disk metadata is restored, to avoid the operation results not meeting requirements and affecting the operation of front-end services.

[0088] Step S208: executing the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0089] Specifically, after the database service program is started according to the program data, it means that the database service program can run normally at this time, and after the disk metadata is loaded into the memory of the control node, it further means that the database service corresponding to the target database can provide the corresponding database operation service. Therefore, according to the program startup result and the metadata loading result, the database operation task corresponding to the database operation request can be executed to complete the database operation processing for the target database. Among them, the database operation task specifically refers to the task of performing any type of operation on the target database, including but not limited to the task of opening the disk, releasing the disk, changing the data storage location, etc.

[0090] Continuing with the above example, when the database operation program can run normally and the disk metadata in the control node's memory is complete, the disk metadata in the control node's memory can be sent to the block storage front-end TDC at this time, enabling the block storage front-end TDC to, based on the database service program and the obtained disk metadata, create a storage space that meets the usage requirements on disk 1 in the target storage cluster where the target disk is located. This storage space corresponds to the physical storage device - disk 1, and the size of the storage space is 1T to facilitate downstream use.

[0091] Furthermore, after completing the database operation task, in order to ensure that the cloud storage server can provide a complete database service, a mode switch can be performed; in this embodiment, the specific implementation method is as follows:

[0092] Detect whether the target database is in the disaster recovery mode; if so, switch the disaster recovery mode to the baseline mode and run the target database service associated with the target database according to the mode switch result; if not, reconstruct the target connection pool for the target database and run the target database service associated with the target database according to the connection pool reconstruction result.

[0093] Specifically, the disaster recovery mode specifically refers to a model in which the target database still supports database operations in case of an abnormality. Correspondingly, the baseline mode specifically refers to the mode in which the target database is in a normal state; it should be noted that the disaster recovery mode can provide partial functions, while the baseline mode can provide complete functions. Correspondingly, the description of the target database service can refer to the description of the above database service, and this embodiment does not make any limitations here.

[0094] Based on this, when the database returns to normal, in order to be able to provide a complete database service, it is possible to detect whether the target database is in the disaster recovery mode; if so, it means that the target database at the current stage still cannot provide a complete database service. Therefore, the disaster recovery mode can be switched to the baseline mode, and the target database service associated with the target database can be run according to the mode switch result to provide a complete database service. If not, it means that the target database at the current stage still cannot provide the database service, then at this time, the target connection pool can be reconstructed for the target database, and the target database service associated with the target database can be run according to the connection pool reconstruction result to provide a complete database service.

[0095] That is to say, if the target database is currently in the disaster recovery mode, that is, the control node is restarted after the database abnormality, then at this time, it can be switched from the disaster recovery mode to the normal mode to provide a stable and complete database service. If it is not in the disaster recovery mode, the connection pool of the target database can be re-established, and thus a stable and complete database service can be provided according to the connection pool establishment result.

[0096] In summary, by switching the schema of the target database, the target database can be switched to the benchmark mode after returning to normal, thereby ensuring that the database service can provide a complete service.

[0097] In addition, considering that a database failure may occur in the database service program during the running state, in this case, it is not necessary to restart the database service program. It only needs to load the disk metadata into the memory to support the database service program to continue executing the database operation tasks. In this embodiment, the specific implementation method is as follows:

[0098] When it is determined that the target database is abnormal according to the database operation request and the database service program is normal, read the disaster recovery disk metadata recorded in the backup period in the storage node; load the disaster recovery disk metadata into the memory of the management and control node; execute the database operation task corresponding to the database operation request according to the running database service program and the metadata loading result. Among them, the description of the disaster recovery disk metadata can refer to the description of the disk metadata in the above embodiment, and this embodiment will not elaborate too much here.

[0099] Based on this, when it is determined that the target database is abnormal according to the database operation request and the database service program is normal, it means that the target database is abnormal at this time but the database service program has not been restarted. Furthermore, it means that the database service program does not need to be restarted. At the same time, in order to support the execution of the database operation request, only the disk metadata can be restored. That is to say, the disaster recovery disk metadata recorded in the backup period can be read in the storage node; then the disaster recovery disk metadata is loaded into the memory of the management and control node; at this time, the database operation task corresponding to the database operation request can be directly executed according to the running database service program and the metadata loading result.

[0100] In summary, the data processing method provided in this embodiment can achieve the degradation of the cloud storage control link and the backup of metadata, and at the same time can solve the impact on the IO link when the metadata is unavailable, ensuring that the IO link can still operate normally when the original metadata is unavailable.

[0101] The data processing method provided in this embodiment is applied to a cloud storage server. To protect against database exception events that are less likely but still exist and avoid the inability to execute database operation requests submitted by computing nodes, program data and disk metadata can be backed up during the normal operation of the service. When a database operation request for a target database is received and the target database is abnormal at this time, the program data recorded in the backup period can be directly read from the local data file, and the disk metadata recorded in the backup period can be read from the storage node. After obtaining the basic data that can support the normal execution of the database operation request, the database service program associated with the target database can be started according to the program data, and at the same time, the disk metadata is loaded into the memory of the management and control node, so that the database service program can run normally and can read the disk metadata associated with the database operation request from the memory of the management and control node. Subsequently, the database operation task corresponding to the database operation request can be executed according to the program start result and the metadata loading result. It is ensured that at any time, the execution of the database operation task will not be affected by the database exception, and the impact of unavailable metadata on the IO link during database exception is solved, thereby achieving the purpose of automatic recovery without intervention and effectively reducing the impact on database operations.

[0102] The following combines the attached Figure 5 , taking the application of the data processing method provided in this specification in the disk opening scenario under cloud storage as an example, further illustrates the data processing method. Among them, Figure 5 shows the processing procedure flowchart of a data processing method provided in an embodiment of this specification, which specifically includes the following steps.

[0103] Step S502, when the database service corresponding to the target database is in a running state, determine the target disk corresponding to the target database.

[0104] Step S504, read the initial disk metadata corresponding to the target disk, determine the metadata identifier corresponding to the initial disk metadata, and determine the backup metadata identifier corresponding to the backup disk metadata in the memory of the management and control node.

[0105] Step S506, when the backup metadata identifier is different from the metadata identifier, determine the disk metadata from the initial disk metadata according to the backup metadata identifier and the metadata identifier, and load the disk metadata into the memory of the management and control node and back up it to the storage node.

[0106] Step S508, determine the database service program corresponding to the database service, and read the global program data corresponding to the database service program.

[0107] Step S510, determine the local program data for starting the database service program in the global program data as the program data, and load it into the local data file.

[0108] Step S512, receive a database operation request submitted for the target database.

[0109] Step S514, when it is determined that the target database is abnormal according to the database operation request, read the program data recorded in the backup period in the local data file, and read the disk metadata recorded in the backup period in the storage node.

[0110] Specifically, in response to the database operation request, detect the program phase corresponding to the database service program; when the database service program is in the program startup phase, connect to the target database at a set time interval and a set number of detection times. If the connection fails, determine that the target database is abnormal, and execute the step of reading the program data recorded in the backup period in the local data file.

[0111] When the database service program is in the program running phase, determine the connection pool corresponding to the target database, and detect the threads in the connection pool. If the detection fails, determine that the target database is abnormal, and execute the step of reading the program data recorded in the backup period in the local data file.

[0112] Furthermore, when it is determined that the database is abnormal and the process is restarted, it means that the database service program needs to be restarted at this time, and the disk metadata is provided at the same time. Therefore, the necessary startup data for starting the database service program can be read from the local static data file, and the disk metadata can also be read from the underlying storage node to support the block storage front-end TDC of the computing node to obtain the disk metadata and then perform the subsequent disk opening process.

[0113] Step S516, load the program data into the server memory, and start the database service program associated with the target database according to the loading result.

[0114] Step S518, determine the control node associated with the database service program, and obtain the data conversion relationship between the storage node and the control node.

[0115] Step S520, convert the disk metadata into the target disk metadata according to the data conversion relationship, and load the target disk metadata into the control node memory of the control node.

[0116] Step S522, read the first disk information corresponding to the target database in the program data, and determine the second disk information according to the disk metadata loaded in the control node memory.

[0117] Step S524: Compare the first disk information with the second disk information. When it is determined that the disk metadata loaded in the control node memory is incomplete according to the comparison result, read supplementary disk metadata from the storage node based on the first disk information and the second disk information.

[0118] Step S526: Load the supplementary disk metadata into the control node memory, and execute the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0119] Furthermore, it is possible to detect whether the target database is in the disaster recovery mode thereafter; if so, switch the disaster recovery mode to the benchmark mode, and run the target database service associated with the target database according to the mode switching result; if not, reconstruct the target connection pool for the target database, and run the target database service associated with the target database according to the connection pool reconstruction result.

[0120] In summary, in order to protect against database exception events that have a low probability but still exist and avoid the inability to execute database operation requests submitted by computing nodes, program data and disk metadata can be backed up during the normal operation of the service. When a database operation request is submitted for the target database and the target database is abnormal at this time, the program data recorded in the backup period can be directly read from the local data file, and the disk metadata recorded in the backup period can be read from the storage node, so as to obtain the basic data that can support the normal execution of the database operation request. Then, the database service program associated with the target database can be started according to the program data, and at the same time, the disk metadata is loaded into the control node memory, so that the database service program can run normally, and the disk metadata associated with the database operation request can be read from the control node memory. Subsequently, the database operation task corresponding to the database operation request can be executed according to the program startup result and the metadata loading result. It is ensured that at any time, the execution of the database operation task will not be affected by database exceptions, and the impact of unavailable metadata on the IO link during database exceptions is solved, thereby achieving the purpose of automatic recovery without intervention and effectively reducing the impact on database operations.

[0121] Corresponding to the above method embodiment, this specification also provides a data processing system embodiment. Figure 6 It shows a schematic structural diagram of a data processing system provided by an embodiment of this specification. As Figure 6 shown, the data processing system 600 includes a cloud storage server 610, a storage node 620, and a control node 630. The system includes:

[0122] The cloud storage server 610 is configured to receive a database operation request submitted for a target database; when it is determined that the target database is abnormal according to the database operation request, read the program data recorded in the backup period from the local data file, and send a metadata acquisition request to the storage node;

[0123] The storage node 620 is configured to, in response to the metadata acquisition request, send the disk metadata recorded in the backup period to the cloud storage server;

[0124] The cloud storage server 610 is further configured to start the database service program associated with the target database according to the program data, and load the disk metadata into the control node memory of the control node 630; execute the database operation task corresponding to the database operation request according to the program start result and the metadata loading result.

[0125] In an optional embodiment, before the step of receiving the database operation request submitted for the target database, the method further includes: when the database service corresponding to the target database is in a running state, determining the target disk corresponding to the target database; reading the disk metadata corresponding to the target disk, loading the disk metadata into the control node memory, and backing it up to the storage node; determining the database service program corresponding to the database service, and reading the global program data corresponding to the database service program; determining the local program data for starting the database service program in the global program data as the program data, and loading it into the local data file.

[0126] In an optional embodiment, the step of reading the disk metadata corresponding to the target disk and loading the disk metadata into the control node memory includes: reading the initial disk metadata corresponding to the target disk, determining the metadata identifier corresponding to the initial disk metadata, and determining the backup metadata identifier corresponding to the backup disk metadata in the control node memory; when the backup metadata identifier is different from the metadata identifier, determining the disk metadata from the initial disk metadata according to the backup metadata identifier and the metadata identifier, and performing the step of loading the disk metadata into the control node memory.

[0127] In an optional embodiment, when it is determined that the target database is abnormal according to the database operation request, reading the program data recorded in the backup period from the local data file includes: detecting the program phase corresponding to the database service program in response to the database operation request; when the database service program is in the program startup phase, connecting to the target database at a set time interval and a set number of detection times, if the connection fails, determining that the target database is abnormal, and executing the step of reading the program data recorded in the backup period from the local data file; when the database service program is in the program running phase, determining the connection pool corresponding to the target database, and detecting the threads in the connection pool, if the detection fails, determining that the target database is abnormal, and executing the step of reading the program data recorded in the backup period from the local data file.

[0128] In an optional embodiment, before executing the database operation task step corresponding to the database operation request according to the program startup result and the metadata loading result, it further includes: reading the first disk information corresponding to the target database from the program data, and determining the second disk information according to the disk metadata loaded in the control node memory; comparing the first disk information and the second disk information; when it is determined that the disk metadata loaded in the control node memory is incomplete according to the comparison result, reading the supplementary disk metadata from the storage node according to the first disk information and the second disk information; loading the supplementary disk metadata into the control node memory, and executing the step of executing the database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0129] In an optional embodiment, starting the database service program associated with the target database according to the program data, and loading the disk metadata into the control node memory includes: loading the program data into the server memory, and starting the database service program associated with the target database according to the loading result; determining the control node associated with the database service program, and loading the disk metadata into the control node memory of the control node.

[0130] In an optional embodiment, loading the disk metadata into the control node memory of the control node includes: obtaining the data conversion relationship between the storage node and the control node; converting the disk metadata into target disk metadata according to the data conversion relationship; loading the target disk metadata into the control node memory of the control node.

[0131] In an alternative embodiment, after the step of receiving a database operation request submitted for a target database, the method further includes: when it is determined according to the database operation request that the target database is abnormal and the database service program is normal, reading disaster recovery disk metadata recorded in a backup period from a storage node; loading the disaster recovery disk metadata into the memory of the control node; and performing a database operation task corresponding to the database operation request according to the database service program in the running state and the metadata loading result.

[0132] In an alternative embodiment, after the step of performing a database operation task corresponding to the database operation request according to the program startup result and the metadata loading result, the method further includes: detecting whether the target database is in a disaster recovery mode; if so, switching the disaster recovery mode to a reference mode and running a target database service associated with the target database according to the mode switching result; if not, reconstructing a target connection pool for the target database and running a target database service associated with the target database according to the connection pool reconstruction result.

[0133] The data processing system provided in this embodiment realizes that at any time, the execution of database operation tasks will not be affected by database abnormalities, solves the problem of the unavailability of metadata affecting the IO link during database abnormalities, thereby achieving the purpose of automatic recovery without intervention, and effectively reducing the impact on database operations.

[0134] The above is a schematic solution of a data processing system according to this embodiment. It should be noted that the technical solution of this data processing system and the technical solution of the above data processing method belong to the same concept. For the details not described in detail in the technical solution of the data processing system, reference can be made to the description of the technical solution of the above data processing method.

[0135] Corresponding to the above method embodiment, this specification also provides an embodiment of a data processing device. Figure 7 The following shows a schematic structural diagram of a data processing device provided by an embodiment of this specification. As Figure 7 shown, the device includes:

[0136] A receiving module 702, configured to receive a database operation request submitted for a target database;

[0137] A reading module 704, configured to read program data recorded in a backup period from a local data file and read disk metadata recorded in the backup period from a storage node when it is determined according to the database operation request that the target database is abnormal;

[0138] The loading module 706 is configured to start the database service program associated with the target database according to the program data, and load the disk metadata into the memory of the management and control node;

[0139] The execution module 708 is configured to execute the database operation task corresponding to the database operation request according to the program start result and the metadata loading result.

[0140] In an optional embodiment, the apparatus further includes:

[0141] The backup module is configured to, when the database service corresponding to the target database is in a running state, determine the target disk corresponding to the target database; read the disk metadata corresponding to the target disk, load the disk metadata into the memory of the management and control node, and back it up to the storage node; determine the database service program corresponding to the database service, and read the global program data corresponding to the database service program; determine the local program data for starting the database service program in the global program data as the program data, and load it into the local data file.

[0142] In an optional embodiment, the backup module is further configured to:

[0143] Read the initial disk metadata corresponding to the target disk, determine the metadata identifier corresponding to the initial disk metadata, and determine the backup metadata identifier corresponding to the backup disk metadata in the memory of the management and control node; in the case where the backup metadata identifier is different from the metadata identifier, determine the disk metadata from the initial disk metadata according to the backup metadata identifier and the metadata identifier, and perform the step of loading the disk metadata into the memory of the management and control node.

[0144] In an optional embodiment, the reading module 704 is further configured to:

[0145] Detect the program phase corresponding to the database service program in response to the database operation request; in the case where the database service program is in the program startup phase, connect to the target database at a set time interval and a set number of detection times, if the connection fails, determine that the target database is abnormal, and perform the step of reading the program data of the backup period record in the local data file; in the case where the database service program is in the program running phase, determine the connection pool corresponding to the target database, and detect the threads in the connection pool, if the detection fails, determine that the target database is abnormal, and perform the step of reading the program data of the backup period record in the local data file.

[0146] In an optional embodiment, the apparatus further includes:

[0147] A comparison module, configured to read, from the program data, first disk information corresponding to the target database, and determine second disk information according to disk metadata loaded in the memory of the management and control node; compare the first disk information with the second disk information; when it is determined that the disk metadata loaded in the memory of the management and control node is incomplete according to the comparison result, read supplementary disk metadata in the storage node according to the first disk information and the second disk information; load the supplementary disk metadata into the memory of the management and control node, and execute the step of performing a database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

[0148] In an optional embodiment, the loading module 706 is further configured to:

[0149] Load the program data into the memory of the server, start a database service program associated with the target database according to the loading result; determine a management and control node associated with the database service program, and load the disk metadata into the memory of the management and control node of the management and control node.

[0150] In an optional embodiment, the loading module 706 is further configured to:

[0151] Obtain a data conversion relationship between the storage node and the management and control node; convert the disk metadata into target disk metadata according to the data conversion relationship; load the target disk metadata into the memory of the management and control node of the management and control node.

[0152] In an optional embodiment, the apparatus further includes:

[0153] A disaster recovery module, configured to, when it is determined according to the database operation request that the target database is abnormal and the database service program is normal, read disaster recovery disk metadata recorded in the backup period in the storage node; load the disaster recovery disk metadata into the memory of the management and control node; perform a database operation task corresponding to the database operation request according to the running database service program and the metadata loading result.

[0154] In an optional embodiment, the apparatus further includes:

[0155] A detection mode module, configured to detect whether the target database is in a disaster recovery mode; if so, switch the disaster recovery mode to a reference mode, and run a target database service associated with the target database according to the mode switching result; if not, rebuild a target connection pool for the target database, and run a target database service associated with the target database according to the connection pool rebuilding result.

[0156] The data processing device provided in this embodiment can ensure that at any time, the execution of database operation tasks will not be affected by database anomalies, solve the problem of the unavailability of metadata for the IO link during database anomalies, thereby achieving the goal of automatic recovery without intervention and effectively reducing the impact on database operations.

[0157] The above is a schematic solution of a data processing device according to this embodiment. It should be noted that the technical solution of this data processing device and the technical solution of the above data processing method belong to the same concept. For the details not described in the technical solution of the data processing device, reference can be made to the description of the technical solution of the above data processing method.

[0158] Figure 8 The structural block diagram of a computing device 800 provided according to an embodiment of this specification is shown. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 through a bus 830, and a database 850 is used to store data.

[0159] The computing device 800 further includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include the Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 840 can include one or more of any type of wired or wireless network interfaces (for example, a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC).

[0160] In an embodiment of this specification, the above components of the computing device 800 and Figure 8 other components not shown in the figure may also be connected to each other, for example, through a bus. It should be understood that Figure 8The block diagram of the computing device shown is for illustrative purposes only and is not a limitation on the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0161] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 800 can also be a mobile or stationary server.

[0162] Among them, the processor 820 is used to execute the following computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the above data processing method are implemented.

[0163] The above is a schematic solution of a computing device in this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above data processing method belong to the same concept. For the details not described in detail in the technical solution of the computing device, reference can be made to the description of the technical solution of the above data processing method.

[0164] An embodiment of this specification also provides a computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the above data processing method are implemented.

[0165] The above is a schematic solution of a computer-readable storage medium in this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above data processing method belong to the same concept. For the details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solution of the above data processing method.

[0166] An embodiment of this specification also provides a computer program, where when the computer program is executed on a computer, the computer is made to execute the steps of the above data processing method.

[0167] The above is a schematic solution of a computer program in this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the above data processing method belong to the same concept. For the details not described in detail in the technical solution of the computer program, reference can be made to the description of the technical solution of the above data processing method.

[0168] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0169] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, removable hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0170] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of this specification are not limited by the described order of actions, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0171] In the above embodiments, the descriptions of the various embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0172] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The alternative embodiments do not elaborate on all the details and do not limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can well understand and utilize this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data processing method applied to a cloud storage server, comprising: Receiving a database operation request submitted for a target database; When it is determined that the target database is abnormal according to the database operation request, reading the program data recorded in the backup period from a local data file and reading the disk metadata recorded in the backup period from a storage node; Starting a database service program associated with the target database according to the program data, and loading the disk metadata into the memory of a management and control node; Performing a database operation task corresponding to the database operation request according to the program startup result and the metadata loading result.

2. The data processing method according to claim 1, before the step of receiving a database operation request submitted for a target database, further comprising: When the database service corresponding to the target database is in a running state, determining a target disk corresponding to the target database; Reading the disk metadata corresponding to the target disk, loading the disk metadata into the memory of a management and control node, and backing it up to a storage node; Determining a database service program corresponding to the database service, and reading global program data corresponding to the database service program; Determining local program data for starting the database service program in the global program data as program data, and loading it into a local data file.

3. The data processing method according to claim 2, the step of reading the disk metadata corresponding to the target disk and loading the disk metadata into the memory of a management and control node, comprising: Reading initial disk metadata corresponding to the target disk, determining a metadata identifier corresponding to the initial disk metadata, and determining a backup metadata identifier corresponding to the backup disk metadata in the memory of the management and control node; When the backup metadata identifier is different from the metadata identifier, determining the disk metadata from the initial disk metadata according to the backup metadata identifier and the metadata identifier, and performing the step of loading the disk metadata into the memory of the management and control node.

4. The data processing method according to claim 1, the step of reading the program data recorded in the backup period from a local data file when it is determined that the target database is abnormal according to the database operation request, comprising: Detecting a program phase corresponding to the database service program in response to the database operation request; When the database service program is in a program startup phase, connecting to the target database at set time intervals and set detection times. If the connection fails, determining that the target database is abnormal, and performing the step of reading the program data recorded in the backup period from a local data file; When the database service program is in a program running phase, determining a connection pool corresponding to the target database, and detecting threads in the connection pool. If the detection fails, determining that the target database is abnormal, and performing the step of reading the program data recorded in the backup period from a local data file.

5. The data processing method according to any one of claims 1-4, before the step of executing the database operation task corresponding to the database operation request according to the program start result and the metadata loading result, further includes: Reading, from the program data, first disk information corresponding to the target database, and determining second disk information according to the disk metadata loaded in the control node memory; Comparing the first disk information with the second disk information; When it is determined, according to the comparison result, that the disk metadata loaded in the control node memory is incomplete, reading supplementary disk metadata in the storage node according to the first disk information and the second disk information; Loading the supplementary disk metadata into the control node memory, and executing the step of executing the database operation task corresponding to the database operation request according to the program start result and the metadata loading result.

6. The data processing method according to claim 1, the step of starting the database service program associated with the target database according to the program data, and loading the disk metadata into the control node memory, includes: Loading the program data into the server memory, and starting the database service program associated with the target database according to the loading result; Determining the control node associated with the database service program, and loading the disk metadata into the control node memory of the control node.

7. The data processing method according to claim 6, the step of loading the disk metadata into the control node memory of the control node, includes: Obtaining the data conversion relationship between the storage node and the control node; Converting the disk metadata into target disk metadata according to the data conversion relationship; Loading the target disk metadata into the control node memory of the control node.

8. The data processing method according to claim 1, after the step of receiving a database operation request submitted for a target database, further includes: When it is determined, according to the database operation request, that the target database is abnormal and the database service program is normal, reading the disaster recovery disk metadata recorded in the backup period in the storage node; Loading the disaster recovery disk metadata into the control node memory; Executing the database operation task corresponding to the database operation request according to the running state of the database service program and the metadata loading result.

9. The data processing method according to any one of claims 1-4, after the step of executing the database operation task corresponding to the database operation request according to the program start result and the metadata loading result, further includes: Detecting whether the target database is in a disaster recovery mode; If so, switching the disaster recovery mode to a reference mode, and running the target database service associated with the target database according to the mode switching result; If not, reconstructing a target connection pool for the target database, and running the target database service associated with the target database according to the connection pool reconstruction result.

10. A data processing system, including a cloud storage server, a storage node, and a control node, the system includes: The cloud storage server is configured to receive a database operation request submitted for a target database; When it is determined that the target database is abnormal according to the database operation request, read the program data recorded in the backup period from the local data file, and send a metadata acquisition request to the storage node; The storage node is configured to, in response to the metadata acquisition request, send the disk metadata recorded in the backup period to the cloud storage server; The cloud storage server is further configured to start the database service program associated with the target database according to the program data, and load the disk metadata into the memory of the management and control node; perform the database operation task corresponding to the database operation request according to the program start result and the metadata loading result.

11. A data processing device, applied to a cloud storage server, includes: A receiving module configured to receive a database operation request submitted for a target database; A reading module configured to, when it is determined that the target database is abnormal according to the database operation request, read the program data recorded in the backup period from the local data file, and read the disk metadata recorded in the backup period from the storage node; A loading module configured to start the database service program associated with the target database according to the program data, and load the disk metadata into the memory of the management and control node; An execution module configured to perform the database operation task corresponding to the database operation request according to the program start result and the metadata loading result.

12. A computing device includes: A memory and a processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images