Data processing method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of databases, and discloses a data processing method and device, computer equipment and a storage medium. The production library comprises a main node and a plurality of standby nodes; according to a preset delay library construction strategy, selecting a target standby node from the plurality of standby nodes, and constructing a corresponding delay library based on the target standby node; replication delay time between the main node and the delay library is set for the delay library, and data are synchronized to the delay library from the main node based on the replication delay time; therefore, the delayed copying of the data is realized by providing the delay library, and the data recovery speed of the production library during misoperation can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of database technology, and more specifically to a data processing method, apparatus, computer equipment, and computer-readable storage medium. Background Technology

[0002] Currently, in modern database systems, high availability and reliability of production database data are crucial for ensuring business continuity. Traditional database architectures typically employ master-slave replication, where the master node handles all write operations, while slave nodes handle read operations and synchronize data from the master. While this architecture provides a degree of redundancy and fault recovery, if the production database master node experiences a erroneous operation (such as accidental deletion or update), all slave nodes will synchronously perform the erroneous operation, resulting in all copies of the production database becoming corrupted. Even if backup data is available for data recovery, the recovery time can be lengthy, especially when the production database contains a large amount of data.

[0003] Therefore, how to provide a data processing method, apparatus, computer equipment, and computer-readable storage medium that enables delayed data copying by providing a delayed library, thereby effectively improving the speed of production library data recovery in case of erroneous operation, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a data processing method, apparatus, computer equipment and computer-readable storage medium, which can effectively improve the speed of production database data recovery in case of erroneous operation by providing a delayed library to achieve delayed data copying.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a data processing method, comprising: Identify the production database to be processed; wherein the production database includes a master node and multiple backup nodes; According to the preset delay library construction strategy, a target backup node is selected from the multiple backup nodes, and a corresponding delay library is constructed based on the target backup node; Set a replication delay time between the delay library and the master node, and synchronize data from the master node to the delay library based on the replication delay time.

[0006] In a second aspect, the present invention provides a data processing apparatus, comprising: The determination module is used to determine the production database to be processed; wherein the production database includes a master node and multiple backup nodes; The construction module is used to select a target backup node from the plurality of backup nodes according to a preset delay library construction strategy, and construct a corresponding delay library based on the target backup node. The configuration module is used to set the replication delay time between the delay library and the master node, and to synchronize data from the master node to the delay library based on the replication delay time.

[0007] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data processing method described above.

[0008] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the data processing method described above.

[0009] Compared to existing technologies, this invention provides a data processing method, apparatus, computer equipment, and computer-readable storage medium. The method involves determining a production database for data processing, wherein the production database includes a master node and multiple backup nodes. Based on a preset delayed database construction strategy, a target backup node is selected from the multiple backup nodes, and a corresponding delayed database is constructed based on the target backup node. A replication delay time is set between the delayed database and the master node, and data is synchronized from the master node to the delayed database based on the replication delay time. Thus, this invention achieves delayed data replication by providing a delayed database, effectively improving the speed of data recovery from the production database in case of erroneous operations. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the application environment of a data processing method provided in an embodiment of the present invention.

[0012] Figure 2 This is a flowchart illustrating a data processing method according to an embodiment of the present invention.

[0013] Figure 3 This is another schematic diagram of a data processing method provided in an embodiment of the present invention.

[0014] Figure 4 This is a schematic diagram of a program module of a data processing device provided in an embodiment of the present invention.

[0015] Figure 5 This is a schematic diagram of another program module of a data processing device provided in an embodiment of the present invention.

[0016] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention.

[0017] Figure 7 This is another structural schematic diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0021] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0022] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0024] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0025] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0026] An embodiment of the present invention provides a data processing method that can be applied to, for example... Figure 1 In the application environment shown, the client and server communicate via a network. The client includes, but is not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0027] Please see Figures 2-3 An embodiment of the present invention provides a data processing method, wherein the method includes the following steps: S100. Determine the production database to be processed; wherein, the production database includes a master node and multiple backup nodes; S200. According to the preset delay library construction strategy, select a target backup node from the multiple backup nodes, and construct a corresponding delay library based on the target backup node. S300: Set a replication delay time between the delay library and the master node, and synchronize data from the master node to the delay library based on the replication delay time.

[0028] Furthermore, the data processing method, after setting a replication delay time between the delay library and the master node, and synchronizing data from the master node to the delay library based on the replication delay time, further includes the step of: S400. When a misoperation is detected in the master node, the data synchronization task between the delay library and the master node is stopped, and the target data corresponding to the misoperation is determined based on the delay library. The target data is then replayed to the master node to achieve data recovery of the master node.

[0029] In practical implementation, the data processing method of this embodiment achieves delayed data replication by constructing a delayed database and utilizing its delayed synchronization mechanism. This effectively improves the speed of production database data recovery in case of erroneous operations, mainly in the following aspects: 1. Timely Response of the Delayed Repository: When a malfunction occurs on the master node, the delayed repository can immediately stop its data synchronization task with the master node. Since the data in the delayed repository remains unchanged during the replication delay period, it preserves "clean" data unaffected by the malfunction. This mechanism allows the delayed repository to quickly provide the data needed for recovery without waiting to restore from backups, thus significantly reducing data recovery time.

[0030] 2. Precise Target Data Location: By comparing data from the master node and the delayed database, the scope of data affected by erroneous operations can be accurately determined, and unaffected target data can be extracted. This precise location reduces unnecessary data processing and recovery operations, further improving recovery efficiency.

[0031] 3. Dynamic Latency Adjustment: This method supports dynamically adjusting replication latency based on the production environment's workload and data recovery needs. This means that during high-load periods, latency can be appropriately increased to reduce the impact on the production database, while during low-load periods, latency can be reduced to improve the timeliness of data recovery. This flexibility ensures that in the event of a misoperation, the delayed database can provide the most accurate recovery data while minimizing disruption to business operations.

[0032] 4. Intelligent Misoperation Detection: By monitoring the master node's operation logs in real time, the system can intelligently detect misoperations and take immediate action upon detection. This rapid response mechanism ensures that misoperations are detected and handled promptly, preventing further propagation and impact on more data, thereby reducing the scope and time required for recovery.

[0033] 5. High Efficiency of Replay Operations: The process of replaying target data from the delayed database to the master node is efficient and automated. This method avoids the complexity and time consumption of manual data recovery, ensuring rapid execution of the recovery operation. Simultaneously, data consistency checks can be performed during replay to ensure the accuracy of the recovered data.

[0034] In other words, this data processing method achieves delayed data replication by constructing a delayed database and utilizing its delayed synchronization mechanism. Combined with intelligent misoperation detection, dynamic delay time adjustment, and efficient data replay operation, it can effectively improve the speed of production database data recovery in the event of misoperation, reduce business interruption time, and improve the overall reliability and availability of the system.

[0035] Furthermore, in one embodiment, the data processing method, wherein determining the production database to be processed specifically includes the following steps: Identify all available database instances in the current production environment; Based on production requirements, the database instances are selected to obtain one master node and multiple backup nodes; Based on the master node and the multiple backup nodes, a production database for data processing is obtained.

[0036] Furthermore, the data processing method, wherein selecting a target backup node from the plurality of backup nodes according to a preset delay library construction strategy, and constructing a corresponding delay library based on the target backup node, specifically includes the following steps: Based on the production environment and production requirements, determine the strategy for building the delayed library; According to the delay library construction strategy, a target backup node is selected from the plurality of backup nodes, and the delay library is configured and started based on the selected target backup node.

[0037] Furthermore, the data processing method, wherein setting a replication delay time between the delay database and the master node, and synchronizing data from the master node to the delay database based on the replication delay time, specifically includes the following steps: Based on the latency library construction strategy, the replication latency time between the latency library and the master node is determined; Configure the copy delay time in the delay library and detect whether the delay library is in a delay preparation state; When the delay library is in the ready-to-delay state, data is synchronized from the master node to the delay library based on the configured replication delay time.

[0038] In practice, the specific implementation process of the steps in this embodiment is roughly as follows: 1. Identify all available database instances in the current production environment: Use automated scanning tools to comprehensively identify all available database instances in the current production environment, including primary and standby nodes.

[0039] This tool uses the database management system's API interface to periodically collect key performance indicators of instances, such as CPU utilization, memory utilization, disk I / O, and network latency, and analyzes this data using machine learning algorithms to assess the health status of instances.

[0040] 2. Based on production requirements, filter the database instances to obtain one master node and multiple slave nodes: Based on preset high availability and reliability standards, the configuration and performance of each database instance are evaluated, and the primary node and multiple backup nodes that meet the requirements are selected.

[0041] The production requirements include, but are not limited to: the geographical distribution of primary and backup nodes, network redundancy design, data backup frequency, and failover time.

[0042] 3. Based on the master node and the multiple backup nodes, the production database to be processed is obtained: Analyze the business load of the selected primary and backup nodes to ensure that data processing will not significantly impact business operations.

[0043] Verify the data consistency between the primary and backup nodes to ensure data integrity and accuracy.

[0044] Based on the above analysis and verification results, the production database to be processed was determined.

[0045] 4. Based on the production environment and production requirements, determine the delayed library construction strategy: Analyze the high availability, fault tolerance, and data recovery requirements of the production environment to determine the latency of the latency database, the geographical distribution of backup nodes, network latency optimization, and data synchronization integrity checks.

[0046] The production requirements include, but are not limited to: the geographical distribution of primary and backup nodes, network redundancy design, data backup frequency, and failover time.

[0047] 5. Based on the latency library construction strategy, select a target backup node from the plurality of backup nodes, and configure and start the latency library based on the selected target backup node: Utilize automated evaluation tools to monitor the performance and health status of backup nodes in real time, and select the node with the best performance and good health status as the target backup node.

[0048] Configure and start the delay library based on the selected target backup node.

[0049] 6. Based on the latency library construction strategy, determine the replication latency between the latency library and the master node: Analyze the business load patterns and historical data operation frequency of the production database, and use machine learning algorithms to predict the optimal replication latency.

[0050] Increase latency appropriately during high-load periods to reduce the impact on the production database, and reduce latency during low-load periods to improve the timeliness of data recovery.

[0051] 7. Configure the replication delay time in the delay library and check whether the delay library is in a delay preparation state: Configure the replication delay time in the delayed database using the database management system's configuration tools.

[0052] Monitor the performance metrics of the latency library in real time (such as CPU utilization, memory utilization, disk I / O, and network latency) to ensure that the performance of the latency library meets expectations after configuration.

[0053] Check if the delay library has entered the delay preparation state and ensure that all configurations are correct.

[0054] 8. When the delay database is in the ready-to-delay state, data is synchronized from the master node to the delay database based on the configured replication delay time: Start data synchronization and synchronize data from the master node to the delayed database based on the configured replication delay time.

[0055] During the synchronization process, data consistency is monitored in real time, and data inconsistency issues are automatically corrected to ensure the accuracy of data in the delayed database.

[0056] The detailed implementation process described above ensures the clarity and operability of each step, providing a solid foundation for the entire data processing methodology. These steps not only improve the speed of production database data recovery but also enhance the overall reliability and availability of the system.

[0057] Further, in one embodiment, the data processing method, wherein when a misoperation is detected on the master node, stopping the data synchronization task between the delay database and the master node, determining the target data corresponding to the misoperation based on the delay database, and replaying the target data back to the master node to achieve data recovery of the master node, specifically includes the following steps: The operation data of the master node is monitored in real time. When an erroneous operation is detected in the operation data, the data synchronization task between the delay database and the master node is stopped, and the delay database is switched to a temporary master node, and the master node is switched to a temporary backup node. The master node is compared with the data in the delay database, and the target data corresponding to the erroneous operation is determined based on the comparison result; wherein, the target data is the data in the delay database that has not been synchronized within the replication delay time. The target data is replayed from the delay library to the master node to restore the data of the master node, and the temporary master node function of the delay library is revoked and the master node function of the master node is restored.

[0058] Furthermore, the data processing method, wherein the real-time monitoring of the master node's operation data, and stopping the data synchronization task between the delayed library and the master node when an erroneous operation is detected in the operation data, specifically includes the following steps: Analyze the SQL operation logs of the master node in real time; Based on a preset abnormal operation identification strategy, detect whether there are any erroneous operations in the SQL operation log; When the erroneous operation is detected in the SQL operation log, the data synchronization task between the delayed database and the master node is stopped.

[0059] In practice, the specific implementation process of the steps in this embodiment is roughly as follows: 1. Real-time analysis of the master node's SQL operation logs: Utilize real-time monitoring tools to analyze the SQL operation logs of the master node, recording the specific content and timestamps of all SQL operations.

[0060] The real-time monitoring tool can automatically identify and record the specific type and timestamp of erroneous operations, and supports the detection of multiple types of erroneous operations, such as deletion, update, and insertion.

[0061] The monitoring tool captures all SQL operation logs through the database's auditing function and uses machine learning algorithms to analyze operation patterns and identify abnormal operations.

[0062] 2. Based on the preset abnormal operation identification strategy, detect whether there are any erroneous operations in the SQL operation log: The preset abnormal operation identification strategies include, but are not limited to: operation type, operation frequency, operation object, and consistency of business logic of the operation.

[0063] Machine learning algorithms are used to analyze SQL operation logs and automatically identify abnormal operation patterns. For example, when a large number of delete operations or update operations that do not conform to business logic are detected, the system will mark them as erroneous operations.

[0064] By setting thresholds and rules, behaviors that do not conform to normal operating patterns can be identified, such as deleting a large number of records in a short period of time or making unauthorized updates to critical tables.

[0065] 3. When an erroneous operation is detected in the SQL operation log, stop the data synchronization task between the delayed database and the primary node, switch the delayed database to the temporary primary node, and switch the primary node to the temporary backup node: Upon detecting a misoperation, immediately stop the data synchronization task between the delayed database and the master node to ensure that the data in the delayed database is not affected by the misoperation. The data synchronization task can be quickly paused through the database synchronization management module, and the time of pausing synchronization can be recorded to ensure the accuracy of subsequent operations.

[0066] Switch the delayed database to a temporary master node to take over the production database, and switch the master node to a temporary backup node and connect it to the new production database.

[0067] 4. Compare the data on the master node with the data in the delay database, and determine the target data corresponding to the erroneous operation based on the comparison results: By using data comparison tools, the data differences between the master node and the delay database are compared to determine the scope of data affected by the erroneous operation.

[0068] The target data refers to data in the delay library that was not synchronized during the replication delay period. This data is unaffected by erroneous operations and can be used to recover the master node.

[0069] Target data can be extracted using data snapshots and incremental backups. The comparison tool supports intelligent analysis, enabling it to quickly locate data affected by erroneous operations and extract unaffected target data.

[0070] 5. Replay the target data from the delayed database to the master node to achieve data recovery on the master node: The identified target data is gradually replayed from the delayed database to the master node, supporting intelligent conflict detection and automatic repair to ensure the accuracy and consistency of data recovery.

[0071] During playback, data conflicts are monitored in real time, and conflicting data is automatically repaired to ensure the integrity and accuracy of data recovery on the master node.

[0072] Data synchronization tools can be used to gradually replay target data back to the master node. During the replay process, data conflicts are monitored in real time, and conflicting data is automatically repaired. For example, if the master node performs new legitimate operations after synchronization stops, the system will automatically merge these operations with the target data to ensure data consistency.

[0073] 6. Revoke the temporary master node function of the delayed library and restore the master node function of the master node: After the master node data is restored, the temporary master node function of the delayed database is revoked, and the master node function of the master node is restored to ensure that the data of the master node and the delayed database are consistent and to restore the normal master-slave replication relationship; at the same time, the delayed synchronization state between the delayed database and the master node is restored.

[0074] Through the specific implementation process described above, this data processing method can effectively monitor the operation data of the master node, promptly detect erroneous operations and stop data synchronization, accurately identify the target data, and replay it to the master node, achieving rapid data recovery. Simultaneously, by delaying the switching between the production database and the master node, the normal operation of the production database can be effectively guaranteed before data recovery on the master node. This method not only improves the speed of data recovery but also enhances the overall reliability and availability of the system.

[0075] As can be seen from the above method embodiments, the data processing method provided by the present invention includes: determining a production database to be processed; wherein the production database includes a master node and multiple backup nodes; selecting a target backup node from the multiple backup nodes according to a preset delayed database construction strategy, and constructing a corresponding delayed database based on the target backup node; setting a replication delay time between the delayed database and the master node, and synchronizing data from the master node to the delayed database based on the replication delay time; when a misoperation is detected on the master node, stopping the data synchronization task between the delayed database and the master node, determining the target data corresponding to the misoperation based on the delayed database, and replaying the target data to the master node to achieve data recovery of the master node. Thus, the method of the present invention, by constructing a delayed database and utilizing its delayed synchronization mechanism, can effectively improve the speed of production database data recovery in the event of a misoperation.

[0076] It should be understood that although this application provides the method operation steps as described in the embodiments or flowcharts, conventional or non-inventive labor may include more or fewer operation steps, and these operation steps are not necessarily executed sequentially according to the order of the embodiments or flowcharts. The order of steps listed in the embodiments or flowcharts is merely one way of executing many steps and does not represent the only execution order. It should be noted that there is no necessary sequential order between the above steps. Those skilled in the art can understand from the description of the embodiments of the present invention that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in exchange, etc. Moreover, at least some steps in the embodiments or flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn, alternately, or synchronously with other steps or at least a part of the sub-steps or stages of other steps.

[0077] Based on the above method embodiments, please refer to Figures 4-5 Another embodiment of the present invention also provides a data processing apparatus, wherein the apparatus includes: The determination module 11 is used to determine the production database to be processed; wherein the production database includes a master node and multiple backup nodes; The construction module 12 is used to select a target backup node from the plurality of backup nodes according to a preset delay library construction strategy, and construct a corresponding delay library based on the target backup node. The setting module 13 is used to set the replication delay time between the delay library and the master node, and to synchronize data from the master node to the delay library based on the replication delay time.

[0078] Furthermore, the data processing apparatus further includes: The playback module 14 is used to stop the data synchronization task between the delay library and the master node when a misoperation is detected in the master node, and to determine the target data corresponding to the misoperation based on the delay library, and to play back the target data to the master node to realize the data recovery of the master node.

[0079] Furthermore, in one embodiment, the data processing apparatus, wherein determining the production database to be processed specifically includes: Identify all available database instances in the current production environment; Based on production requirements, the database instances are selected to obtain one master node and multiple backup nodes; Based on the master node and the multiple backup nodes, a production database for data processing is obtained.

[0080] Furthermore, in the data processing apparatus, the step of selecting a target backup node from the plurality of backup nodes according to a preset latency library construction strategy, and constructing a corresponding latency library based on the target backup node, specifically includes: Based on the production environment and production requirements, determine the strategy for building the delayed library; According to the delay library construction strategy, a target backup node is selected from the plurality of backup nodes, and the delay library is configured and started based on the selected target backup node.

[0081] Further, in the data processing apparatus, the step of setting a replication delay time between the delay library and the master node, and synchronizing data from the master node to the delay library based on the replication delay time, specifically includes: Based on the latency library construction strategy, the replication latency time between the latency library and the master node is determined; Configure the copy delay time in the delay library and detect whether the delay library is in a delay preparation state; When the delay library is in the ready-to-delay state, data is synchronized from the master node to the delay library based on the configured replication delay time.

[0082] Further, in one embodiment, the data processing apparatus, wherein when a malfunction is detected in the master node, stops the data synchronization task between the delay database and the master node, determines the target data corresponding to the malfunction based on the delay database, and replays the target data to the master node to achieve data recovery of the master node, specifically includes: The operation data of the master node is monitored in real time. When an erroneous operation is detected in the operation data, the data synchronization task between the delay database and the master node is stopped, and the delay database is switched to a temporary master node, and the master node is switched to a temporary backup node. The master node is compared with the data in the delay database, and the target data corresponding to the erroneous operation is determined based on the comparison result; wherein, the target data is the data in the delay database that has not been synchronized within the replication delay time. The target data is replayed from the delay library to the master node to restore the data of the master node, and the temporary master node function of the delay library is revoked and the master node function of the master node is restored.

[0083] Furthermore, in the aforementioned data processing device, the real-time monitoring of the master node's operation data, and stopping the data synchronization task between the delayed library and the master node when an erroneous operation is detected in the operation data, specifically includes: Analyze the SQL operation logs of the master node in real time; Based on a preset abnormal operation identification strategy, detect whether there are any erroneous operations in the SQL operation log; When the erroneous operation is detected in the SQL operation log, the data synchronization task between the delayed database and the master node is stopped.

[0084] It should be noted that, in the device embodiments of the present invention, the information interaction and execution process between the above modules are based on the same concept as in the method embodiments of the present invention. For details on their specific functions and the resulting technical effects, please refer to the aforementioned method embodiments section, which will not be repeated here.

[0085] Based on the above method embodiments, another embodiment of the present invention also provides a computer device, which can be a server, and its internal structure diagram can be as follows. Figure 6As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the data processing method on the server side as described in any of the above method embodiments.

[0086] Based on the above method embodiments, another embodiment of the present invention also provides a computer device, which can be a client, and its internal structure diagram can be as follows. Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements the functions or steps of the data processing method on the client side as described in any of the above method embodiments.

[0087] Those skilled in the art will understand that Figure 6 and Figure 7 The structural diagram shown is only a schematic diagram of a part of the structure related to the present invention and does not constitute a limitation on the computer device on which the present invention is applied. The specific computer device may include more components than shown in the figure, or combine certain components, or have different component arrangements.

[0088] The processor referred to herein can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0089] The memory includes readable storage media, internal memory, etc., where internal memory can be the RAM of a computer device. Internal memory provides an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of the computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal storage units and external storage devices of the computer device. The memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. The memory can also be used to temporarily store data that has been output or will be output.

[0090] Based on the above method embodiments, another embodiment of the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the data processing method as described in any of the above method embodiments. The computer-readable storage medium may be non-volatile or volatile.

[0091] It should be noted that the functions or steps that can be achieved by the computer-readable storage medium or computer device, and the technical effects brought about by the functions / steps, can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0092] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc. The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0093] Those skilled in the art will understand that, for the sake of convenience and brevity, the embodiments of the device of the present invention are only illustrated by the division of the above-mentioned functional units and modules. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the present invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0094] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0095] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0096] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0097] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A data processing method, characterized in that, include: Identify the production database to be processed; wherein the production database includes a master node and multiple backup nodes; According to the preset delay library construction strategy, a target backup node is selected from the multiple backup nodes, and a corresponding delay library is constructed based on the target backup node; Set a replication delay time between the delay library and the master node, and synchronize data from the master node to the delay library based on the replication delay time.

2. The data processing method according to claim 1, characterized in that, The process of determining the production database to be processed includes: Identify all available database instances in the current production environment; Based on production requirements, the database instances are selected to obtain one master node and multiple backup nodes; Based on the master node and the multiple backup nodes, a production database for data processing is obtained.

3. The data processing method according to claim 2, characterized in that, The step of selecting a target backup node from the plurality of backup nodes according to a preset latency library construction strategy, and constructing a corresponding latency library based on the target backup node, includes: Based on the production environment and production requirements, determine the strategy for building the delayed library; According to the delay library construction strategy, a target backup node is selected from the plurality of backup nodes, and the delay library is configured and started based on the selected target backup node.

4. The data processing method according to claim 3, characterized in that, Setting a replication delay time between the delay database and the master node, and synchronizing data from the master node to the delay database based on the replication delay time, includes: Based on the latency library construction strategy, the replication latency time between the latency library and the master node is determined; Configure the copy delay time in the delay library and detect whether the delay library is in a delay preparation state; When the delay library is in the ready-to-delay state, data is synchronized from the master node to the delay library based on the configured replication delay time.

5. The data processing method according to any one of claims 1-4, characterized in that, After setting a replication delay time between the delay database and the master node, and synchronizing data from the master node to the delay database based on the replication delay time, the method further includes: When a misoperation is detected on the master node, the data synchronization task between the delay library and the master node is stopped, and the target data corresponding to the misoperation is determined based on the delay library. The target data is then replayed to the master node to achieve data recovery of the master node.

6. The data processing method according to claim 5, characterized in that, When a misoperation is detected on the master node, the data synchronization task between the delay database and the master node is stopped, and the target data corresponding to the misoperation is determined based on the delay database. The target data is then replayed to the master node to achieve data recovery on the master node, including: The operation data of the master node is monitored in real time. When an erroneous operation is detected in the operation data, the data synchronization task between the delay database and the master node is stopped, and the delay database is switched to a temporary master node, and the master node is switched to a temporary backup node. The master node is compared with the data in the delay database, and the target data corresponding to the erroneous operation is determined based on the comparison result; wherein, the target data is the data in the delay database that has not been synchronized within the replication delay time. The target data is replayed from the delay library to the master node to restore the data of the master node, and the temporary master node function of the delay library is revoked and the master node function of the master node is restored.

7. The data processing method according to claim 6, characterized in that, The real-time monitoring of the master node's operation data, and the cessation of the data synchronization task between the delayed library and the master node when an erroneous operation is detected in the operation data, includes: Analyze the SQL operation logs of the master node in real time; Based on a preset abnormal operation identification strategy, detect whether there are any erroneous operations in the SQL operation log; When the erroneous operation is detected in the SQL operation log, the data synchronization task between the delayed database and the master node is stopped.

8. A data processing apparatus, characterized in that, include: The determination module is used to determine the production database to be processed; wherein the production database includes a master node and multiple backup nodes; The construction module is used to select a target backup node from the plurality of backup nodes according to a preset delay library construction strategy, and construct a corresponding delay library based on the target backup node. The configuration module is used to set the replication delay time between the delay library and the master node, and to synchronize data from the master node to the delay library based on the replication delay time.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the data processing method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the data processing method as described in any one of claims 1-7.

Images