Method of master-slave database data synchronization

Abstract

The embodiment of the present application provides a method for master-slave database data synchronization. The master-slave database data synchronization method is used for a first device deployed with a first database; the method includes: monitoring a second device deployed with a second database; the first database and the second database operate based on a dual-write mechanism; in the case of monitoring an abnormality of the second device, adding a first-class tag to the target data generated in the first database after the moment when the second device is monitored to change from normal to abnormal; in the case of monitoring that the second device has returned to normal, generating a synchronization request based on the first-class tag and the data identified by the first-class tag, and sending the synchronization request to the second device; the synchronization request carries a synchronization operation command for synchronizing the data identified by the first-class tag in the second database.

Description

Technical Field

[0001] The present application relates to the field of computer data storage, more specifically to the field of databases, and in particular to a method for synchronizing master-slave database data. Background Art

[0002] A database is a warehouse used to organize, store and manage data. A database management system that is compatible with a database enables data administrators to manage data stored in computers more conveniently and intuitively. If all data is stored in a database, then when the database fails or the computer equipment that deploys the database fails, the data in the database may be irreversibly lost.

[0003] Therefore, in order to ensure data persistence and non-volatility, a single database is usually not used to store data, but a master-slave database (or master-slave database) is used for data storage. The so-called master-slave database refers to deploying at least two databases in a storage system, that is, one database is the master database and the other is the slave database (of course, there can be two or more slave databases). The data stored in the master database is also stored in the slave database to achieve the purpose of backup. When the data in the master database is lost, the data can still be obtained from the slave database; similarly, when the data in the slave database is lost, the data can still be obtained from the master database.

[0004] In order to further ensure the persistence and non-volatility of data, the master database and the slave database are usually deployed in different computer devices. Among them, the computer device where the master database is deployed is usually called the master device, and the computer device where the slave database is deployed is usually called the slave device. The master device or the slave device runs independently, so the master database and the slave database also run independently, and the two do not interfere with each other. When one of the master database and the slave database fails, it will not affect the operation of the other normal database, which further improves the persistence and non-volatility of data. Summary of the invention

[0005] In the related art, when the master device (or slave device) performs a write operation on the data in the master database (or slave database), a corresponding write operation log is generated and sent to the slave device (or master device). The slave device (or master device) parses the operation log to parse out the historical write operations performed in the master database (or slave database), and then the slave device (or master device) performs all the parsed historical operations in the slave database (or master database), thereby writing the data that was once written to the master database (or slave database) into the slave database (or master database). The purpose of the embodiments of the present application is to provide a method, device, equipment, medium, chip and computer program product for master-slave database data synchronization, which is different from the related art that relies on write operation logs for data synchronization, thereby getting rid of the dependence of master-slave database data synchronization on operation logs.

[0006] A first aspect of an embodiment of the present application provides a method for synchronizing data between a master and a slave database, which is applied to a first device, in which a first database is deployed; the method includes:

[0007] Monitoring the working status of a second device on which a second database is deployed; the first database and the second database operate based on a dual-write mechanism;

[0008] In the case where it is detected that the working state of the second device is abnormal, a preset first-category tag corresponding to the target data generated after the first moment in the first database is added; the first moment is the moment when the working state of the second device is detected to change from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first-category tag includes: an unsynchronized tag and a deleted tag; the newly added data and the modified data both correspond to the unsynchronized tag, and the data to be deleted corresponds to the deleted tag;

[0009] When it is monitored that the working status of the second device has returned to normal, a synchronization request is generated based on the first type of tag and the data identified by the first type of tag, and the synchronization request is sent to the second device; the synchronization request carries a synchronization operation command for synchronizing the data identified by the first type of tag in the second database.

[0010] A second aspect of an embodiment of the present application provides a method for synchronizing data between a master and a slave database, which is applied to a second device in which a second database is deployed; the method includes:

[0011] When receiving a synchronization request sent by a first device on which a first database is deployed, parsing a synchronization operation command from the synchronization request; the second database and the first database operate based on a dual-write mechanism;

[0012] Execute the synchronization operation command to synchronize the data identified by the first type of label in the first database with the second database; the first type of label is a preset first type of label corresponding to the target data generated after the first moment in the first database when the first device monitors that the working state of the second device is abnormal; the first moment is the moment when the first device monitors that the working state of the second device changes from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first type of label includes: an unsynchronized label and a deleted label; the newly added data and the modified data both correspond to the unsynchronized label, and the data to be deleted corresponds to the deleted label.

[0013] A third aspect of an embodiment of the present application provides a device for synchronizing data between a master and a slave database, the device comprising:

[0014] A first monitoring module is used to monitor the working status of a second device on which a second database is deployed; the first database and the second database are operated based on a dual-write mechanism;

[0015] A first adding module is used to add a preset first type of label corresponding to the target data generated after a first moment in the first database when the working state of the second device is detected to be abnormal; the first moment is the moment when the working state of the second device is detected to change from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first type of label includes: an unsynchronized label and a deleted label; the newly added data and the modified data both correspond to the unsynchronized label, and the data to be deleted corresponds to the deleted label;

[0016] The first sending module is used to generate a synchronization request according to the first type of tag and the data identified by the first type of tag, and send the synchronization request to the second device when it is monitored that the working status of the second device returns to normal; the synchronization request carries a synchronization operation command for synchronizing the data identified by the first type of tag in the second database.

[0017] A fourth aspect of an embodiment of the present application provides a device for synchronizing data between a master and a slave database, the device comprising:

[0018] A first parsing module is used to parse a synchronization operation command from the synchronization request when receiving a synchronization request sent by a first device on which a first database is deployed; the second database and the first database operate based on a dual-write mechanism;

[0019] A first execution module is used to execute the synchronization operation command to synchronize the data identified by the first type of label in the first database in the second database; the first type of label is a preset first type of label corresponding to the target data generated after the first moment in the first database, which is added by the first device when the first device monitors that the working state of the second device is abnormal; the first moment is the moment when the first device monitors that the working state of the second device changes from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first type of label includes: unsynchronized labels and deleted labels; the newly added data and the modified data both correspond to the unsynchronized labels, and the data to be deleted corresponds to the deleted labels.

[0020] The fifth aspect of an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the memory storing programs or instructions that can be run on the processor, and the program or instructions, when executed by the processor, implement the steps of the method for master-slave database data synchronization as described in any one of the first to second aspects.

[0021] The sixth aspect of an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method for master-slave database data synchronization as described in any one of the first to second aspects are implemented.

[0022] The seventh aspect of an embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the method for master-slave database data synchronization as described in any one of the first to second aspects.

[0023] An eighth aspect of an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the method for master-slave database data synchronization as described in any one of the first to second aspects.

[0024] In an embodiment of the present application, when the working status of the first device and the second device are both normal, the data written in the first database will be synchronously written into the second database based on the dual-write mechanism; when the working status of the second device is abnormal, the first type of tag will be added to the data written into the first database; when the working status of the second device returns to normal, a synchronization operation command for the data identified by the first type of tag is generated, the synchronization operation command is encapsulated into a synchronization request, and the synchronization request is sent to the second device. After receiving the synchronization request, the second device executes the synchronization operation command in the synchronization request to synchronize the data identified by the first type of tag in the first database to the second database, thereby realizing the writing in the second database of the data written into the first database during the abnormal period of the second device.

[0025] For more specific descriptions on the technical effects, please refer to the specific implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of the structure of a master-slave database provided in the related art and the first schematic diagram of its working principle;

[0027] Figure 2 This is a schematic diagram of the structure of a master-slave database provided in the related art and a second schematic diagram of its working principle;

[0028] Figure 3 It is a third schematic diagram of a structure diagram of a master-slave database provided in the related art and its working principle;

[0029] Figure 4 It is a timing diagram of signaling interaction in a method for synchronizing data between a master and a slave database provided in an embodiment of the present application;

[0030] Figure 5 It is a schematic diagram of the structure of a master-slave database and a schematic diagram of its working principle provided by an embodiment of the present application;

[0031] Figure 6 It is a schematic diagram of the structure of a master-slave database and a schematic diagram of its working principle provided by an embodiment of the present application;

[0032] Figure 7 It is a timing diagram of signaling interaction in a method for synchronizing data between a master and a slave database provided in an embodiment of the present application;

[0033] Figure 8 It is a timing diagram of signaling interaction in a method for synchronizing data between a master and a slave database provided in an embodiment of the present application;

[0034] Fig. 9 It is a flowchart of a method for synchronizing master-slave database data provided by an embodiment of the present application;

[0035] Fig.10 It is a flowchart of a method for synchronizing master-slave database data provided by an embodiment of the present application;

[0036] Fig.11 It is a structural diagram of a device for synchronizing master-slave database data provided by an embodiment of the present application;

[0037] Fig.12 It is a structural diagram of a device for synchronizing master-slave database data provided by an embodiment of the present application;

[0038] Fig.13 The present application also provides a schematic diagram of the structure of an electronic device;

[0039] Fig.14 Another structural diagram of an electronic device is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0041] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0042] In the related art, when a master device (or slave device) performs a write operation on data in a master database (or slave database), a corresponding write operation log is generated and sent to the slave device (or master device). The slave device (or master device) parses the operation log to parse out the historical write operations performed in the master database (or slave database), and then the slave device (or master device) executes all the parsed historical write operations in the slave database (or master database), thereby writing the data that was once written to the master database (or slave database) into the slave database (or master database).

[0043] When data synchronization between master and slave databases is performed based on operation logs, the operation logs may occupy a large amount of storage space on the storage medium, affecting the normal storage of data. At the same time, the slave device (or master device) that performs the synchronization task needs to parse the operation logs first, parse out the historical write operations, and then execute these historical write operations. This process is cumbersome and complicated, and will occupy a large amount of device resources of the slave device (or master device) that performs the synchronization task, resulting in a relatively low synchronization efficiency. The following is a brief description of it.

[0044] like Figures 1 to 3 As shown, Figures 1 to 3 This is a schematic diagram of the structure and working principle of a master-slave database provided in the relevant technology. Figure 1 , a communication connection is established between device 11 and device 12, and device 11 and device 12 monitor each other's working status based on a heartbeat keep-alive mechanism. Databases are deployed in both device 11 and device 12, and the database in device 11 and the database in device 12 are each other's redundant backup databases, that is, the data in the database in device 11 and the data in the database in device 12 are synchronized with each other.

[0045] Please continue to see Figure 1 , device 11 and device 12 work based on a dual-machine hot standby architecture mode. Specifically, device 11 and device 12 provide external data storage services as a database system. When the working status of device 11 and device 12 are normal, one of devices 11 and device 12 acts as a master device to provide external data storage services, and the other device is a slave device. The slave device is also called a redundant device, a substitute device, or a backup device. Optionally, the slave device does not directly provide external data storage services, but the slave database in the slave device and the database in the master device will synchronize data based on a dual-write mechanism (that is, when writing data to one of the databases, the same data will also be written to the other database synchronously).

[0046] Please continue to see Figure 1 For example, when a user initiates a write request, the write request is scheduled to the master device 11. The data write service running on the master device 11 will execute the write operation requested in the write request both in the master database and in the slave database (through communication technology), so as to perform dual writing of the write operation initiated by the user in the master and slave databases, thereby realizing data synchronization between the master and slave databases.

[0047] When the working status of any of the master and slave devices is abnormal, the other normal device will take over the data storage service of the entire database system. At this time, the dual-write mechanism fails and switches to a single-write mechanism, that is, only write data to the database deployed on the device that takes over the data storage service, and generate corresponding operation logs when performing single data writing. After the device with abnormal working status returns to normal, the device that has been working normally will send the operation log to the device that has returned to normal from the abnormal working status. The device that receives the operation log parses the operation log and executes all the parsed historical write operations locally, thereby synchronizing the data written to the database on the device that has been working normally during the device abnormality to the local database.

[0048] See also Figure 2 For example, if the master device 11 fails to detect the "heartbeat" of the slave device 12, it means that the working state of the slave device 12 is abnormal. The master device 11 completely takes over the entire database system and switches from the dual-write mode to the single-write mode. The write request initiated by the user is dispatched to the master device 11. The data write service running on the master device 11 will execute the write operation requested in the write request in the master database and generate the corresponding operation log.

[0049] See also Figure 3 For example, the master device 11 detects the "heartbeat" of the slave device 12 again, indicating that the working status of the slave device 12 has returned to normal. The master device 11 sends the operation log to the slave device 12. The data write service running on the slave device 12 parses the operation log to parse out all historical operations on the master database during the period when the working status of the slave device 12 was abnormal, and executes these historical operations on the slave database, thereby synchronizing the data written into the master database during the abnormal period of the slave device 12 to the local slave database.

[0050] If device 12 fails to return to normal for a long time, a large number of operation logs will be generated on device 11. These operation logs will consume a large amount of physical storage resources and increase the write / read pressure of the physical storage medium, thereby reducing the physical storage resources available to the master database and affecting the performance of the entire database system in providing external storage services. Moreover, in the process of executing synchronization tasks after device 12 returns to normal, it is necessary to parse these operation logs first, parse out all historical write operations performed in the master database during the abnormal period of device 12, and then execute these historical write operations. This process is cumbersome and complicated, and will occupy a large amount of device resources of device 12, and the synchronization efficiency is relatively low.

[0051] It should be noted that in Figures 1 to 3In the scenario shown, the identities of the master database and the slave database are not absolutely fixed, but relatively variable. Usually, the identity of the device currently scheduled to directly receive the user's read and write requests is the master device, for example Figure 1 The device 11 shown in FIG. 1 is the master device, and the database running on the master device is the master database. Figure 1 In the case shown in , device 12 does not detect the "heartbeat" of device 11, indicating that the working status of device 11 is abnormal. Device 12 completely takes over the entire database system, and the user's read and write requests are scheduled to device 12 for processing, and the dual-write mode is switched to the single-write mode. At this time, the identity of device 12 is switched to the master device, the identity of device 11 is switched to the slave device, the database on device 12 is switched to the master database, and the database on device 11 is switched to the slave database.

[0052] It should also be noted that the solutions described in the above-mentioned related technologies should not be directly regarded as the prior art of the method, device, equipment, storage medium, chip and computer program product for master-slave database data synchronization provided by the embodiments of the present application. Unless there is conclusive evidence that can clearly prove that the above-mentioned related technologies belong to the prior art, the solutions described in the above-mentioned related technologies can be regarded as the prior art of the method, device, equipment, storage medium, chip and computer program product for master-slave database data synchronization provided by the embodiments of the present application.

[0053] The method, device, equipment, storage medium, chip and computer program product for master-slave database data synchronization provided by the embodiment of the present application, when an abnormality occurs in any of the master-slave databases operating in dual-write mode, a corresponding label is added to the newly written data in the database that can still be written normally, to indicate that the newly written data has not yet been synchronized between the master and slave databases, and after the abnormal database returns to normal, the labeled data in the normal database is synchronized to the database that has returned to normal from the abnormality. This effectively gets rid of the dependence of master-slave database data synchronization on operation logs, and to a certain extent solves the technical problem that data synchronization of master-slave databases based on operation logs will occupy too much physical storage resources of the device and the synchronization efficiency is relatively low.

[0054] In conjunction with the accompanying drawings, a method, apparatus, device, storage medium, chip and computer program product for synchronizing master-slave database data provided by an embodiment of the present application are described below.

[0055] The present application embodiment provides a method for synchronizing master-slave database data. Figure 4 As shown, Figure 4 This is a timing diagram of signaling interaction in a method for synchronizing master-slave database data provided by an embodiment of the present application. Figure 1The master-slave database and dual-machine hot standby architecture scenario shown in the figure illustrates the master-slave database data synchronization method provided by the embodiment of the present application. Figure 4 The master-slave database data synchronization method comprises the following steps:

[0056] S11, a first device deployed with a first database monitors a working status of a second device deployed with a second database; the first database and the second database operate based on a dual-write mechanism.

[0057] The first device may be any device deployed with a database in a database system based on a master-slave database or a dual-machine hot standby architecture.

[0058] For example, the first device may be Figure 1 The device 11 shown in FIG. 1 may also be Figure 1 Accordingly, if the first device is Figure 1 The device 11 shown in FIG. 1 , the second device is Figure 1 ; conversely, if the first device is Figure 1 , the second device is Figure 1 It should be noted that in the following exemplary description, the first device is Figure 1 The device 11 shown in the figure is used as an example for explanation, which obviously should not be understood as a specific limitation of the present application. Based on the description in the specification, technical personnel in this field can understand the implementation method when the first device is device 12. Therefore, in order to avoid repeated descriptions, the first device is not described as device 12 as an example in the specification of this application.

[0059] In some optional implementations, the first device and the second device may monitor each other's working status based on a heartbeat keep-alive mechanism.

[0060] Exemplarily, the first device and the second device both periodically send heartbeat data packets to each other; if the first device (or the second device) can always periodically receive the heartbeat data packets of the second device (or the first device) within a preset heartbeat period, it is determined that the working status of the second device (or the first device) is normal; if the first device (or the second device) fails to receive the heartbeat data packet of the second device (or the first device) after exceeding the preset heartbeat period, it is determined that the working status of the second device (or the first device) is abnormal.

[0061] The first database is a database deployed on the first device, and the second database is a database deployed on the second device. The first database and the second database operate based on a dual-write mechanism. That is, the first database and the second database are master-slave databases to each other. When the working status of the first device and the second device are both normal, when a write operation is performed on either of the first database and the second database, the same write operation will be performed in the other database accordingly based on the communication technology to achieve data synchronization between the master and slave databases.

[0062] For example, when the working status of the first device and the second device are both normal, when data A is deleted from the first database, the data A previously written synchronously will also be deleted from the second database; when data B is added to the second database, data B will also be added to the first database; when data C that has been synchronized in the master-slave database is modified to data D in the first database, the data C that has been synchronized in the master-slave database will also be modified to data D in the second database.

[0063] S12, when the first device detects that the working state of the second device is abnormal, the first device adds a preset first type label corresponding to the target data generated after the first moment in the first database.

[0064] As mentioned above, if the first device periodically receives the heartbeat data packet of the second device in a preset heartbeat period, it is determined that the working state of the second device is a normal state. If the first device fails to receive the heartbeat data packet of the second device after exceeding the preset heartbeat period, it is determined that the working state of the second device is an abnormal state. The first moment is the moment when the working state of the second device is monitored to change from normal to abnormal.

[0065] Exemplarily, the preset heartbeat cycle is , the first device in the heartbeat cycle The heartbeat data packet of the second device is received at any time, but after a certain period of time The subsequent heartbeat cycle If the heartbeat packet of the second device is not received at the time, it is determined that the second device is After the time, its working state becomes abnormal. That is, the first device "infers" that the working state of the second device is at time Therefore, the moment That is the first moment.

[0066] The target data is at time The data generated by subsequently performing a write operation on the data in the first database.

[0067] The write operation may include at least one of the following operations: a adding operation, a modifying operation, and a deleting operation.

[0068] The so-called adding operation refers to storing new data in the database, the so-called modifying operation refers to modifying the value of the data already stored in the database, and the deleting operation refers to deleting the data previously stored in the database from the database. Accordingly, the adding operation will generate new data in the database, the modifying operation will generate modified data in the database, and the deleting operation will generate data to be deleted that is requested to be deleted but is aborted in the database (it should be noted that the data to be deleted is the data that the user requests to delete but has not actually been deleted in the database, please refer to the relevant instructions below for details). Therefore, the target data includes at least one of the following: new data, modified data, and data to be deleted that is requested to be deleted but is aborted.

[0069] Among them, the first category of tags includes: unsynchronized tags and deleted tags; the newly added data and the modified data both correspond to the unsynchronized tags, and the data to be deleted corresponds to the deleted tags. Of course, optionally, the unsynchronized tags can also include: newly added tags and modified tags, the newly added data corresponds to the newly added tags, and the modified data corresponds to the modified tags. For example, the character "1" is used as the unsynchronized tag, the character "11" is used as the newly added tag, the character "12" is used as the modified tag, and the character "2" is used as the deleted tag.

[0070] In an embodiment of the present application, when the first device monitors that the working state of the second device is in an abnormal state, if the write operation command in the write request initiated by the user received by the first device is a new operation or a modification operation, the data write service running on the first device will directly execute the requested new operation or modification operation in the first database, and accordingly, new data or modified data will be generated in the first database; and an unsynchronized label will be added to the generated new data and modified data.

[0071] However, it should be noted that in the embodiment of the present application, when the first device monitors that the working state of the second device is in an abnormal state, if the write operation command in the write request initiated by the user received by the first device is a deletion operation, the data write service running on the first device will terminate the deletion operation. The so-called termination means suspending execution, suspending the deletion operation task and putting it in a waiting state (after the working state of the second device returns to normal, the data to be deleted by the deletion operation is synchronously deleted in the first database and the second database. For details, please refer to the relevant instructions below), generating data to be deleted that is requested to be deleted but is terminated; and adding a deletion tag to the generated data to be deleted.

[0072] like Figure 5 As shown, Figure 5This is a schematic diagram of the structure of a master-slave database and its working principle provided by the implementation method of this application. Figure 5 , for example, device 11 at time It is detected that the working state of device 12 has become abnormal. Device 11 is at time When receiving a write request from the user, the data write service running on the device 11 determines the type of write operation requested in the write request. If it is determined that the type of the write operation is a new operation or a modification operation, the data write service executes the new operation or the modification operation in the write request in the main database, and adds a new tag to the generated new data or modified data in the database; if it is determined that the type of the write operation is a deletion operation, the data write service terminates the deletion operation and adds a deletion tag in the main database for the to-be-deleted data to be deleted by the terminated deletion operation.

[0073] S13, when the first device monitors that the working status of the second device has returned to normal, the first device generates a synchronization request based on the first type of tag and the data identified by the first type of tag, and sends the synchronization request to the second device; the synchronization request carries a synchronization operation command for synchronizing the data identified by the first type of tag in the second database.

[0074] After the second device recovers from the abnormality, it will resend the heartbeat data packet to the first device based on the heartbeat cycle to notify the first device that it has recovered. Correspondingly, the first device also starts to send heartbeat data packets to the second device so that the first device and the second device can monitor each other's working status.

[0075] The first device at time The moment after When the first device receives the heartbeat data packet sent by the second device again, it is determined that the working state of the monitored second device has returned to normal. When the first device determines that the second device has returned to normal, it finds out the period (i.e., the time) during which the second device was abnormal. To time The time period between ), and based on the first type of tags corresponding to each target data, generate corresponding synchronization operation commands. Add the synchronization operation commands corresponding to each target data to the synchronization request, and the first device sends each synchronization request to the second device.

[0076] The synchronization operation command is a command corresponding to the write operation of the target data (in the first database) to be synchronized in the second database. Therefore, the synchronization operation command includes: a new operation command, a modification operation command and a deletion operation command. Among them, the new operation command corresponds to the new data in the first database, the modification operation command corresponds to the modified data in the first database, and the deletion operation command corresponds to the data to be deleted in the first database.

[0077] The synchronization operation command carries at least one of the following three attributes for identifying the data to be synchronized by the synchronization operation command: a data identity, a physical storage address in the second database, and a logical address in the second database, so that the second device can locate the data to be operated by the synchronization operation command in the second database according to at least one of the three attributes when executing the synchronization operation command.

[0078] Optionally, the synchronization operation command is generated based on the characteristics of the second database, that is, the generated synchronization operation command should be executable in the second database. For example, when the second database is a relational database, the synchronization operation command is generated based on SQL (Structured Query Language).

[0079] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of a master-slave database and its working principle provided by the implementation method of this application. Figure 6 , for example, in the time period In the first database on device 11, newly added data E, modified data F obtained by modifying data f, and data G to be deleted are generated. Data E is identified by label "1" (or "11"), data F is identified by label "1" (or "12"), and data G is identified by label "2". When device 11 detects that the working status of device 12 has returned to normal, it generates a new operation command for data E with label "1" (or "11") and adds the new operation command to a synchronization request, generates a modification operation command for data F with label "1" (or "12") and adds the modification operation command to a synchronization request, and generates a deletion operation command for data G with label "2" and adds the deletion operation command to a synchronization request. Device 11 sends the three synchronization requests generated for data E-data G to device 12.

[0080] In some optional implementations, the first device sends each synchronization request generated for each target data to the second device in series. Serial transmission can reduce the communication pressure between the first device and the second device to a certain extent, so as to ensure the performance of the entire database system in providing storage services to the outside; however, it will obviously lead to a slower synchronization rate and lower efficiency of the master-slave database data.

[0081] In some other optional implementations, the first device sends each synchronization request generated for each target data to the second device in parallel. Parallel transmission may increase the communication pressure between the first device and the second device to a certain extent, resulting in a decrease in the performance of the entire database system in providing storage services to the outside; however, it can effectively improve the master-slave database data synchronization rate and efficiency.

[0082] S14: When the second device receives the synchronization request sent by the first device, it parses the synchronization request to obtain a synchronization operation command.

[0083] When receiving each synchronization request sent by the first device, the second device parses each synchronization request and extracts the synchronization operation command carried by each synchronization request from each synchronization request.

[0084] Please continue to see Figure 6 For example, when device 12 receives three synchronization requests generated for data E-data G sent by device 11, the data write service running on device 12 parses the three synchronization requests separately, and parses out a new operation command for adding data E in the second database from the synchronization request for data E, parses out a modification operation command for modifying data f in the second database to modified data F from the synchronization request for data F, and parses out a deletion operation command for deleting data G from the second database from the synchronization request for data G.

[0085] S15: The second device executes the synchronization operation command to synchronize the data identified by the first type of tags in the first database in the second database.

[0086] The data in the first database identified by the first type of tag is the period during which the second device is abnormal (time period ) data that has not been synchronized to the second database.

[0087] Exemplarily, the data write service running on the second device executes the synchronization operation command parsed from each received synchronization request to synchronize the period (time period) during which the second device is abnormal in the second database. ) data that has not been synchronized to the second database.

[0088] Please continue to see Figure 6 For example, device 12 executes a new operation command to add data E in the second database, thereby adding data E in the second database; device 12 executes a modify operation command to modify data f in the second database to modified data F, thereby modifying data f in the second database to modified data F; device 12 executes a delete operation command to delete data G in the second database, thereby deleting data G in the second database.

[0089] In the embodiment of the present application, when the working status of the first device and the second device are both normal, the data written into the first database will be synchronously written into the second database based on the dual-write mechanism; when the working status of the second device is abnormal, the first type of tag will be added to the data written into the first database; when the working status of the second device returns to normal, a synchronization operation command for the data identified by the first type of tag is generated, the synchronization operation command is encapsulated into a synchronization request, and the synchronization request is sent to the second device. After receiving the synchronization request, the second device executes the synchronization operation command in the synchronization request to synchronize the data identified by the first type of tag in the first database to the second database, thereby realizing writing in the second database the data written into the first database during the abnormality of the second device. Not only does it get rid of the dependence of the master-slave database data synchronization on the database operation log, because it is no longer necessary to generate the operation log, the first device no longer needs to store the operation log in the physical storage medium during the failure of the second device, and can free up more physical storage resources for storing business data, and reduce the write / read pressure of the physical storage medium, to ensure the data storage service performance of the entire database system.

[0090] The present application also provides a method for synchronizing master-slave database data. Figure 7 As shown, Figure 7 This is a timing diagram of signaling interaction in a method for synchronizing master-slave database data provided by an embodiment of the present application. Figure 1 The master-slave database and dual-machine hot standby architecture scenario shown in the figure illustrates the master-slave database data synchronization method provided by the embodiment of the present application. Figure 7 The master-slave database data synchronization method comprises the following steps:

[0091] S11, a first device deployed with a first database monitors a working status of a second device deployed with a second database; the first database and the second database operate based on a dual-write mechanism.

[0092] S12, when the first device detects that the working state of the second device is abnormal, the first device adds a preset first type label corresponding to the target data generated after the first moment in the first database.

[0093] S13, when the first device monitors that the working status of the second device has returned to normal, the first device generates a synchronization request based on the first type of tag and the data identified by the first type of tag, and sends the synchronization request to the second device; the synchronization request carries a synchronization operation command for synchronizing the data identified by the first type of tag in the second database.

[0094] S14: When the second device receives the synchronization request sent by the first device, it parses the synchronization request to obtain a synchronization operation command.

[0095] S15: The second device executes the synchronization operation command to synchronize the data identified by the first type of tags in the first database in the second database.

[0096] Steps S11-S15 have been Figure 4 This will be explained in the relevant implementation method, please refer to the relevant instructions, and will not be repeated here.

[0097] If the synchronization operation command is executed successfully, that is, if the second device executes step S15 successfully, the second device executes subsequent steps S16-S19. If the synchronization operation command is executed unsuccessfully, that is, if the second device fails to execute step S15, the second device executes subsequent steps S20-S21.

[0098] S16: When the synchronization operation command is executed successfully, the second device sends a first reply message to the first device indicating that the synchronization operation command is executed successfully.

[0099] Successful execution of the synchronization operation command by the second device means that the second device successfully executes the write operation in the synchronization operation command in the second database, that is, the second device completes synchronization of the data to be synchronized to the second database by the synchronization operation command in the second database.

[0100] There is a one-to-one correspondence between the first reply message and the synchronization request.

[0101] Please continue to see Figure 6For example, if device 12 successfully executes a new operation command for adding data E to the slave database, it sends a first reply message to device 11 to indicate that data E is successfully added to the slave database, so as to notify device 11 that device 12 successfully executes the synchronization operation command in the synchronization request generated for the newly added data E, that is, the first reply message corresponds to the synchronization request generated for the newly added data E. For another example, if device 12 successfully executes a modification operation command for modifying data f into modified data F in the slave database, it sends a first reply message to device 11 to indicate that data f is successfully modified into modified data F in the slave database, so as to notify device 11 that device 12 successfully executes the synchronization operation command in the synchronization request generated for modifying data F, that is, the first reply message corresponds to the synchronization request generated for modifying data F. For another example, if device 12 successfully executes the deletion operation command for deleting the data G to be deleted from the database, it sends a first reply message to device 11 indicating that the data G is successfully deleted from the database, so as to notify device 11 that device 12 successfully executes the synchronization operation command in the synchronization request generated for the data G to be deleted, that is, the first reply message corresponds to the synchronization request generated for the data G to be deleted.

[0102] Optionally, the synchronization request also carries a data identity identifier for identifying the identity of the data to be operated by the synchronization operation command in the synchronization request, and accordingly, the first reply message also contains the data identity identifier in the synchronization request corresponding to the first reply message. Based on the data identity identifier, the second device can easily indicate to the first device which synchronization operation command in the synchronization request is successfully executed.

[0103] For example, when the first device sends a synchronization request generated for newly added data E to the second device, the data identity ID-e of the data E is added to the synchronization request. When the second device successfully executes the new operation command in the synchronization request and sends a first reply message to the first device, the data identity ID-e is added to the first reply message to indicate that the second device successfully executes the new operation command in the synchronization request generated for the newly added data E.

[0104] S17: When the first device receives the first reply message sent by the second device indicating that the second device successfully executes the synchronization operation command, the first device determines the type of the first category tag of the data to be operated by the synchronization operation command.

[0105] Based on the above, it can be known that the first category of tags includes: unsynchronized tags (unsynchronized tags can also include: new tags and modified tags) and deleted tags. In the first database, for the data identified by the unsynchronized tag, after the synchronization is completed, the unsynchronized tag needs to be modified to a synchronized tag or the unsynchronized tag needs to be deleted, and for the data identified by the deleted tag, after the synchronization is completed, the data identified by the deleted tag and its corresponding deleted tag need to be deleted.

[0106] Therefore, when the first device receives the first reply message sent by the second device in response to a synchronization request, the first device determines which synchronization request the first reply message is replying to, and then the first device needs to determine the type of the first type of tag of the data to be operated by the synchronization operation command in the synchronization request replied by the first reply message, that is, determine whether the first type of tag of the data to be operated by the synchronization operation command in the synchronization request replied by the first reply message is an unsynchronized tag or a deleted tag. If it is determined to be an unsynchronized tag, jump to step S18. If it is determined to be a deleted tag, jump to step S19.

[0107] S18, when the first device determines that the type is an unsynchronized tag, the first device deletes the unsynchronized tag of the data to be operated by the synchronization operation command from the first database or modifies the unsynchronized tag of the data to be operated by the synchronization operation command to a synchronized tag.

[0108] The synchronized tag is used to identify data that has completed data synchronization between the master and slave databases.

[0109] For example, the character "0" is used as the synchronized tag.

[0110] For example, after receiving the first reply message sent by the second device, the first device parses the first reply message and finds out that the data identity carried in the first reply message is the data identity ID-e of data E (or the data identity ID-f of data F), then it is determined that the first reply message is replying to the synchronization request generated for the newly added data E (or modified data F); then the first device determines that the type of the first category tag of the data E (or data F) to be operated by the synchronization operation command carried in the synchronization request replied to by the first reply message is an unsynchronized tag. The first device receives the first reply message, indicating that the synchronization of the newly added data E (or modified data F) has been completed in the second database. Therefore, the unsynchronized tag "1" of the data E (or data F) in the first database is changed to the synchronized tag "0".

[0111] S19: When the first device determines that the type is a deletion tag, the first device deletes the data to be operated by the synchronization operation command and its deletion tag from the first database.

[0112] For example, after receiving the first reply message sent by the second device, the first device parses the first reply message and finds out that the data identity carried in the first reply message is the data identity ID-g of the data G. Then, it is determined that the first reply message is replying to the synchronization request generated for the data G to be deleted that was requested to be deleted but was aborted; then the first device determines that the type of the first type of tag of the data G to be operated by the synchronization operation command carried in the synchronization request replied to by the first reply message is a deletion tag. The first device receives the first reply message, indicating that the data G has been deleted in the second database. Therefore, the first device resumes the aborted deletion operation to delete the data G in the first database, and deletes the deletion tag "2" of the data G at the same time.

[0113] S20: Send a second reply message to the first device to indicate failure in executing the synchronization operation command.

[0114] The failure of the second device to execute the synchronization operation command means that the second device fails to successfully execute the write operation in the synchronization operation command in the second database, that is, the second device fails to complete the synchronization of the data to be synchronized to the second database by the synchronization operation command in the second database.

[0115] There is a one-to-one correspondence between the second reply message and the synchronization request.

[0116] Please continue to see Figure 6 For example, if device 12 fails to execute a new operation command for adding data E to the slave database, a second reply message indicating that data E has not been successfully added to the slave database is sent to device 11 to notify device 11 that device 12 has failed to execute the synchronization operation command in the synchronization request generated for the newly added data E, that is, the second reply message corresponds to the synchronization request generated for the newly added data E. For another example, if device 12 fails to execute a modification operation command for modifying data f to modified data F in the slave database, a second reply message indicating that data f has not been successfully modified to modified data F in the slave database is sent to device 11 to notify device 11 that device 12 has failed to execute the synchronization operation command in the synchronization request generated for modifying data F, that is, the second reply message corresponds to the synchronization request generated for modifying data F. For another example, if device 12 fails to execute the deletion operation command for deleting the data G to be deleted from the database, it sends a second reply message to device 11 indicating that the data G has not been successfully deleted from the database, to notify device 11 that device 12 has failed to execute the synchronization operation command in the synchronization request generated for the data G to be deleted, that is, the second reply message corresponds to the synchronization request generated for the data G to be deleted.

[0117] Optionally, the synchronization request also carries a data identity identifier for identifying the identity of the data to be operated by the synchronization operation command in the synchronization request, and accordingly, the second reply message also contains the data identity identifier in the synchronization request corresponding to the second reply message. Based on the data identity identifier, the second device can easily indicate to the first device which synchronization operation command in the synchronization request failed to execute.

[0118] For example, when the first device sends a synchronization request generated for newly added data E to the second device, the identity identifier ID-e of the data E is added to the synchronization request. When the second device fails to execute the new operation command in the synchronization request and sends a second reply message to the first device, the identity identifier ID-e is added to the second reply message to indicate that the second device fails to execute the new operation command in the synchronization request generated for the newly added data E.

[0119] S21, when the first device receives the second reply message sent by the second device indicating that the second device fails to execute the synchronization operation command or does not receive the first reply message sent by the second device within a preset first time period, the first device re-sends the synchronization request to the second device; the starting time point of the first time period is the moment when the synchronization request is issued.

[0120] In some optional implementations, a waiting time for the second device to reply to the synchronization request is preset on the first device. Exemplarily, the waiting time is preset as a first time, and it is obvious that the starting time point of the waiting time should be calculated from the time when the synchronization request is issued, that is, the starting time point of the first time is the time when the synchronization request is issued.

[0121] If the first device receives a first reply message sent by the second device to a synchronization request within the first time period, it means that the second device successfully executes the synchronization operation command in the synchronization request.

[0122] If the first device does not receive the first reply message sent by the second device for a synchronization request within the first time period, or receives the second reply message sent by the second device for a synchronization request within the first time period, it means that the second device has failed to execute the synchronization operation command in the synchronization request. In this case, the first device resends the synchronization request in which the synchronization operation command failed to be executed to the second device, so that the second device repeats the previously failed synchronization operation command and continues to monitor the execution status of the synchronization operation command in the resent (redundant) synchronization request.

[0123] If the second device successfully executes the synchronization operation command in the (redundant) synchronization request, it will send a first reply message to the first device within a first time period.

[0124] If the first device does not receive the first reply message sent by the second device for the (redundant) synchronization request within the first time period or receives the second reply message sent by the second device for the (redundant) synchronization request within the first time period, the first device again sends the second (redundant) synchronization request to the second device and continues to monitor the execution status of the synchronization operation command in the resent second (redundant) synchronization request.

[0125] The (redundant) synchronization requests are sent cyclically in this way until the first device receives a first reply message indicating that the second device has successfully executed the synchronization operation commands in these (redundant) synchronization requests.

[0126] Optionally, if the number of (redundant) synchronization requests sent by the first device reaches a preset number, and the first reply message indicating that the second device successfully executes the synchronization operation commands in these (redundant) synchronization requests is still not received, an alarm prompt indicating that the master-slave database data synchronization has failed is output, or the data of the first database is directly copied in full to the second database. Exemplarily, the preset number is five.

[0127] For example, the first preset duration is The first device at time The first device sends a synchronization request generated for the newly added data E to the second device at time t, then the first device Start timing. If at time The first reply message for the synchronization request is received when , it is determined that the synchronization operation command in the synchronization request generated for the newly added data E is executed successfully.

[0128] If at the time A second reply message for the synchronization request is received, and , it is determined that the synchronization operation command in the synchronization request generated for the newly added data E has failed to execute; or if at time The first reply message for the synchronization request has not been received yet, and , it is determined that the execution of the synchronization operation command in the synchronization request generated for the newly added data E has failed. In this case, the first device resends the synchronization request generated for the newly added data E to the second device, that is, sends the (redundant) synchronization request generated for the newly added data E to the second device, and continues to monitor the execution status of the synchronization operation command in the resent (redundant) synchronization request.

[0129] In an embodiment of the present application, after the data in the master-slave database is synchronized, corresponding operations are performed on the target data generated in the first database during the abnormal period of the second device, that is, the label of the data identified by the unsynchronized label in the first database is modified to a synchronized label or the unsynchronized label is deleted, and the data identified by the deleted label in the first database and its label are deleted, so as to maintain consistency between the data in the first database and the data in the second database and timely update the synchronization status of the data in the first database.

[0130] Of course, in some optional implementations, when both the first device and the second device are normal, the synchronized label "0" is used to identify the data that has been double-written in the first database and the second database, so as to assist in confirming which data has been synchronized between the master and slave databases and which data has not yet been synchronized between the master and slave databases.

[0131] In some optional implementations, in order to improve the synchronization efficiency of data between the master and slave databases, the first device does not generate a synchronization request based on each target data, but generates a synchronization request for at least one target data, and the synchronization request carries a synchronization operation command for at least one target data, and then sends the synchronization request to the second device. After receiving the synchronization request, the second device batch executes all synchronization operation commands carried in the synchronization request, thereby improving the synchronization efficiency of the master and slave database data to a certain extent. The following is an explanation:

[0132] The present application also provides a method for synchronizing master-slave database data. Figure 8 As shown, Figure 8 This is a timing diagram of signaling interaction in a method for synchronizing master-slave database data provided by an embodiment of the present application. Figure 1 The master-slave database and dual-machine hot standby architecture scenario shown in the figure illustrates the master-slave database data synchronization method provided by the embodiment of the present application. Figure 8 The master-slave database data synchronization method comprises the following steps:

[0133] S11, a first device deployed with a first database monitors a working status of a second device deployed with a second database; the first database and the second database operate based on a dual-write mechanism.

[0134] S12, when the first device detects that the working state of the second device is abnormal, the first device adds a preset first type label corresponding to the target data generated after the first moment in the first database.

[0135] Steps S11-S12 have been Figure 4 This will be explained in the relevant implementation method, please refer to the relevant instructions, and will not be repeated here.

[0136] S131, when the first device monitors that the working state of the second device returns to normal, the first device divides the data identified by the first type of tags according to a preset division standard to obtain at least one data set.

[0137] The classification standard is a criterion or rule for classifying the data identified by the first type of tags, and the classification standard can be flexibly configured according to actual needs.

[0138] Exemplarily and optionally, the division standard may be preset based on the characteristics of the database. For example, if the first database is a relational database, the data table may be used as a basic unit, and the data identified by the first type of label in each data table may be divided into a data set.

[0139] Exemplarily and optionally, the division standard may be preset based on the number of data in each data set. For example, the number of data in each data set may be preset to divide the data identified by the first type of label into several data sets with equal numbers of data in the sets.

[0140] Exemplarily and optionally, the division standard can be preset based on each data set corresponding to a label. For example, the data identified by the first type of label can be divided into at least one data set, wherein the labels of the data in some data sets are all unsynchronized labels, and the labels of the data in some data sets are all deleted labels.

[0141] Exemplarily and optionally, the division standard may be preset based on the characteristics of the data identified by the first type of labels. For example, cluster analysis may be performed on the data identified by the first type of labels, and data of the same type in the data identified by the first type of labels may be divided into the same data set.

[0142] After monitoring that the second device has recovered from the abnormality, the first device divides the target data (ie, the data identified by the first type of label) into at least one data set according to a preset division standard.

[0143] Please continue to see Figure 6 For example, after monitoring that device 12 has returned to normal, device 11 divides the newly added data E, modified data F, and to-be-deleted data G into one data set.

[0144] For another example, after monitoring that device 12 has returned to normal, device 11 divides the newly added data E and the modified data F into the first data set, and divides the to-be-deleted data G into the second data set.

[0145] S132, the first device performs the following steps in parallel for each data set in the at least one data set: generating a synchronization operation command corresponding to each data in the target data set according to each data in the target data set and the first type of label corresponding to each data set, encapsulating all the obtained synchronization operation commands into a target data set synchronization request, and sending the target data set synchronization request to the second device; the target data set is any data set in the at least one data set.

[0146] The relevant implementation method of the first device generating the synchronous operation command corresponding to each data is the same as the implementation method of the first device generating the synchronous operation command in step S13 in the previous text. Please refer to the relevant instructions in the previous text for details, and no further details will be given here.

[0147] In this embodiment, only one synchronization request is generated for each data set, but the synchronization request corresponding to each data set carries the synchronization operation commands corresponding to all the data in the corresponding data set. The first device generates synchronization requests for each data set in parallel. Each time a data set synchronization request is generated, the data set synchronization request can be sent to the second device for data synchronization, without having to wait until the data set synchronization requests corresponding to all data sets are generated before sending them to the second device, thereby improving the efficiency of data synchronization between the master and slave databases.

[0148] For example, the device 11 generates a first data set synchronization request and a second data set synchronization request in parallel for the aforementioned first data set and second data set. That is, the device 11 generates a first synchronization operation command for adding data E in the slave database for the newly added data E in the first data set, generates a second synchronization operation command for modifying data f in the database to data F for the modified data F, and adds the first synchronization operation command and the second synchronization operation command to the first data set synchronization request; and the device 11 generates a third synchronization operation command for deleting data G from the slave database for the data to be deleted G in the second data set, and adds the third synchronization operation command to the second data set synchronization request. Regardless of which synchronization request is generated, the generated synchronization request is immediately sent to the device 12 for execution, so as to improve the efficiency of data synchronization between the master and slave databases.

[0149] S141: When receiving a target data set synchronization request sent by a first device deployed with a first database, the second device parses all synchronization operation commands from the target data set synchronization request.

[0150] When the second device receives any data set synchronization request (for example, a target data set synchronization request), it immediately parses the data set synchronization request and parses out all synchronization operation commands carried in the synchronization request. The more specific implementation method of parsing out synchronization operation commands from data set synchronization requests is similar to the method of parsing out synchronization operation commands from synchronization requests in step S14 above, except that: in step S14, only one synchronization operation command can be parsed out from a synchronization request, while in step S141, one or at least two synchronization operation commands may be parsed out from a data set synchronization request. Please refer to the description of step S14 above for details, which will not be repeated here.

[0151] For example, when device 12 receives the first data set synchronization request sent by device 11, it parses the first data set synchronization request to obtain the first synchronization operation command and the second synchronization operation command. When device 12 receives the second data set synchronization request sent by device 11, it parses the third synchronization operation command from the second data set synchronization request. Device 12 can parse the first data set synchronization request and the second data set synchronization request in parallel.

[0152] S151: The second device executes in parallel all synchronization operation commands parsed from the target data set synchronization request to synchronize the target data set in the second database.

[0153] For the relevant instructions on executing the synchronization operation command in the second database, please refer to the description of step S15 above, which will not be repeated here.

[0154] The second device executes all synchronization operation commands parsed from the data set synchronization request (eg, target data set synchronization request) in parallel to improve the master-slave database synchronization efficiency.

[0155] For example, the device 12 executes the first synchronization operation command and the second synchronization operation command in parallel to parse the first data set synchronization request to synchronize the newly added data E and the modified data F to the second database, that is, to add data E to the slave database and modify data f to F. The device 12 can also execute the third synchronization operation command in parallel to delete data G from the slave database.

[0156] When all synchronization operation commands parsed from the target data set synchronization request by the second device are successfully executed, that is, when the second device successfully executes step S151, the subsequent steps S161-S191 are executed. When at least one synchronization operation command among all synchronization operation commands parsed from the target data set synchronization request by the second device fails to execute, that is, when the second device fails to execute step S151, the subsequent steps S201-S211 are executed.

[0157] S161: When all synchronization operation commands parsed from the target dataset synchronization request are successfully executed, the second device sends to the first device a third reply message indicating that all synchronization operation commands in the target dataset synchronization request are successfully executed.

[0158] The second device successfully executes all synchronization operation commands parsed from the target data set synchronization request means that the second device successfully executes the write operations in all synchronization operation commands parsed from the target data set synchronization request in the second database, that is, the second device completes synchronization in the second database of the data to be synchronized to the second database by all synchronization operation commands parsed from the target data set synchronization request.

[0159] There is a one-to-one correspondence between the third reply message and the dataset synchronization request.

[0160] For example, if device 12 successfully executes the first synchronization operation command and the second synchronization operation command in the first data set synchronization request, then a third reply message is sent to device 11 to indicate that data E is successfully added from the database and data f is successfully modified to data F, so as to notify device 11 that device 12 successfully executes the first synchronization operation command and the second synchronization operation command in the first data set synchronization request, that is, the third reply message corresponds to the first data set synchronization request. For another example, if device 12 successfully executes the third synchronization operation command in the second data set synchronization request, then a third reply message is sent to device 11 to indicate that data G is successfully deleted from the database, so as to notify device 11 that device 12 successfully executes the third synchronization operation command in the second data set synchronization request, that is, the third reply message corresponds to the second data set synchronization request.

[0161] Optionally, the dataset synchronization request also carries a request identity for identifying the identity of the dataset synchronization request, and accordingly, the third reply message also contains the request identity in the dataset synchronization request corresponding to the third reply message. Based on the request identity, the second device can conveniently indicate to the first device which specific dataset synchronization request synchronization operation commands are successfully executed.

[0162] For example, when the first device sends a first data set synchronization request to the second device, the request identity ID-1 is added to the synchronization request. When the second device successfully executes the first synchronization operation command and the second synchronization operation command in the synchronization request, when the third reply message is sent to the first device, the data identity ID-1 is added to the third reply message to indicate that the second device successfully executes the first synchronization operation command and the second synchronization operation command. For another example, when the first device sends a second data set synchronization request to the second device, the request identity ID-2 is added to the synchronization request. When the second device successfully executes the third synchronization operation command in the synchronization request, when the third reply message is sent to the first device, the data identity ID-2 is added to the third reply message to indicate that the second device successfully executes the third synchronization operation command.

[0163] S171: When the first device receives the third reply message sent by the second device in response to the target data set synchronization request, the first device determines the type of the first category label of the data to be operated by each synchronization operation command in the target data set synchronization request.

[0164] When the first device receives the third reply message sent by the second device in response to a data set synchronization request, the first device determines which data set synchronization request the third reply message is replying to, and then the first device needs to determine the type of the first type of label of the data to be operated by each synchronization operation command in the data set synchronization request replied by the third reply message, that is, determine whether the first type of label of the data to be operated by each synchronization operation command in the synchronization request replied by the third reply message is an unsynchronized label or a deleted label. If it is determined that the label of the data to be operated by a synchronization operation command is an unsynchronized label, jump to step S181. If it is determined that the label of the data to be operated by a synchronization operation command is a deleted label, jump to step S191.

[0165] S181, when the first device determines that the type of the first type of label of the data to be operated by a synchronization operation command is an unsynchronized label, the first device deletes the unsynchronized label of the data to be operated by the synchronization operation command from the first database or modifies the unsynchronized label of the data to be operated by the synchronization operation command to a synchronized label.

[0166] For example, after receiving the third reply message sent by the second device, the first device parses the third reply message and finds out that the request identity carried in the third reply message is ID-1, then determines that the third reply message is replying to the first data set synchronization request; then the first device determines that the type of the first category label of the data E to be operated by the first synchronization operation command carried in the first data set synchronization request replied to by the third reply message is an unsynchronized label and the type of the first category label of the data F to be operated by the second synchronization operation command is also an unsynchronized label, then the unsynchronized label "1" of the data E and data F in the first database is modified to the synchronized label "0", or the unsynchronized label "1" of the data E and F is deleted.

[0167] S191: When the first device determines that the type of the first type of tag of the data to be operated by a synchronization operation command is a deletion tag, the first device deletes the data to be operated by the synchronization operation command and its deletion tag from the first database.

[0168] For example, after receiving the third reply message sent by the second device, the first device parses the third reply message and obtains the request identity ID-2 carried in the third reply message, then determines that the third reply message is replying to the second data set synchronization request; then the first device determines that the type of the first type of tag of the data G to be operated by the third synchronization operation command carried in the second synchronization request replied to by the third reply message is a deletion tag, and the first device resumes the aborted deletion operation to delete the data G in the first database, and at the same time deletes the deletion tag "2" of the data G.

[0169] S201: When at least one synchronization operation command parsed from the target dataset synchronization request fails to execute, the second device sends to the first device a fourth reply message indicating that at least one synchronization operation command in the target dataset synchronization request fails to execute.

[0170] The failure of the second device to execute at least one synchronization operation command parsed from the target data set synchronization request means that the second device fails to execute a write operation on the second database of at least one synchronization operation command among all the synchronization operation commands parsed from the target data set synchronization request, that is, the data to be synchronized to the second database by at least one synchronization operation command among all the synchronization operation commands parsed from the target data set synchronization request by the second device cannot be synchronized in the second database.

[0171] There is a one-to-one correspondence between the fourth reply message and the dataset synchronization request.

[0172] For example, if device 12 fails to execute at least one of the first synchronization operation command and the second synchronization operation command in the first data set synchronization request, a fourth reply message is sent to device 11 to indicate that data E was not successfully added from the database and / or data f was not successfully modified to data F, so as to notify device 11 that device 12 failed to successfully execute the first synchronization operation command and / or the second synchronization operation command in the first data set synchronization request, that is, the fourth reply message corresponds to the first data set synchronization request. For another example, if device 12 fails to successfully execute the third synchronization operation command in the second data set synchronization request, a fourth reply message is sent to device 11 to indicate that data G was not successfully deleted from the database, so as to notify device 11 that device 12 failed to successfully execute the third synchronization operation command in the second data set synchronization request, that is, the fourth reply message corresponds to the second data set synchronization request.

[0173] Optionally, the dataset synchronization request also carries a request identity for identifying the dataset synchronization request, and accordingly, the fourth reply message also contains the request identity in the dataset synchronization request corresponding to the fourth reply message. Based on the request identity, the second device can conveniently indicate to the first device which specific dataset synchronization request synchronization operation command failed to execute successfully.

[0174] For example, when the first device sends a first data set synchronization request to the second device, the request identity ID-1 is added to the synchronization request. When the second device fails to execute the first synchronization operation command and / or the second synchronization operation command in the synchronization request, when the fourth reply message is sent to the first device, the data identity ID-1 is added to the fourth reply message to indicate that the second device has failed to execute the first synchronization operation command and / or the second synchronization operation command. For another example, when the first device sends a second data set synchronization request to the second device, the request identity ID-2 is added to the synchronization request. When the second device fails to execute the third synchronization operation command in the synchronization request, when the fourth reply message is sent to the first device, the data identity ID-2 is added to the fourth reply message to indicate that the second device has failed to execute the third synchronization operation command.

[0175] S211, when receiving the fourth reply message sent by the second device in response to the target data set synchronization request or when the third reply message sent by the second device is not received within a preset second time period, resending the target data set synchronization request to the second device; the starting time point of the second time period is the moment when the target data set synchronization request is issued.

[0176] In some optional implementations, a waiting time for the second device to reply to the data set synchronization request is preset on the first device. Exemplarily, the waiting time is preset as the second time. Obviously, the starting time point of the waiting time should be calculated from the moment when the data set synchronization request is issued, that is, the starting time point of the first time is the moment when the synchronization request is issued. Optionally, the second time can be equal to or different from the first time, but the second time is for a data set synchronization request, which contains more synchronization operation commands than a synchronization request containing only one synchronization operation command, that is, the time required to execute all the synchronization operation commands in the data set synchronization request is usually longer than the execution time of one synchronization operation command of the synchronization request. Therefore, the second time can also be set to be longer than the first time.

[0177] If the first device receives the third reply message sent by the second device in response to a data set synchronization request within the second time period, it means that the second device successfully executes all synchronization operation commands in the data set synchronization request.

[0178] If the first device does not receive the third reply message sent by the second device for a data set synchronization request within the second time period or receives the fourth reply message sent by the second device for a data set synchronization request within the second time period, it indicates that the second device has failed to execute at least one synchronization operation command in the data set synchronization request. In this case, the first device resends the data set synchronization request with the synchronization operation command failed to execute to the second device, so that the second device repeats the synchronization operation command in the data set synchronization request with the synchronization operation command failed to execute, and continues to monitor the execution status of the synchronization operation command in the resent (redundant) data set synchronization request.

[0179] If the second device successfully executes the synchronization operation command in the (redundant) data set synchronization request, it will send a third reply message to the first device within the second time period.

[0180] If the first device does not receive the third reply message sent by the second device for the (redundant) data set synchronization request within the second time period or receives the fourth reply message sent by the second device for the (redundant) data set synchronization request within the second time period, the first device again sends the second (redundant) data set synchronization request to the second device and continues to monitor the execution status of the synchronization operation command in the resent second (redundant) data set synchronization request.

[0181] The (redundant) data set synchronization requests are sent cyclically in this way until the first device receives a third reply message indicating that the second device successfully executes the synchronization operation commands in these (redundant) data set synchronization requests.

[0182] Optionally, if the number of (redundant) data set synchronization requests sent by the first device reaches a preset number, and the first device still fails to receive a third reply message indicating that the second device successfully executes the synchronization operation commands in these (redundant) data set synchronization requests, an alarm prompt indicating that the master-slave database data synchronization has failed is output, or the data of the first database is directly copied in full to the second database. Exemplarily, the preset number is five.

[0183] For example, the second preset duration is The first device at time Send a synchronization request for the first data set to the second device at time , then the first device will start from time Start timing. If at time A third reply message for the first data set synchronization request is received when , it is determined that the first synchronization operation command and the second synchronization operation command in the first data set synchronization request are executed successfully.

[0184] If at the time A fourth reply message for the first data set synchronization request is received when , it is determined that the first synchronization operation command and / or the second synchronization operation command in the first data set synchronization request failed to execute; or if at time A third reply message for the first data set synchronization request has not been received yet, and , it is determined that the first synchronization operation command and / or the second synchronization operation command in the first data set synchronization request has failed to execute. In this case, the first device resends the first data set synchronization request to the second device, that is, sends a (redundant) first data set synchronization request to the second device, and continues to monitor the execution status of the synchronization operation command in the resent (redundant) first data set synchronization request.

[0185] In an embodiment of the present application, when data in the master-slave database is synchronized, the first device generates synchronization operation commands in parallel, generates data set synchronization requests for each data set in parallel, sends data set synchronization requests to the second device in parallel, and adds at least one synchronization operation command to the data set synchronization request sent by the first device to the second device. After receiving the data set synchronization request, the second device parses each data set synchronization request in parallel and executes the synchronization operation commands parsed from each data set synchronization request in parallel. This effectively improves the rate and efficiency of data synchronization between the master and slave databases to a great extent.

[0186] In some optional implementations, when sending a synchronization request to the second device (in parallel) in step S13 or when sending a data set synchronization request to the second device (in parallel) in step S132, it is necessary to control the synchronization timing to ensure that the database system composed of the first device and the second device provides data storage services to the outside normally. This is explained below:

[0187] Optionally, the above step S13 (or S132) further includes the following steps 1 to 3:

[0188] Step 1: when it is monitored that the working state of the second device returns to normal, generating a (data set) synchronization request according to the first type of tags and the data identified by the first type of tags;

[0189] Step 1 is the same as the aforementioned step S13 (or S132). Please refer to the relevant instructions in the previous text for details, which will not be repeated here.

[0190] Step 2: determining the sending condition of the (data set) synchronization request according to a preset synchronization strategy; the synchronization strategy is determined according to the service characteristics of the target service, the target service is a service that uses the first database for data storage; the service characteristics are obtained by statistically analyzing the target service according to a preset statistical analysis dimension;

[0191] The target service refers to a storage service for storing data in a database system composed of a first device and a second device, that is, a service for storing data using the first database (or the second database).

[0192] Optionally, the analysis dimensions include at least one of the following: read traffic / write traffic of the first database at each time point within a preset time period (for example, every day) and IOPS (Input / Output Per Second, input and output or read and write times per second) at each time point within the preset time period (for example, every day), load of the first device at each time point within the preset time period (for example, every day), resource consumption of the first device at each time point within the preset time period (for example, every day) (for example, memory usage percentage, CPU usage percentage, disk usage percentage, etc.).

[0193] Optionally, the service feature includes at least one of the following:

[0194] an average value of the read traffic / write traffic of the first database at each time point in a preset time period (for example, every day) determined according to the read traffic / write traffic of the first database at each time point in a preset time period (for example, every day);

[0195] a maximum / minimum value of the read traffic / write traffic of the first database within a preset time period (for example, every day) determined according to the read traffic / write traffic of the first database at each time point within the preset time period (for example, every day);

[0196] A business peak / valley time period of the first database within a preset time period (e.g., every day) determined according to the read traffic / write traffic of the first database at each time point within a preset time period (e.g., every day) and a preset read traffic / write traffic water level value of the first database within a preset time period (e.g., every day);

[0197] An average value of the IOPS of the first database at each time point in a preset time period (for example, every day) determined according to the IOPS of the first database at each time point in a preset time period (for example, every day);

[0198] A maximum / minimum value of the IOPS amount of the first database at each time point in a preset time period (for example, every day) determined according to the IOPS amount of the first database at each time point in a preset time period (for example, every day);

[0199] A business peak / valley time period of the first database within a preset time period (for example, every day) is determined according to the IOPS amount of the first database at each time point within a preset time period (for example, every day) and a preset IOPS amount water level value of the first database within the preset time period (for example, every day);

[0200] A service peak / valley time period of the first device within a preset time period (for example, every day) determined according to the load of the first device at each time point within a preset time period (for example, every day);

[0201] The business peak / valley time period of the first device within a preset time period (e.g., every day) is determined based on the resource consumption (e.g., memory usage percentage, CPU usage percentage, disk usage percentage, etc.) of the first device at various time points within a preset time period (e.g., every day).

[0202] After determining the service characteristics, a synchronization strategy may be determined according to the service characteristics. Exemplarily, the synchronization strategy includes at least one of the following:

[0203] The first strategy is to immediately start data synchronization between the master and slave databases after monitoring that the working status of the second device has returned to normal;

[0204] The second strategy is to synchronize the master and slave databases during the business off-peak period.

[0205] The third strategy is to synchronize the master and slave database data during the preset recovery time period (for example, between 2 am and 3 am every day).

[0206] You can select a synchronization strategy from the preset synchronization strategies as the strategy for actual online operation. For example, the default synchronization strategy - synchronizing the master and slave database data in the preset recovery time period (for example, between 2 am and 3 am every day) - is used as the strategy for actual online operation.

[0207] Among them, if the first strategy is selected, the timeliness of the master-slave database data synchronization can be guaranteed, but it may affect the overall performance of the master-slave database as a whole in providing external data storage services during peak hours. If the second strategy is selected, it can be guaranteed that the overall performance of the master-slave database as a whole in providing external data storage services will not be affected during peak hours, but it will reduce the timeliness of the master-slave database data synchronization. Because the early morning is usually a low-peak period for business, but it is occasionally a peak period for business. Therefore, if the third strategy is selected, it can be guaranteed as much as possible that the overall performance of the master-slave database as a whole in providing external data storage services will not be affected during peak hours, and the timeliness of the master-slave database data synchronization will not be reduced as much as possible. It is a relatively compromise strategy. Therefore, the synchronization strategy can be flexibly selected according to actual needs. The first strategy can be selected for businesses with high requirements for database consistency, and the second strategy can be selected for businesses with high requirements for stability of database external storage services.

[0208] After the synchronization strategy is selected, the sending condition is determined accordingly. For example, if the first strategy is selected, the sending condition is that the master-slave database storage system enters a business trough period.

[0209] Step three: when it is detected that the sending condition is met, send the (data set) synchronization request to the second device.

[0210] For example, when it is detected that the sending condition is met (for example, it is detected that the master-slave database storage system enters a business trough period), a synchronization request is sent to the second device, that is, the master-slave database data synchronization is started.

[0211] In some optional implementations, when sending a synchronization request to the second device (in parallel) in step S13 or when sending a data set synchronization request to the second device (in parallel) in step S132, it is necessary to control the outflow speed to ensure that the database system composed of the first device and the second device provides data storage services normally. The following is an explanation:

[0212] Optionally, the above step S13 (or S132) further includes the following steps:

[0213] Step 4: Obtain the maximum throughput of the first database, the historical peak throughput of the first database, and the resource pressure of the first device;

[0214] In the embodiment of the present application, the throughput is the aforementioned IOPS. The maximum throughput of the first database is determined based on the physical performance of the first device where the first database is located, and is therefore predetermined.

[0215] The historical peak throughput of the first database has been determined in the aforementioned step 2.

[0216] In step 2, the resource consumption (e.g., memory usage percentage, CPU usage percentage, disk usage percentage, etc.) of the first device at various time points within a preset time period (e.g., every day) has been statistically analyzed, and the average amount of resource consumption (e.g., memory usage percentage, CPU usage percentage, disk usage percentage, etc.) of the first device at various time points within the preset time period (e.g., every day) can be determined. The resource pressure of the first device can be determined based on the average amount of resource consumption of various resources at various time points.

[0217] Step 5: Determine a synchronization rate according to the maximum throughput, the historical peak throughput, and the resource pressure amount;

[0218] The synchronization throughput upper limit is determined according to the maximum throughput and the historical peak throughput of the first database. For example, if the maximum throughput is 10,000 and the historical peak throughput is 8,000, the synchronization throughput upper limit is 2,000, that is, 10,000-8,000.

[0219] Determine the synchronization throughput adjustment coefficient according to the resource pressure of the first device, including: obtaining the weight of each resource and the average value of each resource at each time point within a preset time period (for example, every day), and determine the synchronization throughput adjustment coefficient N according to the weight of each resource and the average value of each resource at each time point within the preset time period (for example, every day).

[0220] For example, the average value of memory usage percentage at each time point in a preset time period (such as every day) is mem.avg, the average value of CPU usage percentage at each time point in a preset time period (such as every day) is cpu.avg, and the average value of disk usage percentage at each time point in a preset time period (such as every day) is disk_util.avg. The weight of memory usage percentage is 30%, the weight of CPU usage percentage is 50%, and the weight of disk usage percentage is 20%. Then determine the synchronization throughput adjustment factor . .

[0221] Determine the synchronization rate based on the synchronization throughput adjustment factor N and the synchronization throughput upper limit .For example: .

[0222] Step six: According to the synchronization rate, control the outgoing flow rate of sending the (data set) synchronization request to the second device.

[0223] Optionally, after determining the synchronization rate, controlling the outflow rate of sending the (data set) synchronization request to the second device is to control the sending concurrency of the (data set) synchronization request according to the determined synchronization rate. It is essentially the same as IOPS, so you can control the first device to send IOPS to the second device every second. Synchronous request.

[0224] like ,At this time, the system is in a business trough period, and the control synchronization rate is in a high-speed state;

[0225] like ,At this time, the system is in the business saturation period (i.e., the intermediate state between the trough period and the peak period), and the control synchronization rate is in the medium speed state;

[0226] like At this time, the system is in the peak business period and the control synchronization rate is in a low speed state.

[0227] By controlling the outflow rate of synchronization requests through steps 4 to 6 above, it is possible to minimize the impact on external data storage services during the master-slave database data synchronization period.

[0228] Of course, in some optional implementations, when sending a synchronization request to the second device (in parallel) in step S13 or when sending a data set synchronization request to the second device (in parallel) in step S132, both the synchronization timing and the outflow speed need to be controlled. That is, the first device performs data synchronization on the basis of executing steps 1 to 6. Since steps 1 to 6 have been described in the above description, they will not be repeated here.

[0229] In some optional embodiments, the above-mentioned first device can be a first cluster system composed of at least two computer devices, and the first cluster system works collaboratively based on a preconfigured collaborative algorithm, so that the first cluster system "looks" equivalent to a single device to the outside, and the first database is deployed on the first cluster system; similarly, the above-mentioned second device can also be a second cluster system composed of at least two computer devices, and the second cluster system works collaboratively based on a preconfigured collaborative algorithm, so that the second cluster system "looks" equivalent to a single device to the outside, and the second database is deployed on the second cluster system.

[0230] In some optional implementations, the first device and the second device are two nodes in a cluster system, and the cluster system provides data storage services to the outside as a whole. What is stored on the first device and the second device is metadata (Meta Data) of the data stored in the cluster system. The so-called metadata is data about data, and is data used to describe data (Data that describes other data). In other words, what is stored on the first device and the second device is data used to describe the data stored in the cluster.

[0231] In some optional implementations, the first database and the second database are databases of the same type. For example, the first database and the second database are both relational databases.

[0232] The present application also provides a method for synchronizing master-slave database data. Fig. 9 As shown, Fig. 9 This is a flowchart of a method for synchronizing master-slave database data provided by an embodiment of the present application. Fig. 9 The master-slave database data synchronization method is applied to a first device, in which a first database is deployed. The master-slave database data synchronization method includes the following steps:

[0233] S21, monitoring the working status of a second device on which a second database is deployed; the first database and the second database operate based on a dual-write mechanism;

[0234] S22, when it is detected that the working state of the second device is abnormal, adding a preset first-category tag corresponding to the target data generated after the first moment in the first database; the first moment is the moment when the working state of the second device is detected to change from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first-category tag includes: an unsynchronized tag and a deleted tag; the newly added data and the modified data both correspond to the unsynchronized tag, and the data to be deleted corresponds to the deleted tag;

[0235] S23, when it is monitored that the working status of the second device has returned to normal, a synchronization request is generated according to the first type of tag and the data identified by the first type of tag, and the synchronization request is sent to the second device; the synchronization request carries a synchronization operation command for synchronizing the data identified by the first type of tag in the second database.

[0236] Steps S21 to S23 are similar to steps S11 to S13 performed by the first device in the aforementioned embodiment. Please refer to the relevant descriptions in the foregoing text for details, and will not be repeated here.

[0237] The present application also provides a method for synchronizing master-slave database data. Fig.10 As shown, Fig.10 This is a flowchart of a method for synchronizing master-slave database data provided by an embodiment of the present application. Fig.10 The method for synchronizing data between a master and a slave database is applied to a second device, in which a second database is deployed. The method for synchronizing data between a master and a slave database comprises the following steps:

[0238] S31, when receiving a synchronization request sent by a first device deployed with a first database, parsing a synchronization operation command from the synchronization request; the second database and the first database operate based on a dual-write mechanism;

[0239] S32, execute the synchronization operation command to synchronize the data identified by the first type of label in the first database with the second database; the first type of label is a preset first type of label corresponding to the target data generated after the first moment in the first database, which is added by the first device when the first device monitors that the working state of the second device is abnormal; the first moment is the moment when the first device monitors that the working state of the second device changes from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first type of label includes: unsynchronized labels and deleted labels; the newly added data and the modified data both correspond to the unsynchronized labels, and the data to be deleted corresponds to the deleted labels.

[0240] Steps S21 to S23 are similar to steps S14-S15 performed by the first device in the aforementioned embodiment. Please refer to the relevant descriptions in the previous text for details, and will not be repeated here.

[0241] In some optional implementations, the first database is a relational database, and a tag field is preset in a data table in the first database; the above step S13 further includes:

[0242] When it is monitored that the working state of the second device is abnormal, for the target data generated after the first moment in the data table in the first database, a preset first type label is added under the label field in the tuple where the target data is located.

[0243] In relational databases, data is organized using data tables. A row in a data table is a tuple (or record), and a column is a field. A new column is added to each data table to add a tag field. When a piece of data is written after the first moment, the first type of tag corresponding to the write operation is added to the tag field in the tuple where the data is located.

[0244] For details, please refer to the examples in the following text, and no further examples will be given here.

[0245] To facilitate understanding by those skilled in the art, the master-slave database data synchronization method provided by the present application is described below with an example.

[0246] In this example, Figure 1 The database storage system shown in is a distributed storage system for storing video (or image) data. Figure 1 Device 11 and Device 12 in the other nodes (in Figure 1 ), the two nodes (i.e. Figure 1 Devices 11 and 12 in the storage device store metadata of video (or image) data stored on other nodes. For example, the metadata includes: user information, channel information, file information, etc.

[0247] The user information is the information of each user who stores video (image) data in the distributed storage system, and the user information table is used to store the relevant information of each user, such as Table 1 in the following example.

[0248] Channel information is the source of the stored video (image) data. One channel usually corresponds to a front-end camera (or video camera) that collects video (image) data. Of course, there is also the possibility that one camera (or video camera) corresponds to multiple channels. The channel information table is used to store the relevant information of each channel. For example, Table 2 in the following example.

[0249] File information is information about stored video (image) data. Video (image) data is a persistent view file stored in a distributed storage system on other storage nodes except device 11 and device 12. The streaming media service generates file names after segmenting the streaming data according to a certain video and image segmentation strategy. The file information table is used to store relevant information of each file. For example, Tables 3 to 5 are shown in the following examples.

[0250] The database on device 11 and the database on device 12 are both relational databases. For example, the database on device 11 and the database on device 12 are both MySQL databases, so the database on device 11 and the database on device 12 can perform add, delete, modify, and query operations based on the SQL language. Among them, the add operation, modify operation, and delete operation are collectively referred to as write operations; the query operation is also referred to as a read operation.

[0251] The database should follow the principle of database consistency. Database consistency means that the result of transaction execution must change the database from one consistent state to another consistent state. Ensuring database consistency means that when the transaction is completed, all data must be in a consistent state. In a relational database, all rules must be applied to transaction modifications (for example, modifying the consistency of the state of data in each data table based on the relationship between each data table) in order to maintain the integrity of all data.

[0252] How does the database state change? Every data change will cause the database state to migrate. If the initial state of the database is C0, the submission of the first transaction T1 will cause the system to generate a SYSTEM CHANGE NUMBER (SCN), which is the database state from C0 to C1. When the second transaction T2 is executed, the database state changes from T1 to T2, and so on. When the Tnth transaction is executed, the database state changes from C(n-1) to Cn.

[0253] There are two main aspects to defining consistency: consistent writes and consistent reads.

[0254] Consistent write: Data changes executed by a transaction can only be based on the last consistent state and can only be reflected in one state. The change result of T(n) can only be based on C(n-1), C(n-2), ...C(1) states and can only be reflected in C(n) state. In other words, only one transaction can change data in one state, and two or more transactions are not allowed to change data in one state. As for which state the consistent write is based on, it is necessary to determine whether the T(n) transaction has a dependency relationship with T(n-1), T(n-2), ...T(1).

[0255] Consistent read: A transaction can only read data from one state, not from two or more states. That is, T(n) can only read data from one state among C(n-1), C(n-2)...C(1), and cannot read part of the data from C(n-1) and the other part from C(n-2).

[0256] Example:

[0257] Consistent write:

[0258] Define 100 transactions T(1)...T(100) to implement the same logic update table set i=i+1. The initial value of i is 0. What is the value of i after executing these 100 transactions concurrently? It may be easy to think that it is 100. So how to understand it from the perspective of consistency?

[0259] The database is randomly scheduled to execute at T(50). At this time, the database status is C(0). Other transactions are dependent on T(50). According to the write consistency principle, other transactions must wait until T(50) is completed and the database status changes to C(1) before they can execute. Therefore, the database uses a lock mechanism to block the execution of other transactions. After T(50) is completed, the database status migrates from C(0) to C(1). After waking up other transactions, the database is randomly scheduled to execute at T(89), and so on until all transactions are scheduled and executed, and the database status finally changes to C(100).

[0260] Consistent read:

[0261] Let’s take the above example again. Suppose T(1)…T(100) are executed sequentially and select ifrom table is executed at different times. What value of i do we see?

[0262] 1. During the execution of T(1), the database status has not been migrated yet, and the read i=0;

[0263] 2. After T(1) is executed, during the execution of T(2), the database state is migrated to C(1), and the read i=1.

[0264] In a distributed storage system, when a node fails, such as when P occurs in the CAP theory (the CAP principle is also called the CAP theorem, which refers to consistency (C, Consistency), availability (A, Availability), and partition tolerance (P, Partition tolerance) in a distributed system. The essence of the CAP principle is either AP, CP, or AC, but CAP does not exist), in order to ensure the availability of the business, it is usually necessary to record the business operations when partition fault tolerance occurs by recording operation logs. When the partition fault tolerance P failure is restored, it is usually played back at the previous faulty node based on the operation log to finally achieve data consistency. However, in the embodiments of the present application, the goal is to get rid of the dependence on operation logs and achieve data consistency in a simpler way.

[0265] As mentioned above, device 11 and device 12 and the databases thereon provide metadata storage services for the entire distributed storage system based on a dual-write mode and a dual-machine hot standby mechanism. Under normal circumstances, when a write operation is performed to write data to a database on either device 11 or device 12, the same write operation will be performed on the other database to write the same data. When any one of the devices fails, the other normal device will take over the metadata storage service to provide metadata storage services for the entire distributed storage system. The metadata storage service switches from a dual-write mode to a single-write mode, causing data inconsistencies between the master and slave databases, and begins to execute the master-slave database data synchronization method in any of the aforementioned implementations to achieve database consistency, that is, to synchronize differentiated data on the master and slave databases while the abnormal database in the master-slave database is in an abnormal state. Details are as follows:

[0266] Based on the above, in this example, the databases on device 11 and device 12 are both MySQL databases, so they are both stored in the form of data tables in mathematical logic. The structures of the data tables in the master database and the slave database are modified, and a new label field is added in each data table to identify the consistency status of the data between the master and slave databases. For example, under this field, the character "0" is used as the label of the data that is consistent between the master and slave databases (that is, the synchronization between the master and slave databases has been completed), the character "1" is used as the label of the data that is inconsistent between the master and slave databases (that is, the synchronization between the master and slave databases has not been completed but is not to be deleted), and the character "2" is used as the label of the data to be deleted that was requested to be deleted but was aborted between the master and slave databases.

[0267] For example, Tables 1 to 5 are respectively a user information table, a channel information table under Xiao Zhang's name, a file information table under the First Avenue channel in the First Block, a file information table under the Second Avenue channel in the First Block, and a file information table under the Third Avenue channel in the First Block.

[0268] Table 1 User information table in the first database

[0269]

[0270] Table 2 Channel information table under Xiao Zhang's name in the first database

[0271]

[0272] Table 3 File information table of the First Avenue channel in the First Block in the First Database

[0273]

[0274] Table 4 File information table of the Second Avenue channel in the first block in the first database

[0275]

[0276] Table 5 File information table of the Third Avenue channel in the first block in the first database

[0277]

[0278] From Tables 1 to 5 of the above examples, it can be seen that in this example, three-layer synchronization status marking is performed based on users, channels, and files. It should be noted that if the data status in any table changes, it must be synchronized to all other related data tables. For example, the data in Table 3 has been written, which causes the data status to change. Therefore, it is necessary to synchronize to Tables 2 and 1. Add a label "1" to the label field of the tuple where the first street in the first block is located in Table 2, and add a label "1" to the label field of the tuple where Xiao Zhang is located in Table 1. The label fields in each data table in the above example are the labels added by device 11 in the master database during the abnormal period of device 12.

[0279] Through the above three layers of synchronization status marking, the inconsistent state of the database can be easily identified and displayed on the system page. Compared with traditional operation log records, it supports more precise and specific data inconsistency levels (accurate to users and channels). It can quickly perceive and transparently transmit anomalies, and the granularity of anomaly display is more specific.

[0280] When a faulty machine goes offline, based on the application scenario and CAP theory, this system prioritizes service availability, and then performs business recovery based on the marking of inconsistent states, ultimately achieving consistent business recovery.

[0281] After the failed machine in the dual-machine hot standby architecture goes offline, the host service will record the offline status of the standby machine. First, when the User (i.e., the user) writes and modifies business data, the metadata service will first set the inconsistent status field of the User table to "out of sync", indicating that the user's data has changed while the standby machine is offline, and there is inconsistent data in the database.

[0282] Then, based on the user information, find the channel table that the user needs to operate this time; and update the label field of the channel table to an inconsistent state; this indicates that the data of the channel has changed while the standby machine is offline, and there is inconsistent data in the channel;

[0283] Finally, based on the User, channel and other information, the file table that needs to be operated is finally found, and the actual database file table is added, deleted, modified and checked. If any changes occur, the current file status is recorded as an asynchronous state; all files with status changes will be marked as "asynchronous state" and the recovery thread will trigger recovery when the failed machine is restored.

[0284] Through the precise positioning of the above three-layer marking, the inconsistent status of the file can be marked at the database table level, which can perfectly solve the single-machine offline and final consistency recovery problems of the dual-machine hot standby database in the security scene. At the same time, compared with the traditional fault recording and consistency recovery solution based on operation logs, it can solve the waste of disk storage space caused by operation logs and the performance loss of operation log writing, reading and loading.

[0285] When the failed machine is restored and online, the heartbeat callback thread of the master and standby machines will asynchronously notify and wake up the synchronous recovery task to start working. The synchronous recovery task will restore data consistency according to the pre-set recovery strategy and restrictions. This part of the logic will be expanded in detail later. When the synchronous recovery task is awakened, and under the premise of meeting the pre-set recovery strategy, the recovery thread will first traverse the User table, find users in inconsistent states, and encapsulate them into user tasks to be restored. Then, based on the status of the channel table under the user, it is encapsulated into a channel task to be restored. The next step is to traverse all the marked files in the channel that are in an asynchronous state and meet the recovery strategy, and encapsulate the file recovery task. In order to improve the efficiency of the entire consistency recovery, the system designs a multi-threaded recovery mode for channel-level task batch recovery; files in inconsistent states within the channel will be restored in the corresponding thread.

[0286] Confirmation of recovery results: To ensure that the recovery task is truly executed successfully, each channel task recovery thread will first read the latest inconsistent data information in the host database, modify the standby database to the latest data through the database connection pool of the standby machine, and successfully execute a specific file recovery task, and set the corresponding file status to the synchronization status. Each file recovery task is an atomic operation.

[0287] At the end of each task, a callback is made to the synchronous recovery task to set the task recovery completion, and the mark is changed to the synchronous state. It can support abnormal interruption of the recovery task without affecting the next normal recovery, and finally achieve database consistency.

[0288] When a recovery task is abnormally interrupted, it can be restored from the interrupted position. Different recovery strategies and recovery flow control can be set based on the statistics of the business model to reduce the impact of the recovery task on real-time business.

[0289] The above recovery strategies, such as default and customized, recovery flow control, recovery time period, and recovery rate, all support personalized settings. For scenarios where no strategy is set, the system performs consistency recovery according to the default strategy, minimizing business impact while ensuring system availability and consistency.

[0290] For the specific process of adding tags and synchronizing data based on the added tags, please refer to the relevant instructions in the previous article and will not be explained in detail here.

[0291] The above examples are intended to illustrate the method for master-slave database data synchronization provided in the embodiments of the present application, and are not specific limitations on the application scenarios of the method for master-slave database data synchronization provided in the embodiments of the present application. Obviously, in actual situations, the number and type of data tables in the database, the number and type of data in each data table, the relationship between each data table, etc. will be relatively more and more complex. Due to space limitations, it is difficult to fully include and illustrate them in an enumerated form. However, based on this example, a person skilled in the art can fully understand the technical concept and implementation method of the method for master-slave database data synchronization provided in the embodiments of the present application. Therefore, the technical solution for master-slave database data synchronization implemented based on the technical concept of the method for master-slave database data synchronization provided in the embodiments of the present application should be within the scope of rights requested for protection in the embodiments of the present application.

[0292] The method for master-slave database data synchronization provided in the embodiment of the present application can be executed by a master-slave database data synchronization device. In the embodiment of the present application, the method for master-slave database data synchronization performed by a master-slave database data synchronization device is taken as an example to illustrate the master-slave database data synchronization device provided in the embodiment of the present application.

[0293] like Fig.11 , which shows a schematic diagram of the structure of a device for synchronizing master-slave database data provided by an embodiment of the present application. Fig.11 The master-slave database data synchronization device 100 is applied to a first device, in which a first database is deployed; the device 100 includes:

[0294] The first monitoring module 1001 is used to monitor the working status of the second device deployed with the second database; the first database and the second database are operated based on a dual-write mechanism;

[0295] The first adding module 1002 is used to add a preset first-class label corresponding to the target data generated after the first moment in the first database when the working state of the second device is detected to be abnormal; the first moment is the moment when the working state of the second device is detected to change from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first-class label includes: an unsynchronized label and a deleted label; the newly added data and the modified data both correspond to the unsynchronized label, and the data to be deleted corresponds to the deleted label;

[0296] The first sending module 1003 is used to generate a synchronization request based on the first type of tag and the data identified by the first type of tag, and send the synchronization request to the second device when it is monitored that the working status of the second device returns to normal; the synchronization request carries a synchronization operation command for synchronizing the data identified by the first type of tag in the second database.

[0297] In some optional implementations, the device 100 further includes:

[0298] A first determining module is used to determine the type of the first category label of the data to be operated by the synchronization operation command when receiving a first reply message sent by the second device indicating that the second device successfully executes the synchronization operation command;

[0299] A first modification module is used to delete the unsynchronized tag of the data to be operated by the synchronization operation command from the first database or modify the unsynchronized tag of the data to be operated by the synchronization operation command to a synchronized tag when it is determined that the type is an unsynchronized tag;

[0300] The first deletion module is used to delete the data to be operated by the synchronization operation command and its deletion tag from the first database when it is determined that the type is a deletion tag.

[0301] In some optional implementations, the device 100 further includes:

[0302] A second sending module is used to resend the synchronization request to the second device upon receiving a second reply message sent by the second device indicating that the second device failed to execute the synchronization operation command or when the first reply message sent by the second device is not received within a preset first time period; the starting time point of the first time period is the moment when the synchronization request is issued.

[0303] In some optional implementations, the first sending module 1003 includes:

[0304] A first division module is used to divide the data identified by the first type of tags according to a preset division standard to obtain at least one data set when it is monitored that the working state of the second device returns to normal;

[0305] The third sending module is used to perform the following steps in parallel for each data set in the at least one data set: generate a synchronization operation command corresponding to each data in the target data set according to each data in the target data set and the first type of label corresponding to each data set, encapsulate all the obtained synchronization operation commands into a target data set synchronization request, and send the target data set synchronization request to the second device; the target data set is any data set in the at least one data set.

[0306] In some optional implementations, the device 100 further includes:

[0307] A second determination module is configured to determine the type of the first category label of the data to be operated by each synchronization operation command in the target data set synchronization request when receiving a third reply message sent by the second device in response to the target data set synchronization request; the third reply message is used to indicate that the second device successfully executes all the synchronization operation commands in the target data set synchronization request;

[0308] A second modification module is used to delete the unsynchronized tag of the data to be operated by the synchronization operation command from the first database or modify the unsynchronized tag of the data to be operated by the synchronization operation command to a synchronized tag when it is determined that the type of the first type of tag of the data to be operated by the synchronization operation command is an unsynchronized tag;

[0309] The second deleting module is used to delete the data to be operated by a synchronization operation command and its deletion tag from the first database when it is determined that the type of the first type of tag of the data to be operated by the synchronization operation command is a deletion tag.

[0310] In some optional implementations, the device 100 further includes:

[0311] The fourth sending module re-sends the target dataset synchronization request to the second device when receiving the fourth reply message sent by the second device in response to the target dataset synchronization request or when the third reply message sent by the second device is not received within a preset second time period; the starting time point of the second time period is the moment when the target dataset synchronization request is issued; the fourth reply message is used to indicate that the second device fails to execute at least one synchronization operation command in the target dataset.

[0312] In some optional implementations, the first sending module 1003 includes:

[0313] A first generating submodule is used to generate a synchronization request according to the first type of tags and the data identified by the first type of tags when monitoring that the working state of the second device returns to normal;

[0314] A first determination submodule, configured to determine a sending condition of the synchronization request according to a preset synchronization strategy; the synchronization strategy is determined according to a service feature of a target service, the target service is a service that uses the first database for data storage; the service feature is obtained by statistically analyzing the target service according to a preset statistical analysis dimension;

[0315] The first sending submodule is used to send the synchronization request to the second device when it is detected that the sending condition is met.

[0316] In some optional implementations, the first sending module 1003 further includes:

[0317] A first acquisition submodule, configured to acquire a maximum throughput of the first database, a historical peak throughput of the first database, and a resource pressure amount of the first device;

[0318] A second determination submodule, configured to determine a synchronization rate according to the maximum throughput, the historical peak throughput, and the resource pressure amount;

[0319] The first control submodule is used to control the outgoing flow rate of sending the synchronization request to the second device according to the synchronization rate.

[0320] In some optional implementations, the first database is a relational database, and a tag field is preset in a data table in the first database; the first adding module 1002 includes:

[0321] The first adding submodule is used to add a preset first type of label under the label field in the tuple where the target data is located, for the target data generated after the first moment in the data table in the first database, when it is monitored that the working state of the second device is abnormal.

[0322] like Fig.12 , which shows a schematic diagram of the structure of a device for synchronizing master-slave database data provided by an embodiment of the present application. Fig.12 The master-slave database data synchronization device 110 is applied to a second device, in which a second database is deployed; the device 110 includes:

[0323] The first parsing module 1101 is used to parse a synchronization operation command from the synchronization request when receiving a synchronization request sent by a first device deployed with a first database; the second database and the first database operate based on a dual-write mechanism;

[0324] The first execution module 1102 is used to execute the synchronization operation command to synchronize the data identified by the first type of label in the first database with the second database; the first type of label is a preset first type of label corresponding to the target data generated after the first moment in the first database during the period when the first device monitors that the working state of the second device is abnormal; the first moment is the moment when the first device monitors that the working state of the second device changes from normal to abnormal; the target data includes at least one of the following: newly added data, modified data and data to be deleted that is requested to be deleted but is aborted; the first type of label includes: unsynchronized labels and deleted labels; the newly added data and the modified data both correspond to the unsynchronized labels, and the data to be deleted corresponds to the deleted labels.

[0325] In some optional implementations, the device 110 further includes:

[0326] The fifth sending module is used to send a first reply message to the first device to indicate that the synchronization operation command is successfully executed when the synchronization operation command is successfully executed.

[0327] In some optional implementations, the device 110 further includes:

[0328] The sixth sending module is used to send a second reply message indicating the failure of executing the synchronization operation command to the first device when the synchronization operation command fails to be executed.

[0329] In some optional implementations, the first parsing module 1101 includes:

[0330] A first parsing submodule is used to parse all synchronization operation commands from the target data set synchronization request when receiving the target data set synchronization request sent by the first device deployed with the first database;

[0331] The first execution module 1102 includes:

[0332] The first execution submodule is used to execute in parallel all synchronization operation commands parsed from the target data set synchronization request to synchronize the target data set in the second database; the target data set is any data set in at least one data set obtained by dividing the data identified by the first category tag by the first device according to a preset division standard.

[0333] In some optional implementations, the device 110 further includes:

[0334] The seventh sending module is used to send a third reply message to the first device indicating that all synchronization operation commands in the target dataset synchronization request are successfully executed when all synchronization operation commands parsed from the target dataset synchronization request are successfully executed.

[0335] In some optional implementations, the device 110 further includes:

[0336] The eighth sending module is used to send a fourth reply message to the first device indicating that at least one synchronization operation command in the target dataset synchronization request has failed to execute when at least one synchronization operation command parsed from the target dataset synchronization request has failed to execute.

[0337] The master-slave database data synchronization device 100 and the master-slave database data synchronization device 110 in the embodiment of the present application can be electronic devices, or components in electronic devices, such as integrated circuits or chips. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0338] The master-slave database data synchronization device 100 and the master-slave database data synchronization device 110 in the embodiment of the present application can be devices with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0339] In the embodiment of the present application, the device 100 for synchronizing data from a slave database and the device 110 for synchronizing data from a master-slave database can correspondingly implement the various processes implemented in the aforementioned method embodiment, and will not be described again here to avoid repetition.

[0340] In some optional embodiments, such as Fig.13 As shown, an embodiment of the present application also provides an electronic device 1300, including a processor 1301 and a memory 1302, wherein the memory 1302 stores programs or instructions that can be executed on the processor 1301, and the programs or instructions stored in the memory 1302, when executed by the processor 1301, implement the various steps of the above-mentioned master-slave database data synchronization method embodiment, and can achieve the same technical effect. To avoid repetition, they are not repeated here.

[0341] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0342] Fig.14 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0343] The electronic device 170 includes, but is not limited to, components such as a radio frequency unit 1701, a network module 1702, an audio output unit 1703, an input unit 1704, a sensor 1705, a display unit 1706, a user input unit 1707, an interface unit 1708, a memory 1709, and a processor 17010. Those skilled in the art will appreciate that the electronic device 170 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 17010 through a power management system, so as to implement functions such as managing charging, discharging, and power consumption management through the power management system. Fig.14 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0344] The processor 17010 is used for:

[0345] Monitoring the working status of a second device on which a second database is deployed; the first database and the second database operate based on a dual-write mechanism; the first database is deployed on the first device;

[0346] In the case where it is detected that the working state of the second device is abnormal, a preset first-category tag corresponding to the target data generated after the first moment in the first database is added; the first moment is the moment when the working state of the second device is detected to change from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first-category tag includes: an unsynchronized tag and a deleted tag; the newly added data and the modified data both correspond to the unsynchronized tag, and the data to be deleted corresponds to the deleted tag;

[0347] When it is monitored that the working status of the second device has returned to normal, a synchronization request is generated based on the first type of tag and the data identified by the first type of tag, and the synchronization request is sent to the second device; the synchronization request carries a synchronization operation command for synchronizing the data identified by the first type of tag in the second database.

[0348] Or, processor 17010, configured to:

[0349] In the case of receiving a synchronization request sent by a first device on which a first database is deployed, parsing a synchronization operation command from the synchronization request; the second database and the first database operate based on a dual-write mechanism; the second database is deployed on a second device;

[0350] Execute the synchronization operation command to synchronize the data identified by the first type of label in the first database with the second database; the first type of label is a preset first type of label corresponding to the target data generated after the first moment in the first database when the first device monitors that the working state of the second device is abnormal; the first moment is the moment when the first device monitors that the working state of the second device changes from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first type of label includes: an unsynchronized label and a deleted label; the newly added data and the modified data both correspond to the unsynchronized label, and the data to be deleted corresponds to the deleted label.

[0351] It should be understood that in the embodiment of the present application, the input unit 1704 may include a graphics processor (GPU) 17041 and a microphone 17042, and the graphics processor 17041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1706 may include a display panel 17061, and the display panel 17061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1707 includes a touch panel 17071 and at least one of other input devices 17072. The touch panel 17071 is also called a touch screen. The touch panel 17071 may include two parts: a touch detection device and a touch controller. Other input devices 17072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0352] The memory 1709 can be used to store software programs and various data. The memory 1709 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1709 may include a volatile memory or a non-volatile memory, or the memory 1709 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0353] The processor 17010 may include one or more processing units; optionally, the processor 17010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 17010.

[0354] Any of the above-mentioned product embodiments can implement the various processes of the above-mentioned method embodiment for measuring vibration test parameters through its own processor operation, and can achieve the same technical effect. In order to avoid repetition, they will not be described one by one.

[0355] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by the processor, each process of the method embodiment of the master-slave database data synchronization is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the processor is the processor in the electronic device or electronic system described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a disk or an optical disk, etc.

[0356] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned master-slave database data synchronization method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0357] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0358] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the method embodiment for synchronizing data between master and slave databases as described above, and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0359] In the implementation methods provided in the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0360] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0361] In addition, each functional unit in each implementation of the embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0362] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each implementation method of the embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0363] The above description is only an implementation method of the embodiment of the present application, and does not limit the patent scope of the embodiment of the present application. The above specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can make many forms of equivalent structures or equivalent process changes made by using the description and drawings of the embodiment of the present application, or directly or indirectly apply them in other related technical fields without departing from the scope of protection of the purpose of the present application and the claims, which are also included in the patent protection scope of the embodiment of the present application.

Claims

1. A method for synchronizing master-slave database data, characterized in that: Applied to a first device, a first database is deployed in the first device; the method includes: Monitoring the working status of a second device on which a second database is deployed; the first database and the second database operate based on a dual-write mechanism; In the case where it is detected that the working state of the second device is abnormal, a preset first-category tag corresponding to the target data generated after the first moment in the first database is added; the first moment is the moment when the working state of the second device is detected to change from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first-category tag includes: an unsynchronized tag and a deleted tag; the newly added data and the modified data both correspond to the unsynchronized tag, and the data to be deleted corresponds to the deleted tag; When it is monitored that the working status of the second device has returned to normal, a synchronization request is generated based on the first type of tag and the data identified by the first type of tag, and the synchronization request is sent to the second device; the synchronization request carries a synchronization operation command for synchronizing the data identified by the first type of tag in the second database.

2. The method according to claim 1, characterized in that The method further comprises: Upon receiving a first reply message sent by the second device indicating that the second device successfully executes the synchronization operation command, determining a type of a first category tag of data to be operated by the synchronization operation command; In the case where it is determined that the type is an unsynchronized tag, deleting the unsynchronized tag of the data to be operated by the synchronization operation command from the first database or modifying the unsynchronized tag of the data to be operated by the synchronization operation command to a synchronized tag; In the case where it is determined that the type is a deletion tag, the data to be operated by the synchronization operation command and its deletion tag are deleted from the first database.

3. The method according to claim 2, characterized in that The method further comprises: Upon receiving a second reply message sent by the second device indicating that the second device failed to execute the synchronization operation command or upon not receiving the first reply message sent by the second device within a preset first time period, resending the synchronization request to the second device; the starting time point of the first time period is the moment when the synchronization request is issued.

4. The method according to claim 1, characterized in that The step of generating a synchronization request according to the first-type tag and the data identified by the first-type tag, and sending the synchronization request to the second device when the working state of the second device is detected to be restored to normal, includes: When it is monitored that the working state of the second device returns to normal, the data identified by the first type of tags is divided according to a preset division standard to obtain at least one data set; The following steps are performed in parallel for each data set in the at least one data set: a synchronization operation command corresponding to each data in the target data set is generated according to each data in the target data set and a first-category tag corresponding to each data set, all the obtained synchronization operation commands are encapsulated into a target data set synchronization request, and the target data set synchronization request is sent to the second device; the target data set is any data set in the at least one data set.

5. The method according to claim 4, characterized in that The method further comprises: upon receiving a third reply message sent by the second device in response to the target data set synchronization request, determining the type of the first category label of the data to be operated by each synchronization operation command in the target data set synchronization request; the third reply message is used to indicate that the second device successfully executes all the synchronization operation commands in the target data set synchronization request; When it is determined that the type of the first type of tag of the data to be operated by the synchronization operation command is an unsynchronized tag, deleting the unsynchronized tag of the data to be operated by the synchronization operation command from the first database or modifying the unsynchronized tag of the data to be operated by the synchronization operation command to a synchronized tag; When it is determined that the type of the first type of tag of the data to be operated by a synchronization operation command is a deletion tag, the data to be operated by the synchronization operation command and its deletion tag are deleted from the first database.

6. The method according to claim 5, characterized in that The method further comprises: Upon receiving a fourth reply message sent by the second device in response to the target data set synchronization request or upon not receiving the third reply message sent by the second device within a preset second time period, resending the target data set synchronization request to the second device; the starting time point of the second time period is the moment when the target data set synchronization request is issued; the fourth reply message is used to indicate that the second device has failed to execute at least one synchronization operation command in the target data set.

7. The method according to claim 1 or 4, characterized in that: The step of generating a synchronization request according to the first-type tag and the data identified by the first-type tag, and sending the synchronization request to the second device when the working state of the second device is detected to be restored to normal, includes: When it is monitored that the working state of the second device returns to normal, generating a synchronization request according to the first type of tags and the data identified by the first type of tags; Determining the sending condition of the synchronization request according to a preset synchronization strategy; the synchronization strategy is determined according to the service characteristics of the target service, the target service is a service that uses the first database for data storage; the service characteristics are obtained by performing statistical analysis on the target service according to a preset statistical analysis dimension; When it is detected that the sending condition is met, the synchronization request is sent to the second device.

8. The method according to claim 7, characterized in that The method further comprises: generating a synchronization request according to the first type of tags and the data identified by the first type of tags when the working state of the second device is detected to be restored to normal, and sending the synchronization request to the second device, further comprising: Obtaining a maximum throughput of the first database, a historical peak throughput of the first database, and a resource pressure amount of the first device; Determining a synchronization rate according to the maximum throughput, the historical peak throughput, and the resource pressure amount; According to the synchronization rate, an outbound flow rate of sending the synchronization request to the second device is controlled.

9. The method according to claim 1, characterized in that: The first database is a relational database, and a label field is preset in a data table in the first database; When the working state of the second device is detected to be abnormal, adding a preset first-type label corresponding to the target data generated after the first moment in the first database includes: When it is monitored that the working state of the second device is abnormal, for the target data generated after the first moment in the data table in the first database, a preset first type label is added under the label field in the tuple where the target data is located.

10. A method for synchronizing master-slave database data, characterized in that: Applied to a second device, a second database is deployed in the second device; the method includes: When receiving a synchronization request sent by a first device on which a first database is deployed, parsing a synchronization operation command from the synchronization request; the second database and the first database operate based on a dual-write mechanism; Execute the synchronization operation command to synchronize the data identified by the first type of label in the first database with the second database; the first type of label is a preset first type of label corresponding to the target data generated after the first moment in the first database when the first device monitors that the working state of the second device is abnormal; the first moment is the moment when the first device monitors that the working state of the second device changes from normal to abnormal; the target data includes at least one of the following: newly added data, modified data, and data to be deleted that is requested to be deleted but is aborted; the first type of label includes: an unsynchronized label and a deleted label; the newly added data and the modified data both correspond to the unsynchronized label, and the data to be deleted corresponds to the deleted label.

11. The method according to claim 10, characterized in that The method further comprises: In a case where the synchronization operation command is executed successfully, a first reply message is sent to the first device to indicate that the synchronization operation command is executed successfully.

12. The method according to claim 11, characterized in that The method further comprises: In the event that the execution of the synchronization operation command fails, a second reply message is sent to the first device to indicate that the execution of the synchronization operation command fails.

13. The method according to claim 10, characterized in that The step of, in a case where a synchronization request sent by a first device on which a first database is deployed is received, parsing a synchronization operation command from the synchronization request includes: When receiving a target data set synchronization request sent by a first device deployed with a first database, parse all synchronization operation commands from the target data set synchronization request; The executing the synchronization operation command to synchronize the data in the first database identified by the first type of tags in the second database includes: All synchronization operation commands parsed from the target data set synchronization request are executed in parallel to synchronize the target data set in the second database; the target data set is any data set in at least one data set obtained by dividing the data identified by the first category tag by the first device according to a preset division standard.

14. The method according to claim 13, characterized in that The method further comprises: In a case where all synchronization operation commands parsed from the target dataset synchronization request are successfully executed, a third reply message is sent to the first device to indicate that all synchronization operation commands in the target dataset synchronization request are successfully executed.

15. The method according to claim 14, characterized in that The method further comprises: In a case where at least one synchronization operation command parsed from the target dataset synchronization request fails to execute, a fourth reply message indicating that at least one synchronization operation command in the target dataset synchronization request fails to execute is sent to the first device.

Images