A method and system for synchronizing data from a database to a search engine
By dividing database records into physical logs and logical logs, sorting them based on node sequences, and generating a replica cluster database, the problem of long response time in database-to-search engine data synchronization is solved, achieving efficient and high-precision data synchronization.
Patent Information
- Application Number
- CN202511404673.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-29
- Publication Date
- 2025-12-02
- Estimated Expiration
- 2045-09-29
AI Technical Summary
In existing technologies, the data synchronization process from database to search engine results in a long search response time, making it impossible to retrieve relevant content efficiently and accurately.
The database records are divided into physical logs and logical logs. Logical replication based on rows and statements is processed separately, sorted by node sequence, and a replica cluster database is generated. Updates are tracked in real time and written to the cloud center, which is then linked to the search engine.
It improved the search speed and retrieval accuracy of the search engine and achieved efficient data synchronization between the database and the search engine.
Smart Images

Figure CN120873090B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a method and system for synchronizing data from a database to a search engine. Background Technology
[0002] With the development of data and information, and the emergence of a large number of local-first multi-user collaborative applications in recent years, users expect that operations performed locally can take effect immediately on the end device without the need for a long response time from the backend server.
[0003] Because databases store a large amount of data, data searches need to be performed using search engines. However, the large amount of data in databases causes search engines to have long response times.
[0004] In existing technologies, searching databases using powerful search engines can only reduce retrieval time to a limited extent, but it cannot retrieve relevant content efficiently and with high accuracy. Summary of the Invention
[0005] Therefore, the purpose of this invention is to provide a data synchronization method and system from a database to a search engine, so as to solve the shortcomings of the prior art.
[0006] In a first aspect, the present invention provides a method for synchronizing data from a database to a search engine, the method comprising:
[0007] The database records are divided into physical logs and logical logs based on their different contents.
[0008] The physical logs in the database are replicated based on row logic, and the logical logs in the database are replicated based on statement logic.
[0009] Two sets of node sequences, each containing several nodes, are established. The replicated physical logs and replicated logical logs are loaded into the two sets of node sequences respectively. The replicated physical logs and replicated logical logs are sorted in the two sets of node sequences respectively using physical timestamps.
[0010] A replica cluster database is generated based on the two node sequences, the physical logs replicated on the two sets of node sequences, and the logical logs replicated on the two sets of node sequences. The updated content in the database is tracked in real time, and the updated content is written into the replica cluster database based on the two node sequences.
[0011] The replica cluster database is written to the cloud center multiple times to synchronize the content of the database to the cloud center, and the search engine is linked to the cloud center.
[0012] Compared with the prior art, the beneficial effects of the present invention are: by loading the replicated logical logs and replicated physical logs into the two-node sequence, and sorting the replicated logical logs and physical logs on the two-node sequence by timestamp, the generated replica cluster database can obtain the data from the original database in an orderly manner, and the content of the replica cluster database can be updated in real time by tracking. By writing to the cloud center, the search speed and retrieval accuracy of the search engine are greatly improved when searching for content.
[0013] Furthermore, the step of dividing the content of the database records into physical logs and logical logs based on their different contents includes:
[0014] Obtain the underlying physical changes of the stored data in the database on the storage medium, and obtain the data change logic of the operation or transaction changes in the database;
[0015] Based on the underlying physical changes of the stored data, the associated data in the database is divided into physical logs;
[0016] Based on the data change logic of the operation or transaction change, the related data in the database is divided into logical logs.
[0017] Furthermore, the steps of logically replicating the physical logs in the database based on rows and logically replicating the logical logs in the database based on statements include:
[0018] Read the data page identifier and byte-level change location from the physical log to locate the data area, and copy the physical log based on the data area;
[0019] The logical log is copied and read based on the operation-related data and target identifier in the statement logic.
[0020] Furthermore, the step of establishing two sets of node sequences, each containing several nodes, loading the replicated physical log and the replicated logical log into the two sets of node sequences respectively, and sorting the replicated physical log and the replicated logical log in the two sets of node sequences respectively using physical timestamps includes:
[0021] Establish the corresponding first node sequence and second node sequence based on the physical log processing cluster and the logical log processing cluster, respectively.
[0022] The replicated physical log and the replicated logical log are cached in two sets of buffers respectively, and the replicated physical log and the replicated logical log are loaded into the first node sequence and the second node sequence respectively from the two sets of buffers.
[0023] Timestamps are injected into the first node sequence and the second node sequence, and the replicated physical logs on the first node sequence and the replicated logical logs on the second node sequence are sorted according to the timestamps.
[0024] Furthermore, the step of generating a replica cluster database based on the two node sequences and the replicated physical logs and replicated logical logs on the two sets of node sequences includes:
[0025] Initialize the physical log and the logical log copied from the two node sequences;
[0026] The system queries and decomposes the replicated physical logs and replicated logical logs on the two node sequences, and sends and merges the query and decomposition results in sequence to obtain the replica cluster database.
[0027] Secondly, the present invention also provides a data synchronization system from a database to a search engine, the system comprising:
[0028] The partitioning module is used to partition the content of the database records into physical logs and logical logs based on their different contents.
[0029] The replication module is used to replicate the physical logs in the database based on row logic and to replicate the logical logs in the database based on statement logic.
[0030] A module is established to create two sets of node sequences, each containing several nodes. The copied physical logs and copied logical logs are loaded into the two node sequences respectively, and the copied physical logs and copied logical logs are sorted in the two sets of node sequences respectively by physical timestamps.
[0031] The generation module is used to generate a replica cluster database based on the two node sequences, the physical logs replicated on the two sets of node sequences, and the logical logs replicated on the two sets of node sequences, and to track the updated content in the database in real time, and to write the updated content into the replica cluster database based on the two node sequences.
[0032] The write module is used to write the replica cluster database to the cloud center multiple times in order to synchronize the content of the database to the cloud center and link the search engine to the cloud center.
[0033] Furthermore, the partitioning module includes:
[0034] The acquisition unit is used to acquire the underlying physical changes of the stored data in the database on the storage medium, and to acquire the data change logic of the operation or transaction changes in the database.
[0035] The first partitioning unit is used to partition the associated data in the database into physical logs based on the underlying physical changes of the stored data.
[0036] The second partitioning unit is used to partition the related data in the database into logical logs based on the data change logic of the operation or transaction change.
[0037] Furthermore, the replication module includes:
[0038] The first reading unit is used to read the data page identifier and byte-level change position in the physical log to locate the data area, and copy the physical log based on the data area;
[0039] The second reading unit is used to copy and read the logical log based on the operation-related data and target identifier in the statement logic.
[0040] Thirdly, the present invention also provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for synchronizing data from a database to a search engine.
[0041] Fourthly, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described data synchronization method from database to search engine. Attached Figure Description
[0042] Figure 1 This is a flowchart of the data synchronization method from database to search engine in the first embodiment of the present invention;
[0043] Figure 2 This is a structural block diagram of the database-to-search engine data synchronization system in the second embodiment of the present invention;
[0044] Figure 3 This is a structural block diagram of a computer device according to the third embodiment of the present invention.
[0045] Explanation of key component symbols:
[0046] 11. Divide into modules; 12. Copy modules; 13. Create modules; 14. Generate modules; 15. Write to modules;
[0047] 10. Memory; 20. Processor; 30. Computer program.
[0048] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation
[0049] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
[0050] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.
[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0052] Example 1
[0053] Please refer to the figure, which illustrates a database-to-search engine data synchronization method according to the first embodiment of the present invention. The method includes steps S1 to S5:
[0054] S1, the content of the database records is divided into physical logs and logical logs according to the different contents of the database records;
[0055] Specifically, step S1 includes steps S11 to S13:
[0056] S11, obtain the underlying physical changes of the stored data in the database on the storage medium, and obtain the data change logic of the operation or transaction change in the database;
[0057] S12, based on the underlying physical changes of the stored data, the associated data in the database is divided into physical logs;
[0058] S13, based on the data change logic of the operation or transaction change, the related data in the database is divided into logical logs;
[0059] Understandably, physical logs record data page identifiers, byte-level change locations, the original physical byte sequence before the data change (used to support rollback operations and UNDO recovery), and the new physical byte sequence after the data change (used to support redo operations and REDO recovery). Logical logs record operation types, target representations, and operation-related data. Since physical logs refer to the underlying physical changes of stored data on the storage medium, and logical logs refer to the data change logic recorded by transactions or triggered by transactions, the data content in the database is divided into physical logs and logical logs based on their content.
[0060] S2, replicate the physical logs in the database based on row logic, and replicate the logical logs in the database based on statement logic;
[0061] Specifically, step S2 includes steps S21 to S22:
[0062] S21, Read the data page identifier and byte-level change location in the physical log to locate the data area, and copy the physical log based on the data area;
[0063] S22, Copy and read the logical log based on the operation-related data and target identifier in the statement logic;
[0064] Understandably, the replication engine reads the data page identifiers and byte-level change locations in the physical log to locate the specific data area where the change occurred. By parsing the afterimages in the physical log and combining them with the metadata of the target table, the underlying byte changes are reconstructed into complete data to obtain the replicated physical log. The replication engine reads the original operation type, target identifier, and operation-related data recorded in the logical log and configures statement-based replication to obtain the replicated logical log.
[0065] S3, establish two sets of node sequences including several nodes, load the copied physical log and the copied logical log into the two sets of node sequences respectively, and sort the copied physical log and the copied logical log in the two sets of node sequences respectively by physical timestamp;
[0066] Specifically, step S3 includes steps S31 to S33:
[0067] S31, establish the corresponding first node sequence and second node sequence according to the physical log processing cluster and the logical log processing cluster respectively;
[0068] S32, based on two sets of buffers, the replicated physical log and the replicated logical log are cached respectively, and the replicated physical log and the replicated logical log are loaded from the two sets of buffers into the first node sequence and the second node sequence respectively;
[0069] S33, inject timestamps into the first node sequence and the second node sequence, and sort the replicated physical logs on the first node sequence and the replicated logical logs on the second node sequence according to the timestamps;
[0070] Understandably, targeted distribution is performed based on log type to establish two sets of functional transformation sequences, specifically a first node sequence and a second node sequence. The first node sequence uses the same or compatible physical storage engine as the source database, which can directly identify and apply data page changes. The second node sequence uses an SQL parsing and execution engine, which can understand and replay DML operations. Through a targeted loading mechanism, the replicated physical logs and replicated logical logs are loaded into the first node sequence and the second node sequence, respectively.
[0071] It's worth noting that by using the aforementioned timestamps to establish global order within and between the two node sequences, each node sequence does not immediately apply the loaded log records upon receiving them; instead, it temporarily stores them in an ordered buffer. This buffer sorts all records in ascending order based on the physical timestamps attached to the records.
[0072] S4. A replica cluster database is generated based on the two node sequences, the physical logs replicated on the two sets of node sequences, and the logical logs replicated on the two sets of node sequences. The updated content in the database is tracked in real time, and the updated content is written into the replica cluster database based on the two node sequences.
[0073] Specifically, step S4 includes steps S41 to S42:
[0074] S41, initialize the replicated physical log and replicated logical log on the two node sequences;
[0075] S42, query and decompose the replicated physical logs and replicated logical logs on the two node sequences, and send and merge the query decomposition results in sequence to obtain the replica cluster database;
[0076] Understandably, the data server initializes the replicated physical logs and replicated logical logs to ensure consistency between the physical and logical logs and the source database. The physical state based on data pages provided by the physical logs in the first node sequence and the logical state of rows / columns in the logical logs in the second node sequence together constitute the replica cluster database.
[0077] S5, write the replica cluster database to the cloud center multiple times to synchronize the content of the database to the cloud center, and link the search engine to the cloud center;
[0078] Understandably, by linking the replica cluster database and search engine through the cloud center, the search speed and retrieval accuracy of the search engine can be greatly improved.
[0079] In summary, the database-to-search engine data synchronization method in the above embodiments of the present invention loads the replicated logical logs and replicated physical logs onto a two-node sequence, and sorts the replicated logical logs and physical logs on the two-node sequence using timestamps. This allows the generated replica cluster database to obtain data from the original database in an orderly manner, and the content of the replica cluster database can be updated in real time by tracking. By writing to the cloud center, the search speed and retrieval accuracy of the search engine are greatly improved when searching for content.
[0080] Example 2
[0081] Please see Figure 2 The figure shows a database-to-search engine data synchronization system according to a second embodiment of the present invention, the system comprising:
[0082] The partitioning module 11 is used to partition the content of the database records into physical logs and logical logs according to the different contents of the database records;
[0083] The replication module 12 is used to replicate the physical logs in the database based on row logic and to replicate the logical logs in the database based on statement logic.
[0084] Module 13 is used to establish two sets of node sequences, each containing several nodes. The copied physical logs and the copied logical logs are loaded into the two sets of node sequences respectively, and the copied physical logs and the copied logical logs are sorted in the two sets of node sequences respectively by physical timestamps.
[0085] The generation module 14 is used to generate a replica cluster database based on the two node sequences, the physical logs replicated on the two sets of node sequences, and the logical logs replicated on the two sets of node sequences, and to track the updated content in the database in real time, and to write the updated content into the replica cluster database based on the two node sequences.
[0086] The write module 15 is used to write the replica cluster database to the cloud center multiple times in order to synchronize the content of the database to the cloud center and link the search engine to the cloud center.
[0087] In some optional embodiments, the partitioning module 11 includes:
[0088] The acquisition unit is used to acquire the underlying physical changes of the stored data in the database on the storage medium, and to acquire the data change logic of the operation or transaction changes in the database.
[0089] The first partitioning unit is used to partition the associated data in the database into physical logs based on the underlying physical changes of the stored data.
[0090] The second partitioning unit is used to partition the related data in the database into logical logs based on the data change logic of the operation or transaction change.
[0091] In some alternative embodiments, the copying module 12 includes:
[0092] The first reading unit is used to read the data page identifier and byte-level change position in the physical log to locate the data area, and copy the physical log based on the data area;
[0093] The second reading unit is used to copy and read the logical log based on the operation-related data and target identifier in the statement logic.
[0094] In some alternative embodiments, the establishment module 13 includes:
[0095] A creation unit is used to create the corresponding first node sequence and second node sequence based on the physical log processing cluster and the logical log processing cluster, respectively.
[0096] The loading unit is used to cache the copied physical log and the copied logical log respectively based on two sets of buffers, and load the copied physical log and the copied logical log respectively into the first node sequence and the second node sequence from the two sets of buffers;
[0097] An injection unit is configured to inject timestamps into the first node sequence and the second node sequence, and sort the replicated physical logs on the first node sequence and the replicated logical logs on the second node sequence according to the timestamps.
[0098] In some alternative embodiments, the generation module 14 includes:
[0099] An initialization unit is used to initialize the replicated physical logs and replicated logical logs on the two node sequences.
[0100] The query decomposition unit is used to query and decompose the replicated physical logs and replicated logical logs on the two node sequences, and send and merge the query decomposition results in sequence to obtain the replica cluster database.
[0101] The functions or operation steps implemented by the above modules and units are largely the same as those in the above method embodiments, and will not be repeated here.
[0102] The data synchronization system from database to search engine provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment.
[0103] Example 3
[0104] The present invention also proposes a computer device, please refer to [link / reference]. Figure 3 The diagram shows a computer device according to a third embodiment of the present invention, including a memory 10, a processor 20, and a computer program 30 stored on the memory 10 and executable on the processor 20. When the processor 20 executes the computer program 30, it implements the above-described data synchronization method from the database to the search engine.
[0105] The memory 10 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10 may be an internal storage unit of a computer device, such as the hard disk of that computer device. In other embodiments, the memory 10 may be an external storage device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Furthermore, the memory 10 may include both internal and external storage units of the computer device. The memory 10 can be used not only to store application software and various types of data installed on the computer device, but also to temporarily store data that has been output or will be output.
[0106] In some embodiments, the processor 20 may be an electronic control unit (ECU), a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, used to run program code stored in the memory 10 or process data, such as executing access restriction programs.
[0107] It should be pointed out that, Figure 3 The structure shown does not constitute a limitation on the computer device. In other embodiments, the computer device may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0108] This invention also proposes a readable storage medium storing a computer program that, when executed by a processor, implements the data synchronization method from the database to the search engine as described above.
[0109] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequential list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.
[0110] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.
[0111] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0112] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0113] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. A method for synchronizing data from a database to a search engine, characterized in that, The method includes: The database records are divided into physical logs and logical logs based on their different contents. The physical logs in the database are replicated based on row logic, and the logical logs in the database are replicated based on statement logic. Two sets of node sequences, each containing several nodes, are established. The replicated physical logs and replicated logical logs are loaded into the two sets of node sequences respectively. The replicated physical logs and replicated logical logs are then sorted in the two sets of node sequences using physical timestamps. This step specifically includes: Establish the corresponding first node sequence and second node sequence based on the physical log processing cluster and the logical log processing cluster, respectively. The replicated physical log and the replicated logical log are cached in two sets of buffers respectively, and the replicated physical log and the replicated logical log are loaded into the first node sequence and the second node sequence respectively from the two sets of buffers. Inject timestamps into the first node sequence and the second node sequence, and sort the replicated physical logs on the first node sequence and the replicated logical logs on the second node sequence according to the timestamps; A replica cluster database is generated based on the two sets of node sequences, the replicated physical logs on the two sets of node sequences, and the replicated logical logs. Updates in the database are tracked in real time, and the updated content is written to the replica cluster database based on the two sets of node sequences. This step specifically includes: Initialize the physical logs and the logical logs replicated on the two sets of node sequences; The system queries and decomposes the replicated physical logs and replicated logical logs on the two sets of node sequences, and sends and merges the query and decomposition results in sequence to obtain the replica cluster database. The replica cluster database is written to the cloud center multiple times to synchronize the content of the database to the cloud center, and the search engine is linked to the cloud center.
2. The data synchronization method from database to search engine according to claim 1, characterized in that, The step of dividing the content of the database records into physical logs and logical logs based on their different contents includes: Obtain the underlying physical changes of the stored data in the database on the storage medium, and obtain the data change logic of the operation or transaction changes in the database; Based on the underlying physical changes of the stored data, the associated data in the database is divided into physical logs; Based on the data change logic of the operation or transaction change, the related data in the database is divided into logical logs.
3. The data synchronization method from database to search engine according to claim 1, characterized in that, The steps of performing row-based logical replication of the physical logs in the database and statement-based logical replication of the logical logs in the database include: Read the data page identifier and byte-level change location from the physical log to locate the data area, and copy the physical log based on the data area; The logical log is copied and read based on the operation-related data and target identifier in the statement logic.
4. A data synchronization system from a database to a search engine, characterized in that, The system includes: The partitioning module is used to partition the content of the database records into physical logs and logical logs based on their different contents. The replication module is used to replicate the physical logs in the database based on row logic and to replicate the logical logs in the database based on statement logic. The module is used to establish two sets of node sequences, each containing several nodes. The copied physical logs and copied logical logs are loaded into the two sets of node sequences respectively, and the copied physical logs and copied logical logs are sorted in the two sets of node sequences respectively by physical timestamps. The establishment module includes: A creation unit is used to create the corresponding first node sequence and second node sequence based on the physical log processing cluster and the logical log processing cluster, respectively. The loading unit is used to cache the copied physical log and the copied logical log respectively based on two sets of buffers, and load the copied physical log and the copied logical log respectively into the first node sequence and the second node sequence from the two sets of buffers; An injection unit is used to inject timestamps into the first node sequence and the second node sequence, and to sort the replicated physical logs on the first node sequence and the replicated logical logs on the second node sequence according to the timestamps. The generation module is used to generate a replica cluster database based on the two sets of node sequences, the physical logs replicated on the two sets of node sequences, and the logical logs replicated on the two sets of node sequences, and to track the updated content in the database in real time, and to write the updated content into the replica cluster database based on the two sets of node sequences. The generation module includes: An initialization unit is used to initialize the replicated physical logs and replicated logical logs on the two sets of node sequences; The query decomposition unit is used to query and decompose the replicated physical logs and replicated logical logs on the two sets of node sequences, and send and merge the query decomposition results in sequence to obtain the replica cluster database. The write module is used to write the replica cluster database to the cloud center multiple times in order to synchronize the content of the database to the cloud center and link the search engine to the cloud center.
5. The database-to-search engine data synchronization system according to claim 4, characterized in that, The partitioning module includes: The acquisition unit is used to acquire the underlying physical changes of the stored data in the database on the storage medium, and to acquire the data change logic of the operation or transaction changes in the database. The first partitioning unit is used to partition the associated data in the database into physical logs based on the underlying physical changes of the stored data. The second partitioning unit is used to partition the related data in the database into logical logs based on the data change logic of the operation or transaction change.
6. The database-to-search engine data synchronization system according to claim 4, characterized in that, The replication module includes: The first reading unit is used to read the data page identifier and byte-level change position in the physical log to locate the data area, and copy the physical log based on the data area; The second reading unit is used to copy and read the logical log based on the operation-related data and target identifier in the statement logic.
7. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the data synchronization method from the database to the search engine as described in any one of claims 1 to 4.
8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the database-to-search engine data synchronization method as described in any one of claims 1 to 4.
Citation Information
Patent Citations
Data synchronization method and device and computer readable storage medium
CN113987078A
Two-stage lock-free parallel log playback method and device
CN117009361A