Database-based data synchronization method, device, electronic device, and storage medium

Abstract

The embodiments of the present application provide a database-based data synchronization method, device, electronic device, and storage medium, wherein the method includes: marking the interface class whose execution data needs to be synchronized; storing the class name corresponding to the marked interface class in a cache class; obtaining the write interface class name corresponding to the write method; if the write interface class name exists in the cache class, synchronizing the execution data to obtain a synchronization result. Implementing the embodiments of the present application allows database query data and full-text search data to be synchronized, avoiding the problem of excessive pressure during the database query process and simplifying the writing and differentiation of fields in the database.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a database-based data synchronization method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Business development often requires querying data in a MySQL database. However, MySQL is not good at full-text search and data analysis. When MySQL query and full-text search are connected at the same time, there are major problems with database-based data synchronization between the two.

[0003] Traditional solutions rely on passive synchronization or scheduled data queries in MySQL. However, frequent polling of the MySQL interface to retrieve newly written data places significant pressure on the MySQL service. In extreme cases, this can cause serious online issues. Furthermore, different database tables have different fields, making it difficult to unify them. This makes it difficult to distinguish between unnecessary data writes. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a database-based data synchronization method, device, electronic device and computer-readable storage medium, so that database query data and full-text retrieval data can be synchronized, avoiding the problem of excessive pressure during the database query process and simplifying the writing and differentiation of fields in the database.

[0005] In a first aspect, an embodiment of the present application provides a database-based data synchronization method, the method comprising:

[0006] Mark the interface class that requires synchronization of execution data;

[0007] Store the class name corresponding to the marked interface class in the cache class;

[0008] Get the write interface class name corresponding to the write method;

[0009] If the write interface class name exists in the cache class, the execution data is synchronized to obtain a synchronization result.

[0010] In the above implementation process, the class name corresponding to the interface class of the execution data and the write interface class name of the write method are stored in the cache class to ensure that the cache class contains the corresponding information of the execution data, so that the execution data can be actively synchronized, avoiding the problem of excessive pressure during the database query process and simplifying the writing and differentiation of fields in the database.

[0011] Furthermore, before the step of synchronizing the execution data, the method further includes:

[0012] Determine whether the write interface class name exists in the cache class;

[0013] If so, query the cache class to see whether the value corresponding to the write interface class name exists.

[0014] In the above implementation process, it is determined whether the written interface class name exists in the cache class. If it exists in the cache class, it is further determined whether the value corresponding to the written interface class name exists in the cache class to ensure that the class name corresponding to the interface class and the value corresponding to the written interface class name exist in the cache class at the same time.

[0015] Furthermore, after the step of querying the cache class whether the value corresponding to the write interface class name exists, the method further includes:

[0016] If there is no value corresponding to the write interface class name, store the write interface class name in the cache class;

[0017] If a value corresponding to the write interface class name exists, the value corresponding to the write interface class name is modified, and the modified value corresponding to the write interface class name is stored in the cache class.

[0018] In the above implementation process, if there is no value corresponding to the written interface class name, the value corresponding to the written interface class name is directly stored in the cache class. If there is a value corresponding to the written interface class name, the value corresponding to the written interface class name is modified and then stored in the cache class to ensure the accuracy of the value corresponding to the written interface class name and improve the synchronization efficiency of subsequent execution data.

[0019] Furthermore, the step of synchronizing the execution data to obtain a synchronization result includes:

[0020] Get query request;

[0021] Determine whether the interface name corresponding to the query request exists in the cache class;

[0022] If so, a synchronization result is obtained according to the execution data corresponding to the query request.

[0023] In the above implementation process, it is determined whether the interface name corresponding to the query request exists in the cache class, thereby obtaining a synchronization result, thereby ensuring the efficiency and accuracy of executing data synchronization.

[0024] In a second aspect, an embodiment of the present application further provides a database-based data synchronization device, the device comprising:

[0025] Marking module, used to mark the interface class whose execution data needs to be synchronized;

[0026] A storage module, used to store the class name corresponding to the marked interface class into a cache class;

[0027] Get module, used to get the write interface class name corresponding to the write method;

[0028] The synchronization module is used to synchronize the execution data if the write interface class name exists in the cache class to obtain a synchronization result.

[0029] In the above implementation process, the class name corresponding to the interface class of the execution data and the write interface class name of the write method are stored in the cache class to ensure that the cache class contains the corresponding information of the execution data, so that the execution data can be actively synchronized, avoiding the problem of excessive pressure during the database query process and simplifying the writing and differentiation of fields in the database.

[0030] Furthermore, the device further includes a judgment module, configured to:

[0031] Determine whether the write interface class name exists in the cache class;

[0032] If so, query the cache class to see whether the value corresponding to the write interface class name exists.

[0033] In the above implementation process, it is determined whether the written interface class name exists in the cache class. If it exists in the cache class, it is further determined whether the value corresponding to the written interface class name exists in the cache class to ensure that the class name corresponding to the interface class and the value corresponding to the written interface class name exist in the cache class at the same time.

[0034] Furthermore, the judgment module is further configured to:

[0035] If there is no value corresponding to the write interface class name, store the write interface class name in the cache class;

[0036] If a value corresponding to the write interface class name exists, the value corresponding to the write interface class name is modified, and the modified value corresponding to the write interface class name is stored in the cache class.

[0037] In the above implementation process, if there is no value corresponding to the written interface class name, the value corresponding to the written interface class name is directly stored in the cache class. If there is a value corresponding to the written interface class name, the value corresponding to the written interface class name is modified and then stored in the cache class to ensure the accuracy of the value corresponding to the written interface class name and improve the synchronization efficiency of subsequent execution data.

[0038] Furthermore, the synchronization module is further configured to:

[0039] Get query request;

[0040] Determine whether the interface name corresponding to the query request exists in the cache class;

[0041] If so, a synchronization result is obtained according to the execution data corresponding to the query request.

[0042] In the above implementation process, it is determined whether the interface name corresponding to the query request exists in the cache class, thereby obtaining a synchronization result, thereby ensuring the efficiency and accuracy of executing data synchronization.

[0043] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects.

[0045] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method as described in any one of the first aspects.

[0046] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by practicing the above-mentioned technology of the present disclosure.

[0047] It can be implemented according to the contents of the specification. The following is a detailed description of the preferred embodiments of the present application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A flowchart of a database-based data synchronization method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of the structure of a database-based data synchronization device provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of the structural composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0053] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0054] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0055] Example 1

[0056] Figure 1 This is a flow chart of a database-based data synchronization method provided by an embodiment of the present application. Figure 1 As shown, the method includes:

[0057] S1, marking the interface class that requires synchronization of execution data;

[0058] S2, store the class name corresponding to the marked interface class into the cache class;

[0059] S3, obtain the write interface class name corresponding to the write method;

[0060] S4: If the written interface class name exists in the cache class, the execution data is synchronized to obtain the synchronization result.

[0061] In the above implementation process, the class name corresponding to the interface class of the execution data and the write interface class name of the write method are stored in the cache class to ensure that the cache class contains the corresponding information of the execution data, so that the execution data can be actively synchronized, avoiding the problem of excessive pressure during the database query process and simplifying the writing and differentiation of fields in the database.

[0062] The embodiments of the present application can be applied to databases that require full-text search and data analysis. The functions and services of a search engine can be used to perform search and data analysis in the database, and the database data and the search engine can be synchronized to improve efficiency, stability, and performance. For example, the database can be a MySQL database and the search engine can be Solr.

[0063] In S1, the interface class that needs to be synchronized is defined. For example, an @EsSycn annotation class can be provided to mark the DAO interface class that needs to be synchronized.

[0064] In S2, when the database service starts, all interface classes under the DAO package are scanned, and all class names marked with the @EsSycn annotation are stored in the custom cache class EsSyncCache.

[0065] When a user writes data to MySQL, intercept the write request and obtain the write interface class name corresponding to the write method.

[0066] Furthermore, before the step of synchronizing the execution data, the method further includes:

[0067] Determine whether the written interface class name exists in the cache class;

[0068] If so, query the cache class to see whether the value corresponding to the written interface class name exists.

[0069] Check whether the write interface class name exists in the cache class. If it does, it indicates that MySQL writing is complete and data synchronization needs to be triggered. If it does not exist, it indicates that MySQL writing is complete and data synchronization does not need to be triggered.

[0070] In the above implementation process, it is determined whether the written interface class name exists in the cache class. If it exists in the cache class, it is further determined whether the value corresponding to the written interface class name exists in the cache class to ensure that the class name corresponding to the interface class and the value corresponding to the written interface class name exist in the cache class at the same time.

[0071] Furthermore, after the step of querying whether the value corresponding to the written interface class name exists in the cache class, the method further includes:

[0072] If there is no value corresponding to the written interface class name, the written interface class name is stored in the cache class;

[0073] If a value corresponding to the write interface class name exists, the value corresponding to the write interface class name is modified, and the modified value corresponding to the write interface class name is stored in the cache class.

[0074] Check whether the value corresponding to the written interface class name exists in the cache class. If it does not exist, store the written interface class name as the key and 1 as the value in the cache class; if it exists, add 1 to the value corresponding to the written interface class name and then store it in the cache class again.

[0075] In the above implementation process, if there is no value corresponding to the written interface class name, the value corresponding to the written interface class name is directly stored in the cache class. If there is a value corresponding to the written interface class name, the value corresponding to the written interface class name is modified and then stored in the cache class to ensure the accuracy of the value corresponding to the written interface class name and improve the synchronization efficiency of subsequent execution data.

[0076] Furthermore, the step of synchronizing the execution data to obtain a synchronization result includes:

[0077] Get query request;

[0078] Determine whether the interface name corresponding to the query request exists in the cache class;

[0079] If so, obtain the synchronization result according to the execution data corresponding to the query request.

[0080] In the above implementation process, it is determined whether the interface name corresponding to the query request exists in the cache class, thereby obtaining the synchronization result, thereby ensuring the efficiency and accuracy of data synchronization.

[0081] Call the Solr interface for synchronization. After synchronization is complete, query the value corresponding to the write interface class name in the cache class. If the value is 1, remove the write interface class name from the cache class, indicating that there is no synchronization operation currently being performed on this write interface class. If the value is not 1, decrement the value by 1 and store it in the cache.

[0082] Intercepts query requests and determines whether the interface name corresponding to the query interface in the query request exists in the cache class. If it does, it indicates that the execution data queried by the current interface is being synchronized. At this time, the synchronization result is returned to the user to inform the user that data synchronization is in progress. If it does not exist, it indicates that the current query interface is not being synchronized, and the Solr query request is released.

[0083] In this embodiment, Solr's passive synchronization is changed to active synchronization. During the intervals between data synchronizations, the user can be called and the synchronization results can be sent to the user, informing the user of the status and results of the data synchronization. This prevents users from obtaining incorrect data in Solr. By providing annotation classes to users, users can mark the abstract methods that need to trigger synchronization, thus avoiding frequent data synchronization triggering.

[0084] Switching from passive data synchronization to active synchronization can reduce the frequency of MySQL-to-Solr data synchronization, alleviating the pressure on MySQL. This also avoids query errors caused by brief inconsistencies between MySQL and Solr data during data synchronization.

[0085] Example 2

[0086] In order to execute the method corresponding to the above embodiment 1 to achieve the corresponding functions and technical effects, a data synchronization device based on a database is provided below, such as Figure 2 As shown, the device includes:

[0087] Marking module 1, used to mark the interface class that needs to synchronize execution data;

[0088] Storage module 2 is used to store the class name corresponding to the marked interface class into the cache class;

[0089] Acquisition module 3, used to obtain the write interface class name corresponding to the write method;

[0090] The synchronization module 4 is used to synchronize the execution data if the written interface class name exists in the cache class to obtain a synchronization result.

[0091] In the above implementation process, the class name corresponding to the interface class of the execution data and the write interface class name of the write method are stored in the cache class to ensure that the cache class contains the corresponding information of the execution data, so that the execution data can be actively synchronized, avoiding the problem of excessive pressure during the database query process and simplifying the writing and differentiation of fields in the database.

[0092] Furthermore, the device also includes a judgment module, which is used to:

[0093] Determine whether the written interface class name exists in the cache class;

[0094] If so, query the cache class to see whether the value corresponding to the written interface class name exists.

[0095] In the above implementation process, it is determined whether the written interface class name exists in the cache class. If it exists in the cache class, it is further determined whether the value corresponding to the written interface class name exists in the cache class to ensure that the class name corresponding to the interface class and the value corresponding to the written interface class name exist in the cache class at the same time.

[0096] Furthermore, the judgment module is also used to:

[0097] If there is no value corresponding to the written interface class name, the written interface class name is stored in the cache class;

[0098] If a value corresponding to the write interface class name exists, the value corresponding to the write interface class name is modified, and the modified value corresponding to the write interface class name is stored in the cache class.

[0099] In the above implementation process, if there is no value corresponding to the written interface class name, the value corresponding to the written interface class name is directly stored in the cache class. If there is a value corresponding to the written interface class name, the value corresponding to the written interface class name is modified and then stored in the cache class to ensure the accuracy of the value corresponding to the written interface class name and improve the synchronization efficiency of subsequent execution data.

[0100] Furthermore, the synchronization module 4 is further configured to:

[0101] Get query request;

[0102] Determine whether the interface name corresponding to the query request exists in the cache class;

[0103] If so, obtain the synchronization result according to the execution data corresponding to the query request.

[0104] In the above implementation process, it is determined whether the interface name corresponding to the query request exists in the cache class, thereby obtaining the synchronization result, thereby ensuring the efficiency and accuracy of data synchronization.

[0105] The above-mentioned database-based data synchronization device can implement the method of the above-mentioned embodiment 1. The options in the above-mentioned embodiment 1 are also applicable to this embodiment and will not be described in detail here.

[0106] The rest of the contents of the embodiments of this application can refer to the contents of the above-mentioned embodiment 1, and will not be repeated in this embodiment.

[0107] Example 3

[0108] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the database-based data synchronization method of embodiment 1.

[0109] Optionally, the above-mentioned electronic device may be a server.

[0110] See Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include a processor 31, a communication interface 32, a memory 33, and at least one communication bus 34. The communication bus 34 is used to enable direct communication between these components. The communication interface 32 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The processor 31 may be an integrated circuit chip with signal processing capabilities.

[0111] The processor 31 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor can be a microprocessor, or the processor 31 can also be any conventional processor.

[0112] The memory 33 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory 33 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 31, the device can perform the above-mentioned operations. Figure 1 The various steps involved in the method embodiment.

[0113] Optionally, the electronic device may further include a storage controller and an input / output unit. The memory 33, storage controller, processor 31, peripheral interfaces, and input / output units are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses 34. The processor 31 is configured to execute executable modules stored in the memory 33, such as software function modules or computer programs included in the device.

[0114] The input and output unit is used to provide users with the ability to create tasks and to create optional start time periods or preset execution times for the tasks to enable interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse and keyboard.

[0115] I understand. Figure 3 The structure shown is only for illustration, and the electronic device may also include Figure 3 More or fewer components than shown, or with Figure 3 Different configurations shown. Figure 3 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0116] In addition, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the database-based data synchronization method of embodiment 1.

[0117] An embodiment of the present application further provides a computer program product, which, when running on a computer, enables the computer to execute the method described in the method embodiment.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based device that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0119] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0120] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0121] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0122] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0123] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

Claims

1. A data synchronization method based on a database, characterized in that: Applied to data synchronization between a search engine and a database, the method includes: Define the interface class that needs to be synchronized, which is used to mark the interface class whose execution data needs to be synchronized; When the database service is started, all the marked interface classes are scanned, and the class names corresponding to the marked interface classes are stored in the cache class; When the user writes data to the database, intercept the write request and obtain the write interface class name corresponding to the write method; Determine whether the write interface class name exists in the cache class; If the write interface class name does not exist in the cache class, it is determined that there is no need to trigger data synchronization after the database write is completed; If the write interface class name exists in the cache class, it is determined that the database write is completed and data synchronization needs to be triggered. The execution data is synchronized to obtain a synchronization result. In the interval between executing data synchronization, the synchronization result is sent to the user to inform the user of the status and result of data synchronization, and whether the value corresponding to the write interface class name exists in the cache class is queried; If there is no value corresponding to the write interface class name, storing the write interface class name in the cache class; If a value corresponding to the write interface class name exists, the value corresponding to the write interface class name is modified, and the modified value corresponding to the write interface class name is stored in the cache class.

2. The method for data synchronization based on a database according to claim 1, characterized in that: The step of synchronizing the execution data to obtain a synchronization result includes: Get query request; Determine whether the interface name corresponding to the query request exists in the cache class; If so, a synchronization result is obtained according to the execution data corresponding to the query request.

3. A data synchronization device based on a database, characterized in that: Applicable to data synchronization between a search engine and a database, the device comprises: The marking module is used to define the interface classes that need to be synchronized and to mark the interface classes whose execution data needs to be synchronized; A storage module is used to scan all the marked interface classes when the database service is started, and store the class names corresponding to the marked interface classes into a cache class; The acquisition module is used to intercept the write request when the user writes data to the database and obtain the write interface class name corresponding to the write method; A judgment module, configured to judge whether the write interface class name exists in the cache class; if the write interface class name does not exist in the cache class, determining that data synchronization does not need to be triggered after database writing is completed; A synchronization module is configured to determine that database writing is complete and data synchronization needs to be triggered if the write interface class name exists in the cache class, synchronize the execution data to obtain a synchronization result, and send the synchronization result to the user during the interval of executing data synchronization to inform the user of the status and result of data synchronization; The judgment module is also used to query whether the value corresponding to the write interface class name exists in the cache class; if the value corresponding to the write interface class name does not exist, the write interface class name is stored in the cache class; if the value corresponding to the write interface class name exists, the value corresponding to the write interface class name is modified, and the value corresponding to the modified write interface class name is stored in the cache class.

4. The data synchronization device based on a database according to claim 3, characterized in that: The synchronization module is also used for: Get query request; Determine whether the interface name corresponding to the query request exists in the cache class; If so, a synchronization result is obtained according to the execution data corresponding to the query request.

5. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the database-based data synchronization method according to any one of claims 1 to 2.

6. A computer-readable storage medium, characterized in that The computer program is stored therein, and when the computer program is executed by a processor, the database-based data synchronization method according to any one of claims 1 to 2 is implemented.

Images