A data synchronization method and device in a SaaS scenario and a medium thereof

Abstract

The application discloses a data synchronization method and device in a SaaS scenario and a medium thereof, relates to the technical field of computers, and is used for realizing a data query system, aiming at the problem that data writing time is too long and data consistency cannot be ensured at present, providing a data synchronization method in a SaaS scenario, comprising: only writing data into a target data source; then acquiring data from the target data source, and synchronizing the data to other data sources after analyzing and converting the data into a corresponding format. Compared with the double-writing method currently used, the writing time is saved. In addition, since only one data source is written during data writing, it is not necessary to adopt the double-writing method of two-stage synchronization submission, and the problem that data consistency cannot be ensured caused by the double-writing method does not occur, the accuracy of data writing is further improved, more accurate query results are provided when a user queries, and the use experience of the user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to a data synchronization method and device in a SaaS scenario and a medium thereof. BACKGROUND

[0002] In the current Software-as-a-Service (SaaS) scenario, because the amount of data is very large, multiple data sources are usually used, and a two-phase synchronization commit scheme is usually adopted when a user writes data, that is, double writing operation is performed on multiple data sources to store the data provided by the user into different data sources.

[0003] Currently, when facing data writing of multiple data sources, a two-phase synchronization commit double writing method is usually adopted to store the data written by the user in different data sources, but this double writing method has a long data writing time, and the two-phase synchronization commit scheme cannot guarantee data consistency, resulting in inaccurate user query data results.

[0004] Therefore, the technical personnel in the field urgently need a data synchronization method in a SaaS scenario to solve the problem of long writing time and inability to guarantee data consistency when writing data. SUMMARY

[0005] The purpose of the present application is to provide a data synchronization method and device in a SaaS scenario and a medium thereof to solve the problem of long writing time and inability to guarantee data consistency when writing data.

[0006] To solve the above technical problems, the present application provides a data synchronization method in a SaaS scenario, comprising:

[0007] receiving data sent by a user end;

[0008] writing data into a target data source; wherein the target data source is a predetermined data source among multiple data sources;

[0009] obtaining data from the target data source, and synchronizing the data to other data sources after parsing and converting the data into a corresponding format.

[0010] Preferably, after synchronizing the data to other data sources after parsing and converting the data into a corresponding format, the method further comprises:

[0011] determining whether a write completion signal returned by the other data sources is received; wherein the write completion signal is a signal generated by the data source after the process of writing data into the data source is completed;

[0012] If the write completion signal is not received, the step of synchronizing the data to other data sources after parsing and converting the data into a corresponding format is returned to.

[0013] Preferably, after determining whether the write completion signal returned by the other data source is received, further comprising:

[0014] If the write completion signal is received, updating the synchronization flag field of the data in the target data source to a synchronization success state; the synchronization flag field is a flag field added when the data is written into the target data source, and the initial state thereof is a synchronization failure state.

[0015] Preferably, further comprising:

[0016] receiving a query request sent by a user end;

[0017] determining data to be queried according to the query request;

[0018] obtaining data with the synchronization flag field in a synchronization failure state from the target data source and obtaining data with the synchronization flag field in a synchronization success state from the other data source;

[0019] sending the obtained data to the user end.

[0020] Preferably, after writing the data into the target data source, further comprising:

[0021] determining whether a write completion signal returned by the target data source is received; wherein the write completion signal is a signal generated by the data source after the process of writing the data into the data source is completed;

[0022] if the write completion signal is received, returning a prompt information to prompt the user that the data writing is successful;

[0023] if the write completion signal is not received, returning to the step of writing the data into the target data source;

[0024] Correspondingly, the step of obtaining the data from the target data source and synchronizing the data parsed and converted into a corresponding format to the other data source is run in the background.

[0025] Preferably, when the data source comprises a MySQL data source, the target data source is a MySQL data source.

[0026] Preferably, after determining whether the write completion signal returned by the other data source is received, further comprising:

[0027] counting the number of times of returning to the step of synchronizing the data parsed and converted into a corresponding format to the other data source, and obtaining a count value;

[0028] when the count value exceeds a preset value, returning an alarm information.

[0029] To solve the above technical problems, the application further provides a data synchronization device in a SaaS scenario, comprising:

[0030] receiving a data sent by a user terminal;

[0031] a writing module, configured to write the data into a target data source; the target data source is a predetermined data source in a plurality of data sources;

[0032] a synchronizing module, configured to acquire the data from the target data source, and parse and convert the data into a corresponding format and synchronize the data to other data sources.

[0033] Preferably, the method further comprises:

[0034] a repeating synchronizing module, configured to determine whether a writing completion signal returned by the other data sources is received; the writing completion signal is a signal generated by the data source after a process of writing the data into the data source is completed; if the writing completion signal is not received, the step of parsing and converting the data into the corresponding format and synchronizing the data to the other data sources is returned to.

[0035] a marking module, configured to, if the writing completion signal is received, update a synchronization flag field of the data in the target data source to a synchronization success state; the synchronization flag field is a flag field added when the data is written into the target data source, and an initial state of the synchronization flag field is a synchronization failure state.

[0036] a querying module, configured to receive a query request sent by the user terminal; determine data to be queried according to the query request; acquire data with the synchronization flag field in the target data source being in the synchronization failure state, and acquire data with the synchronization flag field in the other data sources being in the synchronization success state; and send the acquired data to the user terminal.

[0037] a perception optimization module, configured to determine whether the writing completion signal returned by the target data source is received; if the writing completion signal is received, a prompt information is returned to prompt the user that the writing is successful; if the writing completion signal is not received, the step of writing the data into the target data source is returned to; and correspondingly, the step of acquiring the data from the target data source, and parsing and converting the data into the corresponding format and synchronizing the data to the other data sources is run in the background.

[0038] an alarm module, configured to count a number of times of returning to the step of parsing and converting the data into the corresponding format and synchronizing the data to the other data sources, and obtain a count value; when the count value exceeds a preset value, an alarm information is returned.

[0039] To solve the above technical problems, the application further provides a data synchronization device in a SaaS scenario, comprising:

[0040] a memory, configured to store a computer program;

[0041] A processor is configured to implement the steps of the data synchronization method in the SaaS scenario when executing a computer program.

[0042] To solve the above technical problems, the application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program.

[0043] The data synchronization method in the SaaS scenario provided by the application writes data to only one data source when a user writes data, and then synchronizes the data to other data sources, which further saves the writing time compared with the current double-writing method. Meanwhile, since the writing process is a process of writing data from the user end to the data source, it is perceived by the user, and the method saves the data writing process, and the data synchronization of other data sources is completed by the subsequent data synchronization process in an asynchronous manner, which is not perceived by the user and does not affect the user experience. In addition, since only one data source is written when writing data, the two-stage synchronization submission double-writing method is not needed, and the problem of unable to guarantee data consistency caused by the double-writing method is also avoided, which further improves the accuracy of data writing, and provides more accurate query results when the user queries, and further improves the user experience.

[0044] The data synchronization device in the SaaS scenario and the computer readable storage medium provided by the application correspond to the above method, and have the same effect. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0046] Figure 1 A flowchart of the data synchronization method in the SaaS scenario provided by the application;

[0047] Figure 2 A flowchart of another data synchronization method in the SaaS scenario provided by the application;

[0048] Figure 3 A structure diagram of the data synchronization device in the SaaS scenario provided by the application;

[0049] Figure 4 A structure diagram of another data synchronization device in the SaaS scenario provided by the application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0051] The core of the present application is to provide a data synchronization method and device in a SaaS scenario and a medium thereof.

[0052] In order to enable those skilled in the art to better understand the present application, the present application will be further described in detail below in conjunction with the drawings and specific embodiments.

[0053] At present, multiple data sources are generally used in the SaaS scenario, and a two-stage synchronization commit double-write scheme is adopted for multi-data source writing. This scheme not only takes a long time for writing, but also has long and tedious business code, and the consistency of data is difficult to guarantee under extreme conditions, so that the result data obtained by the user when searching and querying data is inaccurate, which seriously affects the user experience.

[0054] Therefore, as shown in the present application, a data synchronization method in a SaaS scenario is provided, comprising: Figure 1

[0055] S11: receiving data sent by a user end.

[0056] S12: writing data into a target data source; wherein the target data source is a data source previously determined from multiple data sources.

[0057] The data source mentioned in the present application is a commonly used data source in the current SaaS business scenario, including: a relational database management system (MySQL), ES, HBase, MongoDB, etc.

[0058] ES: Elasticsearch, which provides a distributed multi-user capable full-text search engine, is a popular enterprise-level search engine. Elasticsearch is used in cloud computing, which can achieve real-time search, stability, reliability, speed, and easy installation and use.

[0059] HBase: Hadoop Database, which is a high-reliability, high-performance, column-oriented, scalable distributed storage system, is very suitable for being used as a database for unstructured data storage.

[0060] ​MongoDB: is a database based on distributed file storage. It supports very loose data structures, so it can store more complex data types. Its query language is very powerful and also supports indexing data.

[0061] For example, in one possible application scenario, the multiple data sources are two data sources, namely a MySQL data source and an ES data source. Generally, the MySQL data source is preferred as the target data source. At this time, when receiving data sent by the user end that needs to be written to the data source, the MySQL data source and the ES data source are no longer subjected to double writing operation, but only the data is written to the MySQL data source, thereby saving data writing time and also not causing the problem of data consistency that cannot be guaranteed caused by the two-phase synchronous commit method.

[0062] It should be noted that among the above-mentioned several data sources, MySQL is widely used as a user's business system due to its strong confidentiality and other advantages. Therefore, the present embodiment provides a preferred implementation scheme: if the multiple data sources used in actual application include a MySQL data source, the MySQL data source is selected as the target data source.

[0063] Selecting the MySQL data source as the target data source can further improve the confidentiality of the data writing process, reduce the risk of data leakage of the data query system, and further ensure security.

[0064] S13: obtaining data from the target data source, and synchronizing the data to other data sources after parsing and converting the data into a corresponding format.

[0065] As mentioned above, the multiple data sources are a MySQL data source and an ES data source, and the MySQL data source is used as the target data source. Therefore, when real-time listening detects that the data of the MySQL data source has changed, step S13 is started: data written by the user end is obtained from the MySQL data source, and the data written into the MySQL data source is converted into a data format required by the ES data source through a data format conversion tool, and the data is synchronized to the ES data source in an asynchronous manner, thereby completing the data writing to the multiple data sources.

[0066] This way of synchronizing data to other data sources in an asynchronous manner can run in the background and will not be perceived by the user, so as not to affect the user's experience.

[0067] From the above, the data synchronization method in the SaaS scenario provided by the application reduces the time of the data writing process that can be perceived by the user by writing data to only one data source when writing data of multiple data sources, and then synchronizing data between the data sources through asynchronous communication, thereby optimizing the user experience. In addition, since the data writing process only writes data to one data source, it is not necessary to adopt a two-phase synchronization commit scheme when writing data, and the consistency of data writing is naturally not affected, thereby improving the accuracy of data writing and providing more accurate data results when the user queries data, further improving the user experience.

[0068] In actual application, when writing data to the data source, the data writing failure may occur, if ignored, it will affect the accuracy of the subsequent query result of the user, therefore, based on the above embodiment, as shown in the figure, Figure 1 a preferred embodiment is provided, after step S13, further comprising:

[0069] S14: determining whether a write completion signal returned by the other data source is received, if the write completion signal is not received, returning to step S13.

[0070] The write completion signal is a signal generated by the data source after the data writing process to the data source is completed. In actual implementation, if the data source writes successfully, a signal will be returned to inform that the write completion signal is the above-mentioned write completion signal, and the data sender can know whether the data is written successfully according to the write completion signal. Therefore, step S14 is actually equivalent to repeatedly writing when synchronizing data to other data sources, to ensure that the data is written into other data sources.

[0071] Similarly, in addition to the method of repeatedly writing to ensure that the data is written into the data source during the data synchronization to other data sources, the application also provides another preferred embodiment, as shown in the figure, Figure 1 after step S12, further comprising:

[0072] S15: determining whether a write completion signal returned by the target data source is received, if yes, entering step S16, if no, returning to step S12.

[0073] S16: returning a prompt information to prompt the user that the data writing is successful.

[0074] Correspondingly, the step of acquiring data from the target data source and synchronizing the data to other data sources by parsing and converting the data into a corresponding format is run in the background.

[0075] The data synchronization between the data sources is implemented in a background running manner, which is not perceived by the user end, and thus does not affect the user experience. Since the data writing process only writes data to the target data source, the data writing process is faster, and after the data writing is completed, a prompt information is returned to prompt the user that the writing is successful, so as to reduce the waiting time of the user and further optimize the user experience.

[0076] The embodiment judges whether the data writing to the target data source and other data sources is successful through the writing success signal returned by the data source after the data writing is successful, and returns to the data writing step if the data writing is not successful, so as to ensure that the data is written into the data source and further improve the accuracy of the data. In addition, the embodiment returns a prompt information to prompt the user that the data writing is successful after the user writes the data into the target data source, and the subsequent data synchronization is run in the background and is not perceived by the user, so as to optimize the user experience.

[0077] As can be seen from the above embodiment, the application provides a preferred scheme, which judges whether the data is written into the data source through the writing completion signal returned by the data source, and repeatedly writes the data if the writing is not successful, so as to ensure that the data writing is successful. However, if the data source fails or other conditions occur, the repeated writing of the data cannot be successfully written into the data source, and ignoring the situation wastes time and occupies system resources. Therefore, based on the above embodiment, the embodiment provides a preferred implementation scheme, as shown in Figure 1 After step S14, the method further includes:

[0078] S17: Counting the number of times of returning to the step of analyzing and converting the data into a corresponding format and synchronizing the data to other data sources, and obtaining a count value, and returning an alarm information when the count value exceeds a preset value.

[0079] For the above step S14, that is, counting the number of times of returning to step S13, when the number of times of repeatedly entering step S13 exceeds a preset value, it is indicated that there may be a failure that causes the repeated writing of the data to be unsuccessful, and an alarm information is returned to prompt the relevant personnel to check the failure as soon as possible.

[0080] Similarly, each time of counting is for counting a data writing process, and an alarm is given when the number of times of repetition exceeds a preset value, so that the count value needs to be cleared. Further, the count value can be cleared after the alarm information is returned, and the count value can be cleared after the data writing is successful.

[0081] Similarly, in addition to taking the preferred solution provided by the embodiment in the data synchronization process between data sources to prevent the problem of failure to be handled in time, the same operation can be performed when writing data to the target data source, that is, counting the number of times of returning to step S12 in step S15, and returning warning information when the count value exceeds the preset value. Correspondingly, the count value can be cleared after the data is successfully written and after the warning information is returned.

[0082] The embodiment counts the number of times of repeated writing when the data writing is unsuccessful in the above-mentioned embodiment, compares with the preset value to determine whether a failure occurs, and returns warning information if a failure occurs to prompt relevant personnel to timely troubleshoot the problem to restore the system to normal operation, further ensuring stability and improving user experience.

[0083] In addition, considering that data synchronization between data sources requires time, if the user performs data query a short time after writing data, there may be a problem that a small part of data has not been synchronized, causing inconsistency of the data in different data sources. Therefore, the embodiment provides a preferred solution based on the above-mentioned embodiment, as shown in Figure 1 After determining whether the write completion signal returned by the other data source is received, that is, after step S14, the method further includes:

[0084] S18: If the write completion signal is received, updating the synchronization flag field of the data in the target data source to a synchronization success state.

[0085] The synchronization flag field is a flag field added when the data is written to the target data source, and its initial state is a synchronization failure state. After the data of the target data source is successfully synchronized to other data sources, the synchronization flag field of the corresponding data is set to a synchronization success state to represent whether the current data is successfully synchronized, so that relevant personnel or modules can know whether the data is successfully synchronized to perform subsequent data query.

[0086] Based on the above-mentioned embodiment, the embodiment further provides a preferred solution when the user needs to perform data query, as shown in Figure 2 The method further includes:

[0087] S21: Receiving a query request sent by the user end.

[0088] S22: Determining the data to be queried according to the query request.

[0089] S23: Obtaining data with a synchronization failure state from the target data source and obtaining data with a synchronization success state from other data sources.

[0090] S24: Sending the obtained data to the user end.

[0091] First, determine the data to be queried according to the query request sent by the user terminal; then, obtain the data with the synchronization flag field in the synchronization failure state from the target data source and the data with the synchronization flag field in the synchronization success state from other data sources; filter, sort, merge and filter the obtained data according to certain conditions to obtain the final data query result; and finally, send the final data query result to the user terminal to complete the entire data query process.

[0092] To further illustrate the preferred scheme provided by the present embodiment, taking an example of the above-mentioned embodiment: the multiple data sources are MySQL data sources and ES data sources, and the MySQL data source is taken as the target data source. When the user sends a query request, multiple threads are started to simultaneously initiate requests to the MySQL data source and the ES data source, thread A queries the data with the synchronization flag field in the synchronization failure state in the MySQL data source and returns the queried result A; thread B queries the data with the synchronization flag field in the synchronization success state in the ES data source and returns the queried result B; then, the result A and the result B are filtered according to the preset conditions to remove redundant data, the remaining data is sorted and merged to obtain the final data result, which is returned to the user terminal, and thus the data query process is completed.

[0093] Since it has been clearly explained in the above-mentioned embodiment that after the data is written into the target data source, the written data is synchronized to other data sources in an asynchronous manner, although this data synchronization may have a certain delay in practice, since the speed of data writing to only one data source is relatively fast, this delay may be only a few seconds, so even if the user performs data query immediately after the data is written, there will be no too much unsynchronized successful data, that is, the unsynchronized successful data is usually less than the synchronized successful data.

[0094] Therefore, taking the two data sources in the example, the MySQL data source has stronger confidentiality, and is the optimal choice as the receiver when the user writes data, while the query ability of the ES data source is stronger, so it is more optimal to take the ES data source as the main data query data source. Therefore, the MySQL data source is selected as the target data source for the user to write data, and the ES data source is selected as the other data source, and when the user queries, the synchronized successful data with usually more data quantity is queried.

[0095] The preferred solution provided by the embodiment when the user performs data query is that the data required by the user to query is distinguished according to the data synchronization result, data query is performed to the target data source and other data sources by multiple threads at the same time, and the multiple results obtained are filtered, sorted, merged and combined according to certain conditions to obtain a final data result and returned to the user end. The multiple-thread simultaneous query further improves the query efficiency, and in the case that the target data source is a MySQL data source and the other data source is an ES data source, the characteristics of each data source are fully utilized, the ES data source with stronger query capability queries more data, and the MySQL data source with strong confidentiality is used as the first preferred data source for the user to write and store data, thereby bringing better use experience to the user.

[0096] In the above embodiment, the data synchronization method in a SaaS scenario is described in detail, and the present application also provides an embodiment of a data synchronization device in a SaaS scenario. It should be noted that the embodiment of the device part is described from two angles, one is based on the functional module angle, and the other is based on the hardware angle.

[0097] Based on the functional module angle, as shown in Figure 3 The embodiment provides a data synchronization device in a SaaS scenario, which comprises:

[0098] The receiving module 31 is configured to receive the data sent by the user end.

[0099] The writing module 32 is configured to write data into the target data source; wherein the target data source is a predetermined data source in the multiple data sources.

[0100] The synchronization module 33 is configured to obtain data from the target data source, and parse and convert the data into a corresponding format and synchronize the data to other data sources.

[0101] Preferably, it further comprises:

[0102] The repeated synchronization module is configured to determine whether a write completion signal returned by the other data source is received; wherein the write completion signal is a signal generated by the data source after the process of writing data into the data source is completed; if the write completion signal is not received, the step of parsing and converting the data into a corresponding format and synchronizing the data to other data sources is returned to.

[0103] The marking module is configured to update the synchronization flag field of the data in the target data source to a synchronization success state if the write completion signal is received; the synchronization flag field is a flag field added when the data is written into the target data source, and the initial state thereof is a synchronization failure state.

[0104] The query module is configured to receive a query request sent by the user terminal, determine data to be queried according to the query request, acquire data with a synchronization flag field in a synchronization failure state from a target data source and acquire data with a synchronization flag field in a synchronization success state from other data sources, and send the acquired data to the user terminal.

[0105] The perception optimization module is configured to determine whether a write completion signal returned by the target data source is received, wherein the write completion signal is a signal generated by the data source after a data writing process is completed, and if the write completion signal is received, prompt information is returned to prompt the user that the writing is successful, and if the write completion signal is not received, the step of writing data into the target data source is returned to, and correspondingly, the step of acquiring data from the target data source and synchronizing the data to other data sources after the data is parsed and converted into a corresponding format is run in the background.

[0106] The alarm module is configured to count the number of times of returning to the step of synchronizing data to other data sources after the data is parsed and converted into a corresponding format, and obtain a count value, and when the count value exceeds a preset value, warning information is returned.

[0107] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part are described in the description of the embodiments of the method part, and are not described here.

[0108] The data synchronization device in the SaaS scenario provided in the embodiment reduces the time of a data writing process that can be perceived by a user by using the writing module to write data into only one data source and then using the synchronization module to synchronize data between data sources in an asynchronous communication manner, thereby optimizing the user experience. The two-phase synchronization commit scheme is not used, and the consistency of data writing is not affected, the accuracy of data is improved, more accurate data results can be provided when the user terminal queries data, and the user experience is further improved.

[0109] Figure 4 The structure diagram of the data synchronization device in the SaaS scenario provided for another embodiment of the present application is shown in FIG. 1, which shows that the data synchronization device in the SaaS scenario includes a memory 40 configured to store a computer program. Figure 4

[0110] The processor 41 is configured to implement the steps of the data synchronization method in the SaaS scenario according to the above-described embodiments when the computer program is executed.

[0111] The data synchronization device in the SaaS scenario provided in the embodiment can include but is not limited to a smart phone, a tablet computer, a notebook computer, or a desktop computer, etc.

[0112] ​The processor 41 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 41 can be implemented in at least one of a hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA), etc. The processor 31 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 41 can be integrated with a graphics processor (GPU) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor 41 can further include an artificial intelligence (AI) processor for processing machine learning related computing operations.

[0113] The memory 40 can include one or more computer-readable storage media that can be non-transitory. The memory 40 can further include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In the present embodiment, the memory 40 is at least used to store the following computer program 401, wherein the computer program is loaded and executed by the processor 41, and can implement the related steps of the data synchronization method in a SaaS scenario disclosed in any of the preceding embodiments. In addition, the resources stored by the memory 40 can further include an operating system 402 and data 403, etc., and the storage mode can be temporary storage or permanent storage. The operating system 402 can include Windows, Unix, Linux, etc. The data 403 can include but is not limited to the data synchronization method in a SaaS scenario, etc.

[0114] In some embodiments, the data synchronization apparatus in a SaaS scenario can further include a display screen 42, an input / output interface 43, a communication interface 44, a power supply 45, and a communication bus 46.

[0115] Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the data synchronization apparatus in a SaaS scenario, and can include more or fewer components than those shown in the drawings. Figure 4 The structure shown in the above embodiments does not constitute a limitation on the data synchronization apparatus in a SaaS scenario, and can include more or fewer components than those shown in the drawings.

[0116] The embodiment of the application provides a data synchronization device in a SaaS scenario, comprising a memory and a processor, and the processor can realize the following method when executing the program stored in the memory: a data synchronization method in a SaaS scenario.

[0117] The embodiment can realize data writing to only one data source and then synchronizing data between different data sources in an asynchronous communication mode, so that the time of the data writing process is reduced, the waiting time perceived by the user is reduced, and the use experience of the user is improved. The data writing to one data source does not need to use a two-stage synchronization commit scheme, does not affect the consistency of data writing, improves the accuracy of data, can provide more accurate data results when the user end performs data query, and further improves the use experience of the user.

[0118] Finally, the application also provides an embodiment corresponding to a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the steps recorded in the above method embodiment.

[0119] It can be understood that if the method in the above embodiment is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk and various program code storage media.

[0120] The computer readable storage medium provided by the embodiment can store a computer program that is executed to write data to only one data source and then synchronize data to other data sources, thereby reducing the data writing time perceived by the user. And the data writing does not affect the consistency, thereby improving the accuracy of the data, and the user can obtain more accurate results when performing data query.

[0121] The above describes in detail the data synchronization method, device and medium in a SaaS scenario provided by the present application. The embodiments in the specification are described in a progressive manner, and each embodiment mainly explains the difference from other embodiments. The same or similar parts of each embodiment can be understood by referring to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be understood by referring to the method part. It should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, the present application can be improved and modified, and these improvements and modifications also fall within the protection scope of the claims of the present application.

[0122] It should also be noted that in the present specification, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A data synchronization method in a SaaS scenario, characterized in that, The method comprises: receiving data sent by a user terminal; writing the data into a target data source; wherein the target data source is a predetermined one of a plurality of data sources, and when the data sources include a MySQL data source, the target data source is the MySQL data source; determining whether a write completion signal returned by the target data source is received; if the write completion signal is received, returning prompt information to prompt the user that the data writing is successful; if the write completion signal is not received, returning to the step of writing the data into the target data source; running in the background: obtaining the data from the target data source, and synchronizing the data to other data sources in a corresponding format after the data is parsed and converted; determining whether a write completion signal returned by other data sources is received; wherein the write completion signal is a signal generated by the data source after the process of writing data into the data source is completed; if the write completion signal is received, updating a synchronization flag field of the data in the target data source to a synchronization success state; the synchronization flag field is a flag field added when the data is written into the target data source, and the initial state of the synchronization flag field is a synchronization failure state; if the write completion signal is not received, returning to the step of synchronizing the data to other data sources in a corresponding format after the data is parsed and converted; The method further comprises: receiving a query request sent by the user terminal; determining the data to be queried according to the query request; starting multiple threads to simultaneously obtain the data whose synchronization flag field is in a synchronization failure state from the target data source, and to obtain the data whose synchronization flag field is in a synchronization success state from other data sources; filtering redundant data according to a preset condition, sorting and merging the remaining data, and obtaining a final data result; sending the data result to the user terminal.

2. The data synchronization method in the SaaS scenario according to claim 1, characterized in that, After the determination of whether the write completion signal returned by other data sources is received, the method further comprises: counting the number of times of returning to the step of synchronizing the data to other data sources in a corresponding format after the data is parsed and converted, and obtaining a count value; when the count value exceeds a preset value, returning warning information.

3. A data synchronization apparatus in a SaaS scenario, characterized in that, The method comprises: a receiving module configured to receive data sent by a user terminal; a writing module configured to write the data into a target data source; wherein the target data source is a predetermined one of a plurality of data sources, and when the data sources include a MySQL data source, the target data source is the MySQL data source; a synchronization module configured to run in the background: obtaining the data from the target data source, and synchronizing the data to other data sources in a corresponding format after the data is parsed and converted; a repeated synchronization module configured to determine whether a write completion signal returned by other data sources is received; wherein the write completion signal is a signal generated by the data source after the process of writing data into the data source is completed; and if the write completion signal is not received, returning to the step of synchronizing the data to other data sources in a corresponding format after the data is parsed and converted; The marking module is configured to update the synchronization flag field of the data in the target data source to a synchronization success state if the write completion signal is received after determining whether the write completion signal returned by the other data sources is received; the synchronization flag field is a flag field added when the data is written into the target data source, and the initial state of the synchronization flag field is a synchronization failure state; wherein the write completion signal is a signal generated by the data source after the process of writing data into the data source is completed; The query module is configured to receive a query request sent by the user end; determine the data to be queried according to the query request; start multiple threads to simultaneously acquire the data whose synchronization flag field is in a synchronization failure state from the target data source and acquire the data whose synchronization flag field is in a synchronization success state from the other data sources; filter the data acquired from the target data source and the other data sources according to a preset condition, remove redundant data, sort and merge the remaining data, and obtain a final data result; and send the data result to the user end; The perception optimization module is configured to determine whether a write completion signal returned by the target data source is received; if the write completion signal is received, return a prompt information to prompt that the data is successfully written; and if the write completion signal is not received, return to the step of writing the data into the target data source.

4. A data synchronization apparatus in a SaaS scenario, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the data synchronization method in the SaaS scenario as claimed in claim 1 or 2. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the data synchronization method in the SaaS scenario as claimed in claim 1 or 2. ​ 5. A computer-readable storage medium, characterized in that, ​

Images