A method for data offline synchronization between a client and a server based on an MQTT protocol

Abstract

The application provides a kind of data offline synchronization method based on MQTT protocol client and server, it is related to information technology field, including that client and server establish MQTT long connection;Client encapsulates message synchronization interface, and judges current network state in interface, if network is unobstructed, go online mode, if network is not connected, go offline mode;In online mode, data is directly synchronized to server through MQTT connection;In offline mode, client first caches synchronization data, and checks network state regularly, when network recovers, immediately start synchronization task;Server receives message, and parses data according to message type and handles it.The application can be applied in places with good network environment such as shopping mall, office building, street, etc., and also can be applied in places with poor network environment such as field, mountainous area, jungle, etc., with wider application scenarios.Using the characteristics of MQTT publishing and subscribing, when a large amount of data is used, the server can process data on demand, reduce the pressure on the service, and improve the stability of the service.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method for offline data synchronization between a client and a server based on the MQTT protocol. Background Technology

[0002] With the continuous development of mobile internet, people's operating habits are increasingly shifting to mobile devices. Various mobile apps are increasingly being used in commercial sectors such as industry, agriculture, and manufacturing. The large amounts of data generated by mobile operations need to be synchronized to servers. However, with the diversification of application areas and the complexity of usage environments, data synchronization faces a series of problems when used in weak or no-network environments, such as unreliable networks, network outages, and inability to connect to the server. Existing data synchronization methods have the following shortcomings:

[0003] 1. Existing data synchronization methods mostly work in environments with good network conditions. In environments without network access, manual intervention is required, and manual media transmission is used.

[0004] 2. Manual transmission requires external equipment, involves cumbersome operating procedures, is lengthy, and is prone to errors and problems.

[0005] 3. Manual transmission is not timely and requires additional operations to integrate the data into the system, making it difficult to achieve good integration with the data processing system.

[0006] 4. Currently, data synchronization in network environments mainly involves the transmission of small amounts of data. Large amounts of data can easily accumulate and put pressure on the system.

[0007] Therefore, in order to address these issues, implementing an offline data synchronization method between the client and server based on the MQTT protocol has become an urgent need for data synchronization in complex environments. Summary of the Invention

[0008] The purpose of this invention is to provide a method for offline data synchronization between a client and a server based on the MQTT protocol, so as to solve the problems in the background art.

[0009] To achieve the above objectives, the technical solution of the present invention is as follows:

[0010] A method for offline data synchronization between client and server based on the MQTT protocol is proposed. Combining the publish-subscribe pattern of the MQTT protocol, a cross-terminal data synchronization method that supports both online and offline data synchronization is presented.

[0011] The data synchronization is divided into client and server sides, and the client data is synchronized to the server side through MQTT long connection.

[0012] The data synchronization method described above supports both online and offline modes. In online mode, client data is directly synchronized to the server. In offline mode, client data is cached first and automatically synchronized to the server after the network is restored.

[0013] In online mode, both the client and server establish long-lived connections with the MQTT service. Using a publish-subscribe model, the client first publishes data to the MQTT service, and the server subscribes to the corresponding topic to receive data synchronized from the client.

[0014] In offline mode, the client and server are not connected to the network and cannot communicate directly. When the client synchronizes data, it checks the network connection through a proxy service. If the network is not connected, the data is cached on the client first. After the network is restored, the offline synchronization task is started to synchronize the cached data in the offline environment to the server.

[0015] The client needs to have a built-in SQLite database (or other client-cached database). In offline mode, the client caches the data in the database for unified management and processes the offline data after the network is restored.

[0016] The client-side cache database can be divided into different data tables depending on the business scenario. In offline mode, a single table is used for processing. It mainly has two fields, which are used to identify the data type and the specific data content stored in different scenarios. Each data type corresponds to a specific content in a different data format.

[0017] The client has an MQTT listener that constantly monitors network connections. When the connection is open, data synchronization goes directly to online mode; when the connection is closed, data synchronization goes to offline mode.

[0018] The client has a built-in offline data synchronization task specifically for handling offline data. When the listener detects that the network connection has been restored, the offline data synchronization task is started immediately. After each piece of offline data is synchronized, the data is deleted from the cache to prevent duplicate data synchronization.

[0019] After receiving the data synchronized from the client, the server parses the data content according to the data type. Each type corresponds to a data format, which can be differentiated using the strategy pattern. The data is then permanently stored in the server's database.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1. Automatic data synchronization in offline environments is achieved through software, avoiding manual intervention, simplifying the process, and improving data synchronization efficiency.

[0022] 2. Data synchronization between the client and server is achieved under conditions of low bandwidth and unstable network environment through the MQTT protocol, which reduces the requirements of the network environment for data synchronization and enables the client to synchronize data to the server in a timely manner.

[0023] 3. By using the publish-subscribe pattern of the MQTT protocol, data synchronization between the client and the server can be achieved. This can effectively avoid excessive server load caused by large amounts of data, which would affect server performance. The server can reasonably control the processing speed according to its own processing capacity.

[0024] 4. This data synchronization method supports both online and offline modes, and the system can automatically switch between them without the user's awareness, improving user convenience. At the same time, this method does not depend on the external environment, meets the data synchronization needs of any scenario, and has stronger versatility. Attached Figure Description

[0025] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is an overall structural diagram of the present invention;

[0027] Figure 2 This is a flowchart illustrating the overall operation of the present invention.

[0028] Figure 3 This is a flowchart of the offline data processing task of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0031] like Figure 1As shown, this paper describes a client-server offline data synchronization method based on the MQTT protocol, comprising four parts: a client, an MQTT service, a server, and a database. The client app is installed on a mobile device, such as a phone or tablet; the MQTT service, server, and database are each deployed independently on a server. The client and server subscribe to the same topic, achieving data synchronization through the MQTT publish-subscribe model. The client provides a unified data reporting interface; users do not need to concern themselves with the specific implementation, but can directly call the interface to achieve data synchronization. This method supports both online and offline modes. In the online environment, data is directly synchronized to the server via an MQTT long connection; in offline mode, data is first cached locally, then the network status is checked periodically. When the network recovers, the cached data is synchronized, and successfully synchronized cached data is deleted. The entire process is seamless for the user, who does not need to pay attention to environmental changes. After the data is synchronized to the server, the server parses and processes it according to corresponding rules and saves the data in the database for permanent storage.

[0032] like Figure 2 As shown in this embodiment, the offline data synchronization method between the client and server based on the MQTT protocol includes the following steps:

[0033] Step 1: Client Data Reporting

[0034] The client provides a unified data reporting interface, which users can call according to specified rules when they need to report.

[0035] Step 2: The listener monitors network changes.

[0036] The client has a network monitoring service that monitors the network status in real time and determines the next step to be executed based on the current network status.

[0037] Step 3: Determine if the network is connected.

[0038] The client's environment is unpredictable. When there is a network connection, data reporting will be done in online mode; when there is no network connection, data reporting will be done in offline mode.

[0039] Step 4: Local Data Caching

[0040] In offline mode, data cannot be immediately synchronized to the server. The data needs to be cached locally first and synchronized after the network is restored.

[0041] Step 5: Start the detection task

[0042] In offline environments, data needs to be synchronized only after the network is restored. Therefore, it is necessary to detect network recovery in a timely manner. For this purpose, a detection task is enabled to periodically check the network status.

[0043] Step 6: Regularly check the network environment.

[0044] In an offline environment, the detection task will run continuously, performing periodic checks when the network is not restored, and exiting the task once the network is restored.

[0045] Step 7: Re-establish the MQTT connection

[0046] If the original MQTT connection is lost in an offline environment, the original MQTT connection needs to be restored once the network is restored.

[0047] Step 8: Start the offline data processing task

[0048] In offline environments, all reported data is cached on the client. The client has a dedicated processing task for this cached data, which is started when the network is restored.

[0049] Step 9: Publish to the server

[0050] When the network connection is normal, data can be directly synchronized to the server through the publishing interface.

[0051] Step 10: Determine whether to cache data.

[0052] Determine whether the published data comes from client-cached data or directly published data. Directly published data requires no special processing after publication, while client-cached data needs to be deleted after publication.

[0053] Step 11: Delete cached data

[0054] When the data being published comes from client-cached data, the data needs to be deleted from the local machine after publication to prevent duplicate publication.

[0055] like Figure 3 As shown, this example implements an offline data processing task based on the MQTT protocol for offline data synchronization between the client and server, including the following steps:

[0056] Step 1: Add a lock to ensure that only one task is executing at a time.

[0057] Tasks may be started multiple times. To prevent tasks from being executed repeatedly, locking is required to ensure that only one task is executing at a time.

[0058] Step 2: Batch query cached data

[0059] After the task starts, it first queries the client's cached data. To improve efficiency, it queries the data in batches and then executes subsequent steps based on the query results.

[0060] Step 3: Send buffered data in a loop

[0061] If cached data can be found, the send interface is called to send the cached data to the server, and subsequent steps are executed based on the sending result.

[0062] Step 4: Monitor whether the data was sent successfully.

[0063] After data is sent, listen for whether the sending was successful. If the sending was successful, delete the data from the cache. If the sending failed, no action is needed, and wait for the next sending attempt.

[0064] Step 5: Delete cached data

[0065] The data is deleted from the client database to prevent duplicate publication, and then the next round of queries is initiated.

[0066] Step 6: Exit the task

[0067] If a data query returns an empty result, it means that all cached data has been synchronized, and the task can be exited, ending the data synchronization process.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for offline data synchronization between a client and a server based on the MQTT protocol, characterized in that, include: The client establishes a long-lived MQTT connection with the server; The client encapsulates a message synchronization interface and checks the current network status within the interface. If the network is accessible, it uses online mode; otherwise, it uses offline mode. In online mode, data is directly synchronized to the server via an MQTT connection. In offline mode, the client first caches the synchronized data and periodically checks the network status. Once the network recovers, the synchronization task begins immediately. Upon receiving a message, the server parses and processes the data according to the message type.

2. The method for offline data synchronization between a client and a server based on the MQTT protocol according to claim 1, characterized in that, Data transmission supports both online and offline modes, and data will not be lost even when the network is disconnected.

3. The method for offline data synchronization between a client and a server based on the MQTT protocol according to claim 1, characterized in that, The MQTT protocol's publish-subscribe pattern is used to synchronize information between the client and the server. The client publishes data to a specified topic, and the server subscribes to the corresponding topic to receive the data published by the client.

4. The method for offline data synchronization between a client and a server based on the MQTT protocol according to claim 1, characterized in that, It supports both online and offline network modes to meet data synchronization needs in any network environment. In online mode, the client can directly synchronize data to the server via an MQTT long connection.

5. The method for offline data synchronization between a client and a server based on the MQTT protocol according to claim 1, characterized in that, It supports data synchronization in offline mode. When the network is unavailable in offline mode, the client's proxy service will cache the data on the client after detecting the network disconnection, and automatically synchronize it to the server when the network is restored.

6. The method for offline data synchronization between a client and a server based on the MQTT protocol according to claim 4, characterized in that, The client has a built-in SQLite database (or other client-cached database). In online mode, data is directly synchronized to the server. In offline mode, data is cached in the client database to ensure that data is not lost in offline mode.

7. The method for offline data synchronization between a client and a server based on the MQTT protocol according to claim 5, characterized in that, The client has an MQTT listener that automatically starts a scheduled service when it detects that the MQTT connection has been lost. This service periodically checks the network connection status, and once the network connection is restored, it establishes the MQTT connection and starts an offline data synchronization task.

8. The method for offline data synchronization between a client and a server based on the MQTT protocol according to claim 7, characterized in that, The client has a built-in offline data synchronization task that is responsible for synchronizing offline data. After each piece of client-cached data is synchronized, the local cached data is deleted to prevent duplicate synchronization of client data.

9. A method for offline data synchronization between a client and a server based on the MQTT protocol according to claim 6, characterized in that, The client has a built-in cache database. Offline data is cached in a table with two main fields, which are used to identify the data type for different scenarios and to store the specific data content.

10. A method for offline data synchronization between a client and a server based on the MQTT protocol according to claim 1, characterized in that, The server parses data from different sources based on the different types of data received. Each data type corresponds to a data structure, and the server can use the strategy pattern to perform differentiated data processing.

Images