Differential data transmission method and system based on message middleware

Abstract

The application discloses a differential data transmission method based on a message middleware, which comprises the following steps: a differential service on an Internet side receives differential data broadcasted by the Internet and generates a first intermediate message; a message middleware in a user security device receives and consumes the first intermediate message, and compares ephemeris of current differential data with ephemeris of differential data cached in the user security device when the message middleware consumes the first intermediate message; it is determined that the ephemeris of the current differential data is after the ephemeris of the cached differential data, and the message middleware generates a second intermediate message according to the current differential data; a differential service in a user private network receives and consumes the second intermediate message, and broadcasts to terminals in the private network after consumption. The application realizes application of Internet differential data to the user private network within a valid age period.

Description

Technical Field

[0001] This application relates to the field of positioning technology, and in particular to a differential data transmission method and system based on message middleware. Background Technology

[0002] For certain industries, such as power grids, which have high data security requirements, high-precision positioning cannot be achieved using internet-based differential data services. Instead, differential data must be used within their dedicated industry-specific business networks. Therefore, differential data from the internet side needs to be synchronized to the user's internal business network.

[0003] Security devices are often used to isolate the internet and the user's intranet, preventing bidirectional TCP communication between the two sides of the isolation device and even hindering the support for long TCP connections. This inevitably affects the transmission of RTCM (Radio Technical Commission for Maritime Services) data.

[0004] Traditional methods convert differential data into files using a specific method and transfer them via FTP (File Transfer Protocol). This first achieves cross-network or cross-platform text transfer, and then the transferred files are converted into data streams for application. This approach has two drawbacks:

[0005] Defect 1: Converting streaming data into text data requires reading and writing to the hard drive, which cannot guarantee persistent operation.

[0006] Defect 2: When the text file is converted back into a data stream, the data transmission efficiency is severely reduced, and the validity of the differential data cannot be guaranteed. Summary of the Invention

[0007] The purpose of this application is to provide a differential data transmission method and system based on message middleware, which enables the application of differential data from the Internet to the user's private network within the validity period.

[0008] One embodiment of this application discloses a differential data transmission method based on message middleware, comprising:

[0009] The differential service on the Internet side receives differential data broadcast on the Internet and produces the first intermediate message;

[0010] The message middleware within the user security device receives and consumes the first intermediate message, and when consuming the first intermediate message, it compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user security device.

[0011] Upon determining that the ephemeris of the current differential data is after the ephemeris of the cached differential data, the message broker generates a second intermediate message based on the current differential data; and

[0012] The differential service within the user's private network receives and consumes the second intermediate message, and then broadcasts it to the terminals within the private network after consumption.

[0013] In a preferred embodiment, the method further includes: determining that the ephemeris of the current differential data is earlier than the ephemeris of the cached differential data, and then discarding the current differential data.

[0014] In a preferred embodiment, the method further includes: determining that the ephemeris of the current differential data is after the ephemeris of the cached differential data, and then the user security device replaces the cached differential data with the current differential data.

[0015] In a preferred embodiment, the method further includes: when the differential service within the user private network consumes the second intermediate message, comparing the ephemeris of the current differential data with the ephemeris of the differential data cached within the user private network, and determining that the ephemeris of the current differential data is after the ephemeris of the cached differential data, broadcasting it to the terminal within the private network, and the user private network replacing the cached differential data with the current differential data.

[0016] In a preferred embodiment, the method further includes: the differential service within the user private network determines that the ephemeris of the current differential data is earlier than the ephemeris of the cached differential data, and then discards the current differential data.

[0017] In a preferred embodiment, after consuming the second intermediate message, the differential service of the user private network converts it into broadcastable differential data and broadcasts it to terminals within the private network.

[0018] In a preferred embodiment, the Internet side broadcasts differential data based on the user's private network subscription or the range of the user's private network.

[0019] In a preferred embodiment, the message middleware is selected from one or more of the following group: Kafka, RabbitMQ, ActiveMQ, ZeroMQ, RocketMQ, and Redis.

[0020] In a preferred embodiment, the user security device is selected from one or more of the following groups: network gateway, optical gateway, and boundary.

[0021] One embodiment of this application also discloses a differential data transmission system based on message middleware, comprising:

[0022] The differential data access module is used to receive differential data broadcast over the Internet and generate the first intermediate message;

[0023] A message middleware, located within the user's security device, is used to receive and consume the first intermediate message. When consuming the first intermediate message, the message middleware compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user's security device.

[0024] A verification module is configured to determine that the ephemeris of the current differential data is after the ephemeris of the cached differential data, thereby enabling the message middleware to generate a second intermediate message based on the current differential data; and

[0025] The differential data broadcasting module, located within the user's private network, is used to receive and consume the second intermediate message, and then broadcast it to the terminals within the private network after consumption.

[0026] The embodiments of this application have the following beneficial effects:

[0027] 1) This invention enables near real-time data transmission of differential data from the Internet to the user's private business network, ensuring the validity of the differential data within its lifespan.

[0028] 2) By using a message middleware to facilitate the transmission of streaming data across network security devices.

[0029] 3) Combining the characteristics of RTCM data, and making judgments based on time series, ensure the validity of differential data.

[0030] The specification of this application contains numerous technical features distributed across various technical solutions. Listing all possible combinations of these technical features (i.e., technical solutions) would make the specification excessively lengthy. To avoid this problem, the various technical features disclosed in the above-described invention, the various technical features disclosed in the following embodiments and examples, and the various technical features disclosed in the accompanying drawings can be freely combined to form various new technical solutions (all of which should be considered as described in this specification), unless such a combination of technical features is technically infeasible. For example, one example discloses feature A+B+C, and another example discloses feature A+B+D+E. Features C and D are equivalent technical means that serve the same function, and technically only one needs to be used; they cannot be used simultaneously. Feature E can technically be combined with feature C. Therefore, the solution A+B+C+D should not be considered as described because it is technically infeasible, while the solution A+B+C+E should be considered as described. Attached Figure Description

[0031] Figure 1This is a flowchart illustrating a differential data transmission method based on message middleware according to one embodiment of this application.

[0032] Figure 2 and Figure 3 This is a schematic diagram illustrating the process of transmitting differential data across different networks according to one embodiment of this application.

[0033] Figure 4 This is a flowchart illustrating the process of transmitting data using a message middleware according to one embodiment of this application.

[0034] Figure 5 This is a process diagram of data transmission using a message middleware with timing verification according to one embodiment of this application. Detailed Implementation

[0035] In the following description, many technical details are presented to help the reader better understand this application. However, those skilled in the art will understand that the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0036] Explanation of some concepts:

[0037] Message Queues (MQ) are software systems that provide reliable, synchronous or asynchronous message transmission for application systems in a network environment, based on queue and message delivery technologies. A message queue is essentially a container that stores messages during transmission. It acts as an intermediary when relaying messages from their source to their destination. The primary purpose of a queue is to provide routing and ensure message delivery. If the receiver is unavailable when a message is sent, the message queue will retain the message until it can be successfully delivered; however, there is a time limit for message queues to store messages.

[0038] Producers produce messages for message middleware.

[0039] Consumers consume messages for the message middleware.

[0040] A thread is the smallest unit of computation that an operating system can schedule. It is contained within a process and is the actual unit of operation within that process. A thread refers to a single, sequential flow of control within a process. Multiple threads can run concurrently within a process, each executing different tasks in parallel.

[0041] Private network: refers to a professional network that provides secure and reliable wireless services to specific departments or groups (such as government affairs and public safety industries), and its basic network is generally independent of the public cellular mobile communication network.

[0042] Security devices: These can prevent internet or external network users from accessing the internal network, or impose more restrictions on such access.

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0044] The first embodiment of this application relates to a differential data transmission method based on message middleware, the process of which is as follows: Figure 1 As shown, the method includes the following steps:

[0045] Step 101: The differential service on the internet side receives the differential data broadcast from the internet and produces a first intermediate message. In one embodiment, the internet side broadcasts the differential data based on the user's private network subscription or the range of the user's private network.

[0046] Step 102: The message middleware in the user security device receives and consumes the first intermediate message, and when consuming the first intermediate message, it compares the ephemeris of the current differential data with the ephemeris of the differential data cached in the user security device.

[0047] In one embodiment, the user security device is selected from one or more of the following groups: network gateway, optical gateway, and boundary.

[0048] In one embodiment, the message middleware is selected from one or more of the following group: Kafka, RabbitMQ, ActiveMQ, ZeroMQ, RocketMQ, and Redis.

[0049] Step 103: Determine that the ephemeris of the current differential data is after the ephemeris of the cached differential data, and the message middleware generates a second intermediate message based on the current differential data.

[0050] In one embodiment, the differential data transmission method further includes: determining that the ephemeris of the current differential data is earlier than the ephemeris of the cached differential data, and then the message middleware discards the current differential data.

[0051] In one embodiment, the differential data transmission method further includes: determining that the ephemeris of the current differential data is after the ephemeris of the cached differential data, and the user security device replacing the cached differential data with the current differential data.

[0052] Step 104: The differential service within the user's private network receives and consumes the second intermediate message, and then broadcasts it to the terminals within the private network after consumption.

[0053] In one embodiment, the differential data transmission method further includes: when consuming the second intermediate message, the differential service within the user private network compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user private network. Furthermore, if it is determined that the ephemeris of the current differential data is after the ephemeris of the cached differential data, it broadcasts it to the terminals within the private network. The user private network then replaces the cached differential data with the current differential data.

[0054] In one embodiment, the differential data transmission method further includes: the differential service within the user private network determines that the ephemeris of the current differential data is earlier than the ephemeris of the cached differential data, and then discards the current differential data.

[0055] In one embodiment, after consuming the second intermediate message, the differential service of the user private network converts it into broadcastable differential data and broadcasts it to terminals within the private network.

[0056] It should be understood that the production and consumption processes of the first and second intermediate messages are the same.

[0057] This invention enables streaming data to be transmitted across network security devices through a message middleware intermediary, and achieves near real-time data transmission of differential data from the Internet to the user's private business network, ensuring the validity of the differential data within its lifespan.

[0058] To better understand the technical solution of this application, a specific example is provided below. The details listed in this example are mainly for ease of understanding and are not intended to limit the scope of protection of this application.

[0059] Existing differential data transmission technologies for terminal devices are not suitable for differential data transmission between different networks or software systems.

[0060] This application utilizes message middleware for data transfer, enabling the transmission of differential data from the internet to the user's private network within its validity period. Message middleware includes Kafka, RabbitMQ, ActiveMQ, ZeroMQ, RocketMQ, Redis, and other message middleware whose processing mechanism is message production and consumption. Figure 2 and Figure 3 This is a schematic diagram illustrating the process of transmitting differential data across different networks according to one embodiment of this application. The specific process of differential data transmission is as follows:

[0061] 1.1 From public services to user safety protection

[0062] SGR data is received outside the user's security protection device using conventional data push methods (e.g., internet push / dedicated line push). Differential data can be specific differential data subscribed to based on user needs, or regional differential data pushed based on the user's usage range.

[0063] 1.2 Dedicated service for synchronizing differential data from the external security protection equipment to the business private network.

[0064] Differential data is synchronized to the data receiving module of the business private network via message middleware.

[0065] The implementation process includes the following steps:

[0066] 1.2.1 The differential data receiving service on the outside of the security protection equipment (dedicated service 1, i.e. the client of the message middleware) generates a message upon receiving differential data.

[0067] 1.2.2 After receiving and consuming the message, the message middleware (server) inside the security device forwards it to the internal message middleware of the security device through the internal logic of the security device.

[0068] 1.2.3 The message middleware inside the security device generates new messages from the differential data and forwards them to the differential service built into the private network outside the security device.

[0069] 1.2.4 The built-in differential service (dedicated service 2, message middleware client) of the private network consumes messages generated by security devices, converts them into differential data that can be broadcast externally, and broadcasts the differential data to terminal devices (e.g., smart terminals, vehicles, etc.) within the user's private network, enabling the terminal devices to achieve high-precision positioning within the user's private network.

[0070] To ensure efficiency, message queues typically do not constrain the execution time of data consumption, which can lead to discrepancies between the timeline of the consumed data and the original data. For differential data, if the timeline is incorrect, its validity cannot be guaranteed.

[0071] Taking the message broker Kafka as an example, the message transmission process in Kafka itself does not result in message order disruption. A Kafka partition function is consumed by only one consumer; data is written to the partition in the correct order, and the consumer retrieves data from the partition in a certain order as well. However, multiple threads calling the same function within the consumer can potentially lead to message order disruption. (Reference) Figure 4As shown, the producer generates data 1, data 2, and data 3 in normal time sequence to the message middleware Kafka. Kafka writes data 1, data 2, and data 3 into different functions partition1 and partition2, respectively. Due to the multi-threaded processing of the message middleware, the consumer receives data in the order of data 2, data 3, and data 1. The time sequence of data 1 is disordered, making it unusable for location and calculation.

[0072] To overcome this problem, this application performs data order reordering and verification based on the timestamps of the differential data after data consumption. Before data consumption, this invention caches the latest, time-accurate differential data in memory. Upon receiving new consumed differential data, it compares the timing with the data in the cache. If the timing is correct, the data is retained and broadcast downwards, while the differential data in the cache is updated, thus ensuring data real-time performance. If the received data is time-delayed, the differential data is discarded and not broadcast downwards, preventing location data anomalies caused by terminal device calculation errors. Figure 5 As shown, after differential data 1, data 2, and data 3 are transmitted, a timing check reveals that the order of data 1 is disordered, so data 1 is discarded. It should be noted that since there are many differential data streams, subsequent data will be added, so the loss of one or two data packets will not significantly affect the positioning effect.

[0073] For example, when a message broker within a user's security device receives and consumes a message, it compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user's security device. If it determines that the ephemeris of the current differential data is later than the ephemeris of the cached differential data, the message broker generates a new message based on the current differential data and replaces the cached differential data with the current differential data. If it determines that the ephemeris of the current differential data is earlier than the ephemeris of the cached differential data, the message broker discards the current differential data.

[0074] For example, when the differential service within the user's private network consumes messages generated by a security device, it compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user's private network. If it determines that the ephemeris of the current differential data follows the ephemeris of the cached differential data, it broadcasts it to the terminals within the private network, and the user's private network replaces the cached differential data with the current differential data. If the differential service within the user's private network determines that the ephemeris of the current differential data precedes the ephemeris of the cached differential data, it discards the current differential data.

[0075] The second embodiment of this application relates to a message middleware-based differential data transmission system, which includes: a differential data access module, a message middleware, a verification module, and a differential data broadcasting module. Wherein:

[0076] The differential data access module is located on the Internet side and is used to receive differential data broadcast by the Internet and produce the first intermediate message.

[0077] The message middleware is located within the user's security device and is used to receive and consume the first intermediate message. When the message middleware consumes the first intermediate message, it compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user's security device.

[0078] The verification module is used to determine that the ephemeris of the current differential data is after the ephemeris of the cached differential data, so that the message middleware generates a second intermediate message based on the current differential data.

[0079] The differential data broadcasting module is located within the user's private network and is used to receive and consume the second intermediate message, and then broadcast it to the terminals within the private network after consumption.

[0080] The first embodiment is a method embodiment corresponding to this embodiment. The technical details in the first embodiment can be applied to this embodiment, and the technical details in this embodiment can also be applied to the first embodiment.

[0081] It should be noted that those skilled in the art should understand that the implementation functions of each module shown in the above-described implementation of the message middleware-based differential data transmission system can be understood with reference to the relevant description of the aforementioned message middleware-based differential data transmission method. The functions of each module shown in the above-described implementation of the message middleware-based differential data transmission system can be implemented by a program (executable instructions) running on a processor, or by specific logic circuits. If the above-described message middleware-based differential data transmission system is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0082] It should be noted that in this patent application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this patent application, if it refers to performing an action according to an element, it means performing the action at least according to that element, including two cases: performing the action only according to that element, and performing the action according to that element and other elements. Expressions such as "multiple," "repeatedly," and "various" include two, two times, two kinds, and more than two, more than two times, and more than two kinds.

[0083] All references to this specification are considered to be incorporated integrally into the disclosure of this application so that they can serve as the basis for modifications if necessary. Furthermore, it should be understood that the above descriptions are merely preferred embodiments of this specification and are not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of one or more embodiments of this specification.

[0084] In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A differential data transmission method based on message middleware, characterized in that, include: The differential service on the Internet side receives differential data broadcast on the Internet and produces the first intermediate message; The message middleware within the user security device receives and consumes the first intermediate message, and when consuming the first intermediate message, it compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user security device. Once it is determined that the ephemeris of the current differential data is after the ephemeris of the cached differential data, the message middleware generates a second intermediate message based on the current differential data, and the user security device replaces the cached differential data with the current differential data. If the ephemeris of the current differential data is determined to be earlier than the ephemeris of the cached differential data, the message middleware discards the current differential data. as well as The differential service within the user private network receives and consumes the second intermediate message, and broadcasts it to the terminals within the private network after consumption. When consuming the second intermediate message, the differential service within the user private network compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user private network, and determines that the ephemeris of the current differential data is after the ephemeris of the cached differential data, then broadcasts it to the terminals within the private network. The user private network replaces the cached differential data with the current differential data. If the differential service within the user private network determines that the ephemeris of the current differential data is before the ephemeris of the cached differential data, then discards the current differential data. The user security device isolates the Internet and the user private network, preventing them from engaging in bidirectional TCP communication.

2. The differential data transmission method according to claim 1, characterized in that, After consuming the second intermediate message, the differential service of the user private network converts it into broadcastable differential data and broadcasts it to the terminals within the private network.

3. The differential data transmission method according to claim 1, characterized in that, The internet side broadcasts differential data based on the user's private network subscription or the range of the user's private network.

4. The differential data transmission method according to claim 1, characterized in that, The message middleware is selected from one or more of the following groups: Kafka, RabbitMQ, ActiveMQ, ZeroMQ, RocketMQ, and Redis.

5. The differential data transmission method according to claim 1, characterized in that, The user security device is selected from one or more of the following groups: network gateway, optical gateway, and boundary.

6. A differential data transmission system based on message middleware, characterized in that, include: The differential data access module is used to receive differential data broadcast over the Internet and generate the first intermediate message; A message middleware, located within the user's security device, is used to receive and consume the first intermediate message. When consuming the first intermediate message, the message middleware compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user's security device. The verification module is used to determine that the ephemeris of the current differential data is after the ephemeris of the cached differential data, so that the message middleware generates a second intermediate message based on the current differential data, and the user security device replaces the cached differential data with the current differential data. If the ephemeris of the current differential data is determined to be earlier than the ephemeris of the cached differential data, the message middleware will discard the current differential data. as well as A differential data broadcasting module, located within the user private network, is used to receive and consume the second intermediate message, and broadcast it to terminals within the private network after consumption. When consuming the second intermediate message, the differential service within the user private network compares the ephemeris of the current differential data with the ephemeris of the differential data cached within the user private network, and determines that the ephemeris of the current differential data is after the ephemeris of the cached differential data, before broadcasting it to terminals within the private network. The user private network then replaces the cached differential data with the current differential data. If the differential service within the user private network determines that the ephemeris of the current differential data is before the ephemeris of the cached differential data, it discards the current differential data. The user security device isolates the Internet and the user private network, preventing them from engaging in bidirectional TCP communication.

Images