Remote data reporting method, system and medium for remote database

Abstract

This application discloses a remote data reporting method, system, and medium for off-site databases, which mainly relates to the field of remote data reporting technology. It is used to solve the problem that the existing message middleware adopts a static topic mapping mechanism and a sliding time window technology that is not effectively combined with a dynamic topic template mechanism, making it difficult to achieve real-time perception of data features and automatic adaptation of storage strategies. It includes: sending collected data to the EMQX message server via the MQTT protocol; wherein the collected data includes a timestamp and data type; calculating the data change frequency of each data type within the sliding time window at a fixed time interval; determining the preset main structure template corresponding to the collected data based on the data change frequency, and then storing the data according to the preset main structure template; when any preset main structure template updates the topic structure, updating the collected data under the original preset main structure template to the new topic structure, and notifying the preset client to update the subscription.

Description

Technical Field

[0001] The present application relates to the technical field of remote data reporting for off-site libraries, and in particular to a method, system, and medium for remote data reporting for off-site libraries. Background Art

[0002] With the rapid development of industrial Internet and Internet of Things technologies, remote data reporting from off-site databases has become a key link for enterprises to achieve distributed data collection and analysis. However, existing technical solutions still have significant shortcomings in dynamic data processing and transmission optimization. Traditional methods usually adopt a fixed-frequency full data synchronization mechanism, which not only causes a huge waste of network bandwidth, but also makes it difficult to adapt to the differences in real-time requirements of different types of data. Especially when faced with massive amounts of time-series data generated by industrial equipment, static sampling strategies often lead to the loss of high-frequency important data or the redundancy of low-frequency non-critical data, seriously affecting the accuracy and timeliness of data analysis. In addition, although the MQTT protocol can guarantee basic data transmission, it lacks the ability to intelligently perceive data characteristics and cannot realize dynamic adjustment of transmission strategies.

[0003] Existing solutions also face the challenge of data migration when topic structures change. Most MQTT-based messaging middleware uses a static topic mapping mechanism. When changing business requirements lead to adjustments to topic structures, manual intervention is required to re-index and migrate historical data, which is not only inefficient but also prone to service interruptions. Although sliding time window technology has been applied to local data analysis, it has not yet been effectively integrated with dynamic topic template mechanisms, making it difficult to achieve real-time perception of data characteristics and automatic adaptation of storage strategies. Summary of the Invention

[0004] The present application provides a remote data reporting method, system and medium for an off-site library to solve the problem that the existing MQTT-based message middleware adopts a static topic mapping mechanism and a sliding time window technology that is not effectively combined with a dynamic topic template mechanism, making it difficult to achieve real-time perception of data features and automatic adaptation of storage strategies.

[0005] In a first aspect, the present application provides a remote data reporting method for a remote database, the method comprising:

[0006] Deploy a data collection program on the remote library server to collect data of preset data types in real time according to the preset sampling frequency, and add a timestamp to the collected data; send the collected data to the EMQX message server via the MQTT protocol; the collected data contains a timestamp and data type; maintain a sliding time window for each data type on the EMQX message server; when new collected data is received, store the collected data in the window queue of the sliding time window of the corresponding data type; calculate the data change frequency of each data type within the sliding time window at a fixed time interval; determine the preset main structure template corresponding to the collected data based on the data change frequency, and then store it according to the preset main structure template; when any preset main structure template updates the topic structure, update the collected data under the original preset main structure template to the new topic structure, and notify the preset client to update the subscription.

[0007] In one implementation of the present application, notifying a preset client to update a subscription specifically includes:

[0008] When any preset subject structure template updates the subject structure, a subject change notification is sent to the client subscribing to the subject corresponding to the original preset subject structure template; wherein the subject change notification includes the mapping relationship between the old preset subject structure template and the new preset subject structure template;

[0009] After the client receives the topic change notification but fails to respond in time, the EMQX message server automatically forwards the old subscription topic message to the new subscription topic corresponding to the new preset main structure template.

[0010] In one implementation of the present application, data change frequency=data change amount / time interval.

[0011] In one implementation of the present application, determining a preset main structure template corresponding to the collected data according to the frequency of data changes specifically includes:

[0012] When the data change frequency is within the preset high frequency range, the corresponding four-level theme is determined; the four-level theme is: remote library / device / type / real-time;

[0013] When the data change frequency is within the preset medium frequency range, the corresponding third-level theme is determined; the third-level theme is: remote library / device / type;

[0014] When the data change frequency is within a preset low-frequency range, the corresponding secondary theme is determined; wherein the secondary theme is: remote library / equipment.

[0015] In one implementation of the present application, before sending a topic change notification to a client subscribing to a topic corresponding to the original preset main structure template, the method further includes:

[0016] The preset rule engine obtains the updated subject structure of the preset main structure template, thereby triggering data migration.

[0017] In one implementation of the present application, after determining a preset main structure template corresponding to the collected data and then storing the data according to the preset main structure template, the method includes:

[0018] Specify QoS2 level for subscription / publishing operations of the four-level topic in the preset main structure template;

[0019] Specify QoS 0 for the subscription / publish operations of the secondary topic in the preset main structure template.

[0020] In a second aspect, the present application provides a remote data reporting system for a remote database, the system comprising:

[0021] The data collection module is used to deploy a data collection program on the remote database server, collect data of preset data types in real time according to the preset sampling frequency, and add a timestamp to the collected data; send the collected data to the EMQX message server via the MQTT protocol; the collected data includes the timestamp and data type;

[0022] The topic matching module is used to maintain a sliding time window for each data type in the EMQX message server. When new collected data is received, the collected data is stored in the window queue of the sliding time window of the corresponding data type. The data change frequency of each data type within the sliding time window is calculated at fixed time intervals. Based on the data change frequency, the preset main structure template corresponding to the collected data is determined, and then stored according to the preset main structure template.

[0023] The dynamic adjustment module is used to update the collected data under the original preset main structure template to the new topic structure when any preset main structure template updates the topic structure, and notify the preset client to update the subscription.

[0024] In one implementation of the present application, the dynamic adjustment module includes a dynamic adjustment unit,

[0025] When any preset subject structure template updates the subject structure, a subject change notification is sent to the client subscribing to the subject corresponding to the original preset subject structure template; wherein the subject change notification includes a mapping relationship between the old preset subject structure template and the new preset subject structure template;

[0026] After the client receives the topic change notification but fails to respond in time, the EMQX message server automatically forwards the old subscription topic message to the new subscription topic corresponding to the new preset main structure template.

[0027] In one implementation of the present application, the topic matching module includes a topic matching unit for determining a corresponding four-level topic when the data change frequency is within a preset high frequency range; wherein the four-level topics are: remote library / device / type / real-time;

[0028] When the data change frequency is within the preset medium frequency range, the corresponding third-level theme is determined; the third-level theme is: remote library / device / type;

[0029] When the data change frequency is within a preset low-frequency range, the corresponding secondary theme is determined; wherein the secondary theme is: remote library / equipment.

[0030] In a third aspect, the present application provides a non-volatile computer storage medium on which computer instructions are stored. When the computer instructions are executed, they implement a remote data reporting method for an off-site library as described above.

[0031] It can be seen from the above technical solutions that this application has the following advantages:

[0032] By using a sliding time window to calculate the frequency of change for each data type in real time, dynamic perception of data characteristics is achieved. This automatically matches the optimal main structure template, resolving the issues of "high-frequency data loss" or "low-frequency data redundancy" caused by traditional fixed sampling strategies. This application reduces unnecessary data transmission (compared to static sampling solutions) while ensuring the complete reporting of critical data.

[0033] Through the dual identification of timestamp and data type, lossless migration of old template data to the new topic structure is completed, and the client is actively notified to update the subscription through the $SYS topic of EMQX to avoid manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for the description. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 This is a flow chart of a remote data reporting method for an off-site library provided in an embodiment of the present application.

[0036] Figure 2 This is a schematic diagram of the internal structure of a remote data reporting system for an off-site library provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] It should be understood by those skilled in the art that the embodiments described below are merely preferred embodiments of the present disclosure and do not imply that the present disclosure can only be implemented through these preferred embodiments. These preferred embodiments are merely intended to explain the technical principles of the present disclosure and are not intended to limit the scope of protection of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should still fall within the scope of protection of the present disclosure.

[0039] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0040] The technical solutions proposed in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0041] The embodiment provides a remote data reporting method for a remote database. Figure 1 As shown, the method provided in the embodiment of the present application mainly includes the following steps:

[0042] Step 110: Deploy a data collection program on the remote database server to collect data of the preset data type in real time according to the preset sampling frequency, and add a timestamp to the collected data; send the collected data to the EMQX message server via the MQTT protocol.

[0043] The collected data includes a timestamp and a data type.

[0044] In some embodiments, the data collection logic of the data collection program may specifically be:

[0045] Trigger data at preset sampling frequencies (e.g., tank level data collected every 5 seconds, basic equipment information collected at 1:00 AM daily). Also supports event triggering (e.g., active data collection when a sudden change in equipment status occurs).

[0046] Data processing:

[0047] Add a timestamp (accurate to milliseconds) and type identifier (such as basic information about tank level equipment) to each piece of data. Encapsulate the data into messages using the MQTT protocol and send them to the default topic of the EMQX server (such as raw_data / remote repository A).

[0048] Step 120: On the EMQX message server, a sliding time window is maintained for each data type. When new collected data is received, the collected data is stored in the window queue of the sliding time window corresponding to the data type. The data change frequency of each data type within the sliding time window is calculated at a fixed time interval. Based on the data change frequency, the preset main structure template corresponding to the collected data is determined, and then stored according to the preset main structure template.

[0049] The data change frequency of each data type within the sliding time window is calculated according to a fixed time interval, which can be specifically:

[0050] By formula:

[0051] Data change frequency = data change amount / time interval.

[0052] More specifically, the frequency calculation logic is:

[0053] Sliding window mechanism:

[0054] A 1-hour sliding time window (updated in real time, retaining only the most recent hour's data) is maintained for each data type (e.g., liquid level, equipment information). When new data arrives, it is queued for the corresponding window by type, and old data outside the window (e.g., data older than 1 hour) is discarded.

[0055] Frequency calculation trigger:

[0056] The calculation is automatically triggered every 10 minutes (configurable) to count the number of data changes within the window (consecutive reports of the same value are not counted as changes, only the latest value is recorded).

[0057] The preset main structure template corresponding to the collected data is determined according to the frequency of data changes, which can be specifically:

[0058] When the data change frequency is within the preset high frequency range, the corresponding four-level theme is determined; the four-level theme is: remote library / device / type / real-time;

[0059] When the data change frequency is within the preset medium frequency range, the corresponding third-level theme is determined; the third-level theme is: remote library / device / type;

[0060] When the data change frequency is within a preset low-frequency range, the corresponding secondary theme is determined; wherein the secondary theme is: remote library / equipment.

[0061] As an example, the following table shows:

[0062] Rule predefined mapping table

[0063]

[0064] It should also be added that the rule engine reads the frequency data one by one, matches the above rules according to the frequency value, and determines the corresponding topic structure template.

[0065] For example, if the liquid level frequency is 120 times / hour, the high-frequency rule is matched and the template remote warehouse A / storage tank 1 / liquid level / real-time is generated.

[0066] The conflict handling logic can be as follows: when different data types on the same device have significantly different frequencies (e.g., high frequency for liquid level and medium frequency for temperature), separate topics are generated for each data type, without affecting each other. Manual priority configuration is supported (e.g., forcing a certain data type to use a fixed topic structure).

[0067] In addition, the storage logic can be specifically as follows:

[0068] Storage logic table

[0069]

[0070] In addition, after determining the preset main structure template corresponding to the collected data and then storing it according to the preset main structure template, the method includes:

[0071] Specify QoS2 level for subscription / publishing operations of the four-level topic in the preset main structure template;

[0072] Specify QoS 0 for the subscription / publish operations of the secondary topic in the preset main structure template.

[0073] Those skilled in the art will understand that for high-frequency topics, QoS 2 (ensuring at least one reliable transmission) is mandatory and suitable for real-time data that cannot be lost (such as device alarms). For low-frequency topics, QoS 0 (best-effort transmission) is the default and reduces network bandwidth usage (such as daily device inspection records).

[0074] In addition, the compression strategy involved in the transmission process of this application can be specifically as follows:

[0075] Text data (such as logs): Use Zstandard compression (compression ratio of approximately 3:1) before sending, and automatically decompress it on the receiving end. Binary data (such as raw sensor values): Transmit directly without compression to reduce computational overhead.

[0076] Step 130: When any preset main structure template updates the subject structure, the collected data under the original preset main structure template is updated to the new subject structure, and the preset client is notified to update the subscription.

[0077] Before sending a topic change notification to a client subscribing to a topic corresponding to the original preset main structure template, the method further includes:

[0078] The preset rule engine obtains the updated subject structure of the preset main structure template, thereby triggering data migration.

[0079] More specifically:

[0080] 1. Old theme data migration logic:

[0081] Trigger conditions:

[0082] After the rule engine outputs the new topic structure, data migration is automatically triggered.

[0083] Migration steps:

[0084] Data retrieval: Query all historical data based on old topics (such as remote warehouse A / storage tank 1 / liquid level).

[0085] Data Reorganization: Extract information such as the remote repository name, device number, and data type from the data metadata. Reassemble the topic path according to the new topic structure (such as a four-level topic), for example: Remote Repository A / Tank 1 / Liquid Level / Real-time.

[0086] Storage update: Write the reorganized data to the storage location corresponding to the new topic (such as a new partition of a message queue or a new table in a database), while retaining the old topic data for a period of time (such as 24 hours) for disaster recovery.

[0087] 2. New topic creation and publishing logic:

[0088] Theme Generation:

[0089] Dynamically concatenate topic names (such as remote warehouse B / pump group 2 / pressure) based on the template and equipment identifier output by the rule engine.

[0090] EMQX interaction:

[0091] Create a new topic through the EMQX management API (such as POST / api / v5 / topics) and set the message retention time (such as 30 minutes for high-frequency topics and 7 days for low-frequency topics).

[0092] Example API call:

[0093] { "topic": "Remote Storage A / Storage Tank 1 / Liquid Level / Real-time", "retain_interval": 1800 / / Retention time (seconds)}.

[0094] Notifying the preset client to update the subscription includes:

[0095] When any preset subject structure template updates the subject structure, a subject change notification is sent to the client subscribing to the subject corresponding to the original preset subject structure template; wherein the subject change notification includes the mapping relationship between the old preset subject structure template and the new preset subject structure template;

[0096] After the client receives the topic change notification but fails to respond in time, the EMQX message server automatically forwards the old subscription topic message to the new subscription topic corresponding to the new preset main structure template.

[0097] In some embodiments, the execution unit sends a topic change notification to all clients subscribed to the old topic (via EMQX publishing a special message to the $SYS / topic / changes topic). The notification content includes the mapping between the old and new topics (e.g., {"old_topic": "Remote Storage A / Storage Tank 1 / Liquid Level", "new_topic": "Remote Storage A / Storage Tank 1 / Liquid Level / Real-time"}).

[0098] Client response:

[0099] The client monitors the $SYS / topic / changes topic and automatically executes the following actions upon receiving a notification: unsubscribe from the old topic (UNSUBSCRIBE Remote Repository A / Storage Tank 1 / Liquid Level) and subscribe to the new topic (SUBSCRIBE Remote Repository A / Storage Tank 1 / Liquid Level / Real-time).

[0100] If the client does not respond in time, EMQX will automatically forward the old topic message to the new topic (transition period is 5 minutes) to ensure that data is not lost.

[0101] Based on the above description, this application involves 1. Adaptive hierarchical acquisition system (step 110):

[0102] Dynamic triggering mechanism: A combination of timed triggering (e.g., 5-second liquid level acquisition) and event-driven triggering (active reporting of equipment mutations) improves data collection efficiency (compared to a purely timed solution).

[0103] Data identification standardization: Mandatory addition of millisecond-level timestamps and type identifiers solves the time synchronization problem of heterogeneous data sources (the error is reduced from seconds).

[0104] ‌MQTT protocol deep optimization‌: raw data aggregation is achieved through the default topic (raw_data / remote repository A) (an improvement over traditional HTTP transmission)

[0105] ‌2. Intelligent Frequency Sensing Engine (Step 120):‌

[0106] Sliding window dynamic calculation: The 1-hour window configuration reduces memory usage (compared to the full storage solution) while retaining sufficient time series features.

[0107] ‌Hierarchical Topic Mapping Rules‌:

[0108] Level 4 topics (real-time) correspond to Redis cache, with a write latency of <5ms;

[0109] The third-level topics (logs) are stored in MySQL shards, which improves query efficiency;

[0110] The secondary topic (archive) is connected to the Hive warehouse, and the compression ratio is improved.

[0111] ‌Intelligent transmission quality control‌:

[0112] QoS 2 ensures zero loss of high-frequency data;

[0113] Zstandard compression of text data reduces bandwidth consumption.

[0114] ‌3. Smooth migration of theme structure (step 130):‌

[0115] ‌Three stages of lossless data migration‌:

[0116] Metadata-driven reconstruction: Extract key fields through device number + type identification to improve reconstruction efficiency; Double-write transition strategy: Retain old topics for 24 hours to ensure zero data loss during the migration process; Storage engine adaptation: Automatically identify new topic levels and switch storage media (such as Redis to MySQL).

[0117] ‌Client-side seamless switching mechanism‌:

[0118] By broadcasting change instructions on the $SYS topic, subscription switching latency is reduced; 5-minute message automatic forwarding provides a safety net to ensure business continuity.

[0119] In addition, this application Figure 2 This invention provides a remote data reporting system for remote libraries. Figure 2 As shown, the system provided in the embodiment of the present application mainly includes:

[0120] The data acquisition module 210 is used to deploy a data acquisition program on the remote library server, and then collect data of a preset data type in real time according to a preset sampling frequency, and timestamp the collected data; send the collected data to the EMQX message server via the MQTT protocol; wherein the collected data includes a timestamp and data type.

[0121] The topic matching module 220 is used to maintain a sliding time window for each data type in the EMQX message server; when new collected data is received, the collected data is stored in the window queue of the sliding time window of the corresponding data type; at a fixed time interval, the data change frequency of each data type within the sliding time window is calculated; based on the data change frequency, the preset main structure template corresponding to the collected data is determined, and then stored according to the preset main structure template.

[0122] The topic matching module 220 includes a topic matching unit, which is used to determine the corresponding fourth-level topic when the data change frequency is in a preset high-frequency range; wherein the fourth-level topic is: off-site library / device / type / real-time; when the data change frequency is in a preset medium-frequency range, the corresponding third-level topic is determined; wherein the third-level topic is: off-site library / device / type; when the data change frequency is in a preset low-frequency range, the corresponding second-level topic is determined; wherein the second-level topic is: off-site library / device.

[0123] The dynamic adjustment module 230 is used to update the collected data under the original preset main structure template to the new theme structure when any preset main structure template updates the theme structure, and notify the preset client to update the subscription.

[0124] The dynamic adjustment module 230 includes a dynamic adjustment unit,

[0125] When any preset subject structure template updates the subject structure, it sends a subject change notification to the client that subscribed to the subject corresponding to the original preset subject structure template; wherein, the subject change notification contains the mapping relationship between the old preset subject structure template and the new preset subject structure template; after the client receives the subject change notification but fails to respond in time, the EMQX message server automatically forwards the old subscription subject message to the new subscription subject corresponding to the new preset subject structure template.

[0126] Based on the above description, this system uses a sliding time window to calculate the frequency of change of each data type in real time, achieving dynamic perception of data characteristics. Based on this, it automatically matches the optimal main structure template, solving the problems of "high-frequency data loss" or "low-frequency data redundancy" caused by traditional fixed sampling strategies. This application can reduce unnecessary data transmission (compared to static sampling solutions) while ensuring the complete reporting of critical data.

[0127] Through the dual identification of timestamp and data type, lossless migration of old template data to the new topic structure is completed, and the client is actively notified to update the subscription through the $SYS topic of EMQX to avoid manual intervention.

[0128] In addition, an embodiment of the present application further provides a non-volatile computer storage medium on which executable instructions are stored. When the executable instructions are executed, a remote data reporting method for an off-site library as described above is implemented.

[0129] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote data reporting method for a remote database, characterized in that: The method comprises: Deploy a data collection program on the remote database server to collect data of the preset data type in real time according to the preset sampling frequency and add a timestamp to the collected data; send the collected data to the EMQX message server via the MQTT protocol; the collected data includes the timestamp and data type; In the EMQX message server, a sliding time window is maintained for each data type. When new collected data is received, the collected data is stored in the window queue of the sliding time window of the corresponding data type. At fixed time intervals, the data change frequency of each data type within the sliding time window is calculated. Based on the data change frequency, the preset main structure template corresponding to the collected data is determined, and then stored according to the preset main structure template. Among them, according to the frequency of data changes, the preset main structure template corresponding to the collected data is determined, specifically including: when the data change frequency is in the preset high-frequency range, the corresponding four-level theme is determined; wherein, the four-level theme is: remote library / equipment / type / real-time; when the data change frequency is in the preset medium-frequency range, the corresponding three-level theme is determined; wherein, the three-level theme is: remote library / equipment / type; when the data change frequency is in the preset low-frequency range, the corresponding two-level theme is determined; wherein, the two-level theme is: remote library / equipment; When any preset main structure template updates the topic structure, the collected data under the original preset main structure template is updated to the new topic structure, and the preset client is notified to update the subscription; Among them, notifying the preset client to update the subscription specifically includes: when any preset main structure template updates the topic structure, sending a topic change notification to the client that subscribed to the topic corresponding to the original preset main structure template; wherein the topic change notification includes the mapping relationship between the old preset main structure template and the new preset main structure template; after the client receives the topic change notification but fails to respond in time, the EMQX message server automatically forwards the old subscription topic message to the new subscription topic corresponding to the new preset main structure template; Before sending a topic change notification to the client subscribing to the topic corresponding to the original preset main structure template, the method further includes: obtaining the updated topic structure of the preset main structure template through a preset rule engine, thereby triggering data migration.

2. The remote data reporting method of the remote database according to claim 1 is characterized in that: The data change frequency=data change amount / time interval.

3. The remote data reporting method of the remote database according to claim 1 is characterized in that: After determining the preset main structure template corresponding to the collected data and then storing it according to the preset main structure template, the method includes: Specify QoS2 level for subscription / publishing operations of the four-level topic in the preset main structure template; Specify QoS 0 for the subscription / publish operations of the secondary topic in the preset main structure template.

4. A remote data reporting system for remote libraries, characterized in that: The system comprises: The data collection module is used to deploy a data collection program on the remote database server, collect data of preset data types in real time according to the preset sampling frequency, and add a timestamp to the collected data; send the collected data to the EMQX message server via the MQTT protocol; the collected data includes the timestamp and data type; The topic matching module is used to maintain a sliding time window for each data type in the EMQX message server. When new collected data is received, the collected data is stored in the window queue of the sliding time window of the corresponding data type. The data change frequency of each data type within the sliding time window is calculated at fixed time intervals. Based on the data change frequency, the preset main structure template corresponding to the collected data is determined, and then stored according to the preset main structure template. The topic matching module includes a topic matching unit, which is used to determine the corresponding fourth-level topic when the data change frequency is within a preset high-frequency range; the fourth-level topic is: remote library / device / type / real-time; when the data change frequency is within a preset medium-frequency range, the corresponding third-level topic is determined; the third-level topic is: remote library / device / type; when the data change frequency is within a preset low-frequency range, the corresponding second-level topic is determined; the second-level topic is: remote library / device; Dynamic adjustment module, used for updating the collected data under the original preset main structure template to the new theme structure when any preset main structure template updates the theme structure, and notifying the preset client to update the subscription; The dynamic adjustment module includes a dynamic adjustment unit, which is used to send a topic change notification to the client that subscribed to the topic corresponding to the original preset main structure template when any preset main structure template updates the topic structure; wherein the topic change notification includes the mapping relationship between the old preset main structure template and the new preset main structure template; if the client receives the topic change notification but fails to respond in time, the EMQX message server automatically forwards the old subscription topic message to the new subscription topic corresponding to the new preset main structure template; Before sending a topic change notification to the client subscribing to the topic corresponding to the original preset main structure template, the dynamic adjustment module further includes: obtaining the updated topic structure of the preset main structure template through a preset rule engine, thereby triggering data migration.

5. A non-volatile computer storage medium, characterized in that Computer instructions are stored thereon, and when the computer instructions are executed, they implement a remote data reporting method for an off-site library as described in any one of claims 1-3.

Images