Data transmission method and system of data gateway

Abstract

The invention provides a data transmission method and system of a data gateway, and aims to solve the problems that in the prior art, data in a database is randomly modified due to the fact that an encrypted account enters a management control background of the data gateway, data storage is disordered, and data security is affected. The method comprises the steps of collecting data from a data source; carrying out preprocessing and format conversion on the collected data; caching the data subjected to format conversion into a database of a data gateway; when a user accesses the database of the data gateway through the management control background, user verification is carried out firstly; after verification is passed, if the user modifies the data stored in the database, data files before and after modification are stored respectively, modification information of the user is recorded, and then user login information is logged out; and when data transmission needs to be carried out, extracting data to be transmitted from the database of the data gateway, and carrying out encrypted transmission on the data to be transmitted according to a preset transmission protocol and a preset target address.

Description

Technical Field

[0001] The present invention relates to the field of data domain data gateways, and in particular to a data transmission method and system for a data gateway. Background Art

[0002] In today's digital age, various devices and systems generate a large amount of different types of data, which need to be efficiently and reliably transmitted to different destinations for processing and analysis. Data gateways can facilitate transmission. During transmission, data gateways can temporarily store data in a database and save it. When it is needed, the data gateway can extract the data and send it to a designated location.

[0003] However, the data in the database can be modified. When many newly modified data are stored, the entire data storage path is messy. Therefore, when the encrypted account enters the management control background, the data backup may be modified by others, and the modified data cannot be retrieved in time, which affects data security. Summary of the invention

[0004] In order to solve the problem in the prior art that an encrypted account enters the management control background of a data gateway and arbitrarily modifies data in a database, causing confusion in stored data and affecting data security, the present invention provides a data transmission method and system for a data gateway.

[0005] The technical solution of the present invention is: The present invention provides a data transmission method of a data gateway, comprising: Collect data from a variety of data sources; Preprocess the collected data and convert the preprocessed data into different formats; Cache the format-converted data in the database of the data gateway; When a user accesses the database of the data gateway through the management control background, user verification is first performed; after verification, if the user has modified the data stored in the database, the new data file obtained after the modification and the original data file before the modification are stored separately, and the user's modification information is recorded, and then the user login information is logged out; When data transmission is required, the data to be transmitted is extracted from the database of the data gateway, and the data to be transmitted is encrypted and transmitted according to the preset transmission protocol and the preset target address; If the user has modified the data, the data to be transmitted is the new data file obtained after the user's modification.

[0006] Preferably, the step of recording the user's modification information includes: Record user information, access time, modification time and modification content.

[0007] Preferably, the step of encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: Perform data integrity check on the data to be transmitted, and use a retransmission mechanism to encrypt the data to be transmitted.

[0008] Preferably, the step of encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: Prioritize the transmission of data with high real-time requirements and / or high priority.

[0009] Preferably, the steps of preprocessing the collected data and converting the preprocessed data in different formats include: Clean and filter the collected data; After cleaning and filtering, data in different formats are converted into a unified data format.

[0010] On the other hand, the present invention also provides a data transmission system of a data gateway, comprising: A data collection module, used to collect data from various data sources; The data preprocessing module is used to preprocess the collected data and convert the preprocessed data into different formats; The data cache module is used to cache the format-converted data into the database of the data gateway; The data security module is used to verify the user when the user accesses the database of the data gateway through the management control background. After the verification is passed, if the user has modified the data stored in the database, the new data file obtained after the modification and the original data file before the modification are stored separately, and the user's modification information is recorded, and then the user login information is logged out; The data transmission module is used to extract the data to be transmitted from the database of the data gateway when data transmission is required, and encrypt and transmit the data to be transmitted according to the preset transmission protocol and the preset target address; If the user has modified the data, the data to be transmitted is the new data file obtained after the user's modification.

[0011] Preferably, in the data security module, recording the user's modification information includes: Record user information, access time, modification time and modification content.

[0012] Preferably, in the data transmission module, encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: Perform data integrity check on the data to be transmitted, and use a retransmission mechanism to encrypt the data to be transmitted.

[0013] Preferably, in the data transmission module, encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: Prioritize the transmission of data with high real-time requirements and / or high priority.

[0014] Preferably, in the data preprocessing module, preprocessing the collected data and converting the preprocessed data in different formats includes: Clean and filter the collected data; After cleaning and filtering, data in different formats are converted into a unified data format.

[0015] The beneficial effects of the present invention are: When users access the database of the data gateway through the management control background, if the user modifies the data in the database, the original data file before the modification is backed up and stored, and the user's modification information is recorded; when an error occurs due to the modification, the source of the modification can be quickly traced, which facilitates timely locating the modifier and data recovery through the backed-up original data file. In addition, the intermediate links of data conversion and transmission are reduced, the efficiency of data transmission is improved, and data verification, retransmission mechanism and encryption technology are also used to ensure the reliability and security of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of the data transmission method of the data gateway in the embodiment of the present application; Figure 2 This is a structural block diagram of the data transmission system of the data gateway. DETAILED DESCRIPTION

[0017] The method of the present invention is further described below in conjunction with the embodiments and accompanying drawings. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0018] Reference Figure 1 , the embodiment of the present application provides a data transmission method of a data gateway, comprising: S1, collect data from multiple data sources; S2, preprocessing the collected data and converting the preprocessed data in different formats; S3, caches the format-converted data in the database of the data gateway; S4, when the user accesses the database of the data gateway through the management control background, the user is first authenticated; after the authentication is passed, if the user has modified the data stored in the database, the new data file obtained after the modification and the original data file before the modification are stored separately, and the user's modification information is recorded, and then the user login information is logged out; S5, when data transmission is required, extract the data to be transmitted from the database of the data gateway, and encrypt and transmit the data to be transmitted according to the preset transmission protocol and the preset target address; If the user has modified the data, the data to be transmitted is the new data file obtained after the user's modification.

[0019] In step S1, data is collected from various data sources. The data sources in the embodiment of the present application include but are not limited to sensors, devices and databases. The data acquisition module supports multiple data interfaces and protocols to realize data collection from multiple data sources. The data acquisition module can automatically collect data according to the preset collection frequency and trigger conditions, that is, the data acquisition module obtains data from the data source by means of timed collection, event triggering, etc.

[0020] In step S2, the steps of preprocessing the collected data and converting the preprocessed data in different formats include: Clean and filter the collected data; After cleaning and filtering, data in different formats are converted into a unified data format.

[0021] Among them, the cleaning and filtering of the collected data specifically refers to: removing noise, outliers and invalid data.

[0022] In the embodiment of the present application, the data gateway is a gateway device that can realize data conversion and transmission between different networks, support multiple communication protocols, have data caching function, and can perform intelligent scheduling according to the priority and real-time requirements of the data. The gateway device can automatically adjust the transmission parameters according to the network status and data importance.

[0023] The management and control background is used to enable users to configure, monitor and manage the data gateway system, provide a visual interface, support users to remotely manage and configure, and regularly maintain and upgrade the system. Users can view data collection, transmission status and equipment operation in real time through the management and control background.

[0024] The management control background can provide visitors and managers with login systems. Both visitors and managers need to undergo identity authentication. Visitors are only allowed to access the data in the database but cannot modify it; managers are allowed to modify the data in the database.

[0025] In step S4, the step of recording the user's modified information includes: Record user information, access time, modification time and modification content.

[0026] In step S5, the step of encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: Perform data integrity check on the data to be transmitted, and use a retransmission mechanism to encrypt the data to be transmitted.

[0027] In step S5, the step of encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: The data to be transmitted with high real-time requirements and / or high priority is transmitted first. The above method of the present invention can realize the unified collection, processing and transmission of different types of data, reduce the intermediate links of data conversion and transmission, improve the efficiency of data transmission, and also adopt data verification, retransmission mechanism and encryption technology to ensure the reliability and security of data.

[0028] In addition, the above method of the present invention verifies the user who needs to access the database of the data gateway. When the user has completed accessing the data, it is necessary to determine whether the user has modified the data; when the data has not been modified, the user's login information automatically exits; when the data is modified, the user's modified information is stored and recorded, and the user's access time, modification time, modification content and backup of the original data file are stored; when the user's modified information is stored, the user's login information automatically exits. This method ensures that after each data modification is completed, it can be traced back to the source of the modification, which is convenient for timely finding the modifier, and for those data with incorrect file modification, the backup file is convenient for restoring the data to the initial state.

[0029] Reference Figure 2 On the other hand, the present invention also provides a data transmission system of a data gateway, comprising: A data collection module, used to collect data from various data sources; The data preprocessing module is used to preprocess the collected data and convert the preprocessed data into different formats; The data cache module is used to cache the format-converted data into the database of the data gateway; The data security module is used to verify the user when the user accesses the database of the data gateway through the management control background. After the verification is passed, if the user has modified the data stored in the database, the new data file obtained after the modification and the original data file before the modification are stored separately, and the user's modification information is recorded, and then the user login information is logged out; The data transmission module is used to extract the data to be transmitted from the database of the data gateway when data transmission is required, and encrypt and transmit the data to be transmitted according to the preset transmission protocol and the preset target address; If the user has modified the data, the data to be transmitted is the new data file obtained after the user's modification.

[0030] Preferably, in the data security module, recording the user's modification information includes: Record user information, access time, modification time and modification content.

[0031] Preferably, in the data transmission module, encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: Perform data integrity check on the data to be transmitted, and use a retransmission mechanism to encrypt the data to be transmitted.

[0032] Preferably, in the data transmission module, encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: Prioritize the transmission of data with high real-time requirements and / or high priority.

[0033] Preferably, in the data preprocessing module, preprocessing the collected data and converting the preprocessed data in different formats includes: Clean and filter the collected data; After cleaning and filtering, data in different formats are converted into a unified data format.

[0034] When users access the database of the data gateway through the management control background, if the user modifies the data in the database, the original data file before the modification is backed up and stored, and the user's modification information is recorded; when an error occurs due to the modification, the source of the modification can be quickly traced, which facilitates timely locating the modifier and data recovery through the backed-up original data file. In addition, the intermediate links of data conversion and transmission are reduced, the efficiency of data transmission is improved, and data verification, retransmission mechanism and encryption technology are also used to ensure the reliability and security of data transmission.

[0035] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0036] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0037] It should also be noted that, in this article, the orientation or position relationship indicated by the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, relational terms such as "first" and "second" are used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations, nor can they be understood as indicating or implying relative importance. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements does not include those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0038] The technical solution provided by the present invention is described in detail above. The principle and implementation mode of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the present invention, and the content of this specification should not be understood as limiting the present invention. At the same time, for those skilled in the art, according to the present invention, there will be different forms of changes in the specific implementation mode and application scope. It is not necessary and impossible to list all the implementation modes here, and the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A data transmission method for a data gateway, characterized in that: include: Collect data from a variety of data sources; Preprocess the collected data and convert the preprocessed data into different formats; Cache the format-converted data in the database of the data gateway; When a user accesses the database of the data gateway through the management control background, user verification is first performed; after verification, if the user has modified the data stored in the database, the new data file obtained after the modification and the original data file before the modification are stored separately, and the user's modification information is recorded, and then the user login information is logged out; When data transmission is required, the data to be transmitted is extracted from the database of the data gateway, and the data to be transmitted is encrypted and transmitted according to the preset transmission protocol and the preset target address; If the user has modified the data, the data to be transmitted is the new data file obtained after the user's modification.

2. The data transmission method of the data gateway according to claim 1, characterized in that: The steps of recording the user's modified information include: Record user information, access time, modification time and modification content.

3. The data transmission method of the data gateway according to claim 1, characterized in that: The steps of encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address include: Perform data integrity check on the data to be transmitted, and use a retransmission mechanism to encrypt the data to be transmitted.

4. The data transmission method of the data gateway according to claim 3, characterized in that: The steps of encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address include: Prioritize the transmission of data with high real-time requirements and / or high priority.

5. The data transmission method of the data gateway according to claim 1, characterized in that: The steps of preprocessing the collected data and converting the preprocessed data into different formats include: Clean and filter the collected data; After cleaning and filtering, data in different formats are converted into a unified data format.

6. A data transmission system for a data gateway, characterized in that: include: A data collection module, used to collect data from various data sources; The data preprocessing module is used to preprocess the collected data and convert the preprocessed data into different formats; The data cache module is used to cache the format-converted data into the database of the data gateway; The data security module is used to verify the user when the user accesses the database of the data gateway through the management control background. After the verification is passed, if the user has modified the data stored in the database, the new data file obtained after the modification and the original data file before the modification are stored separately, and the user's modification information is recorded, and then the user login information is logged out; The data transmission module is used to extract the data to be transmitted from the database of the data gateway when data transmission is required, and encrypt and transmit the data to be transmitted according to the preset transmission protocol and the preset target address; If the user has modified the data, the data to be transmitted is the new data file obtained after the user's modification.

7. The data transmission system of the data gateway according to claim 6, characterized in that: In the data security module, the user's modification information is recorded including: Record user information, access time, modification time and modification content.

8. The data transmission system of the data gateway according to claim 6, characterized in that: In the data transmission module, encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: Perform data integrity check on the data to be transmitted, and use a retransmission mechanism to encrypt the data to be transmitted.

9. The data transmission system of the data gateway according to claim 8, characterized in that: In the data transmission module, encrypting and transmitting the data to be transmitted according to the preset transmission protocol and the preset target address includes: Prioritize the transmission of data with high real-time requirements and / or high priority.

10. The data transmission system of the data gateway according to claim 6, characterized in that: In the data preprocessing module, the collected data is preprocessed and the format conversion of the preprocessed data in different formats is performed, including: Clean and filter the collected data; After cleaning and filtering, data in different formats are converted into a unified data format.

Images