A gateway data interaction system and method

Abstract

This invention relates to the field of gateway data and discloses a gateway data interaction system and method, including a data acquisition module, a communication module, a data processing module, a storage module, and a remote communication module. The data acquisition module is responsible for collecting data from various sensors, devices, or data sources. The communication module is used to manage communication with external devices, sensors, or remote servers. The data processing module is used to analyze, process, and transform the data collected from sensors. The storage module is used to store the processed data locally and automatically adjust the data storage strategy according to the network connection status and storage space. The remote communication module is used to communicate with remote servers or cloud platforms. This invention has the advantages of more intelligent, efficient, and secure data interaction and application.

Description

Technical Field

[0001] This invention relates to the field of gateway data, specifically to an interactive system and method for gateway data. Background Technology

[0002] The gateway data interaction field has faced numerous challenges in the past, including but not limited to low data transmission efficiency, poor security, insufficient data processing capabilities, unstable network topology, and data redundancy and corruption. Traditional data interaction methods are often limited by the choice of communication protocols, network topology design, and security protection capabilities, leading to unstable data transmission, vulnerability to attacks, and difficulty in management. In the past, solutions in the gateway data interaction field mainly focused on improving transmission efficiency, enhancing data security, and optimizing data processing workflows. This includes improving the design of transmission protocols, applying encryption technologies, optimizing network topology, and using data compression and deduplication techniques.

[0003] However, with the continuous advancement and innovation of science and technology, the field of gateway data interaction is also gradually developing and improving. The emergence of new technologies and methods, such as edge computing, blockchain, and artificial intelligence, provides new ideas and solutions for addressing bottlenecks in traditional data interaction. The application of these technologies makes gateway data interaction more efficient, intelligent, and secure, laying a solid foundation for the future development of data interaction. Therefore, it is essential to design a more intelligent, efficient, and secure gateway data interaction system and method. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] To address the shortcomings of existing technologies, this invention provides a gateway data interaction system and method that offers advantages such as more intelligent, efficient, and secure data interaction and application, thus solving the problems mentioned in the background technology.

[0006] (II) Technical Solution

[0007] To achieve the aforementioned goals of more intelligent, efficient, and secure data interaction and application, this invention provides the following technical solution: a gateway data interaction system and method, comprising a data acquisition module, a communication module, a data processing module, a storage module, and a remote communication module. The data acquisition module is responsible for collecting data from various sensors, devices, or data sources, converting the collected data into a unified data format, acquiring data in real time through interfaces with various sensors and devices, and performing data acquisition according to preset sampling frequencies or trigger conditions. It dynamically adjusts the sampling frequency or selectively acquires data based on real-time changes in sensor data and system load. The communication module manages communication with external devices, sensors, or remote servers, dynamically adjusting communication parameters based on communication quality and data processing requirements. The data processing module analyzes, processes, and transforms the data collected from sensors. The storage module stores the processed data locally, automatically adjusting data storage strategies based on network connection status and storage space availability. The remote communication module communicates with remote servers or cloud platforms, enabling remote viewing and operation of devices through mobile applications or web interfaces.

[0008] According to the above technical solution, the data acquisition module includes an interface management module, a data parsing module, and an acquisition control module. The interface management module is used to manage the communication interfaces with various sensors, devices, or data sources. The data parsing module is used to parse and process the raw data obtained from the sensors, devices, or data sources. The acquisition control module is used to manage the control logic and strategy of data acquisition, and control the start and stop of data acquisition and the adjustment of the sampling frequency according to the preset sampling frequency, triggering conditions, or priority.

[0009] According to the above technical solution, the communication module includes a communication protocol processing module and a network communication module. The communication protocol processing module is responsible for processing the communication protocol with external devices or systems, and implementing corresponding protocol parsing and processing logic according to different communication protocols. The network communication module is used to manage network connections with remote servers or other devices, and is responsible for establishing, maintaining and closing network connections, including physical layer and transport layer communication, processing data transmission and reception, including network configuration, connection status management, and handling network anomalies.

[0010] According to the above technical solution, the dynamic adjustment of communication parameters based on communication quality and data processing requirements includes calculating the average values ​​of network conditions, equipment characteristics, and communication objectives as a comprehensive indicator of the overall network situation; calculating the comprehensive complexity of the communication protocol, including the average values ​​of communication protocol complexity C, reliability E, and bandwidth utilization B; subtracting this from 1 to obtain the adaptability of the communication protocol; and multiplying the comprehensive indicator of the overall network situation by the adaptability of the communication protocol to obtain the adaptability score S of the communication protocol, expressed by the formula:

[0011]

[0012] Where N represents the stability and congestion level of the network, D represents the performance and power consumption of the device, P represents the importance and urgency of the communication, C represents the processing complexity of the protocol, E represents the signal stability and fault tolerance of the protocol, and B represents the bandwidth utilization efficiency of the protocol.

[0013] According to the above technical solution, the data processing module is used to process the raw sensor data and effectively identify and process outliers. First, a sliding window average is applied to the raw data D(t) to obtain the average value μ(t):

[0014]

[0015] Then, calculate the standard deviation σ(t) of the data within the current window:

[0016]

[0017] Where D(t) represents the raw sensor data, N represents the size of the sliding window used for averaging, μ(t) represents the mean of the data within the current window, and σ(t) represents the standard deviation of the data within the current window. Next, a weighted moving average is applied to smooth the data, and the standard deviation is used for anomaly detection. If the deviation of the current data point exceeds the threshold k*σ(t), where k represents the threshold used for anomaly detection, and T(t) represents the processed data, it is marked as an outlier; otherwise, the original data is retained. Finally, the processed data T(t) is output, where outliers have been marked or removed.

[0018] According to the above technical solution, the storage module includes a storage management module and a backup module. The storage management module monitors the usage of storage space, allocates and reclaims storage resources, and manages the lifecycle of data storage. The backup module periodically backs up the stored data to ensure data security and integrity. It also provides data recovery functionality, enabling rapid data recovery in case of loss or corruption. Furthermore, it designs intelligent storage management algorithms to dynamically manage data storage locations and strategies based on data importance, access frequency, and storage requirements.

[0019] According to the above technical solution, the intelligent storage management algorithm includes calculating the combined impact of data importance, access frequency, and storage demand, subtracting the impact of storage resource availability on storage location to obtain a base score, dividing the base score by the product of storage cost, storage utilization, and storage scalability, and adding the impact of data complexity on storage demand and access frequency (a lower value indicates more complex data, lower storage demand, and higher access frequency), subtracting the impact of data timeliness on storage demand and access frequency (a higher value indicates more timely data and higher access frequency), and adding the impact of data redundancy on storage demand and access frequency (a higher value indicates higher data redundancy, lower storage demand, and higher access frequency). Finally, the storage priority P is obtained, with a higher value indicating higher data storage priority, expressed by the formula:

[0020]

[0021] Where I represents the importance of the data, F represents the efficiency of data access, R represents the storage requirement, A represents the availability of storage resources, S represents the security of the storage location, C represents the storage cost, D represents the complexity of the data, T represents the timeliness of the data, E represents the redundancy of the data, U represents the storage utilization rate, and V represents the scalability of the storage.

[0022] According to the above technical solution, the method for interacting with gateway data includes the following steps:

[0023] S1: The gateway needs to collect data from various sensors, devices, or data sources;

[0024] S2: After collecting the data, the gateway performs data processing operations, including data cleaning, noise reduction, and format conversion;

[0025] S3: The processed data needs to be transmitted to the designated target via the network. The gateway uses the communication module to encapsulate the data and send it out through the network transmission protocol.

[0026] S4: After receiving the data, the receiving device or server parses and processes the data, and performs corresponding processing and storage according to the data format and content;

[0027] S5: The collected data needs to be stored and managed long-term for subsequent analysis and application. The data should be stored on local storage devices or uploaded to cloud storage systems for backup and management.

[0028] (III) Beneficial Effects

[0029] Compared with the prior art, the present invention provides a gateway data interaction system and method, which has the following beneficial effects:

[0030] The gateway data interaction system and method dynamically adjust communication parameters based on communication quality and data processing requirements. These parameters include calculating the average values ​​of network status, device characteristics, and communication objectives as a comprehensive indicator of the overall network condition. The system also calculates the overall complexity of the communication protocol, processes raw sensor data, and effectively identifies and handles outliers. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Please see Figure 1 This invention provides a technical solution: an interactive system and method for gateway data, comprising a data acquisition module, a communication module, a data processing module, a storage module, and a remote communication module. The data acquisition module is responsible for collecting data from various sensors, devices, or data sources, converting the collected data into a unified data format, acquiring data in real time through interfaces with various sensors and devices, and performing data acquisition according to preset sampling frequencies or trigger conditions. It dynamically adjusts the sampling frequency or selectively acquires data based on real-time changes in sensor data and system load. The communication module manages communication with external devices, sensors, or remote servers, dynamically adjusting communication parameters based on communication quality and data processing requirements. The data processing module analyzes, processes, and transforms the data collected from sensors. The storage module stores the processed data locally, automatically adjusting data storage strategies based on network connection status and storage space availability. The remote communication module communicates with remote servers or cloud platforms, enabling remote viewing and operation of devices through mobile applications or web interfaces.

[0034] The data acquisition module includes an interface management module, a data parsing module, and an acquisition control module. The interface management module is used to manage the communication interfaces with various sensors, devices, or data sources. The data parsing module is used to parse and process the raw data obtained from the sensors, devices, or data sources. The acquisition control module is used to manage the control logic and strategy of data acquisition, and control the start and stop of data acquisition and the adjustment of the sampling frequency according to the preset sampling frequency, triggering conditions, or priority.

[0035] The communication module includes a communication protocol processing module and a network communication module. The communication protocol processing module is responsible for processing communication protocols with external devices or systems, and implementing corresponding protocol parsing and processing logic according to different communication protocols. The network communication module is used to manage network connections with remote servers or other devices, and is responsible for establishing, maintaining and closing network connections, including physical layer and transport layer communication, processing data transmission and reception, including network configuration, connection status management, and handling network anomalies.

[0036] Based on communication quality and data processing requirements, communication parameters are dynamically adjusted. This includes calculating the average values ​​of network conditions, equipment characteristics, and communication objectives as a comprehensive indicator of the overall network situation. The comprehensive complexity of the communication protocol is calculated, including the average values ​​of protocol complexity (C), reliability (E), and bandwidth utilization (B). Subtracting this average from 1 yields the protocol's adaptability. Multiplying the comprehensive indicator of the overall network situation by the protocol's adaptability gives the protocol's adaptability score (S), expressed by the formula:

[0037]

[0038] Where N represents the stability and congestion level of the network, D represents the performance and power consumption of the device, P represents the importance and urgency of the communication, C represents the processing complexity of the protocol, E represents the signal stability and fault tolerance of the protocol, and B represents the bandwidth utilization efficiency of the protocol.

[0039] The storage module includes a storage management module and a backup module. The storage management module monitors storage space usage, allocates and reclaims storage resources, and manages the lifecycle of data storage. The backup module periodically backs up stored data to ensure data security and integrity. It also provides data recovery functionality to quickly recover data in case of loss or corruption. Furthermore, it incorporates intelligent storage management algorithms that dynamically manage data storage locations and strategies based on data importance, access frequency, and storage requirements.

[0040] Intelligent storage management algorithms consider the combined impact of data importance, access frequency, and storage demand. Subtracting the impact of storage resource availability on storage location yields a base score. This base score is then divided by the product of storage cost, storage utilization, and storage scalability. Adding the impact of data complexity on storage demand and access frequency (a lower value indicates more complex data, lower storage demand, and higher access frequency), subtracting the impact of data timeliness on storage demand and access frequency (a higher value indicates more timely data and higher access frequency), and adding the impact of data redundancy on storage demand and access frequency (a higher value indicates higher data redundancy, lower storage demand, and higher access frequency). Finally, the storage priority P is obtained; a higher value indicates higher data storage priority, expressed by the formula:

[0041]

[0042] Where I represents the importance of the data, F represents the efficiency of data access, R represents the storage requirement, A represents the availability of storage resources, S represents the security of the storage location, C represents the storage cost, D represents the complexity of the data, T represents the timeliness of the data, E represents the redundancy of the data, U represents the storage utilization rate, and V represents the scalability of the storage.

[0043] One type of network S1: The gateway needs to collect data from various sensors, devices or data sources;

[0044] S2: After collecting the data, the gateway performs data processing operations, including data cleaning, noise reduction, and format conversion;

[0045] S3: The processed data needs to be transmitted to the designated target via the network. The gateway uses the communication module to encapsulate the data and send it out through the network transmission protocol.

[0046] S4: After receiving the data, the receiving device or server parses and processes the data, and performs corresponding processing and storage according to the data format and content;

[0047] S5: The collected data needs to be stored and managed long-term for subsequent analysis and application. This involves storing the data on local storage devices or uploading it to a cloud storage system for backup and management. The data interaction methods include the following steps:

[0048] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0049] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A gateway data interaction system, comprising a data acquisition module, a communication module, a data processing module, a storage module, and a remote communication module, characterized in that, The data acquisition module is responsible for collecting data from various sensors, devices, or data sources, converting the data into a unified data format, and acquiring data in real time through interfaces with various sensors and devices. Data acquisition is performed according to preset sampling frequencies or trigger conditions, and the sampling frequency is dynamically adjusted or selective data acquisition is performed based on real-time changes in sensor data and system load. The communication module manages communication with external devices, sensors, or remote servers, dynamically adjusting communication parameters according to communication quality and data processing requirements. Specifically, it calculates the average values ​​of network status, device characteristics, and communication purpose as a comprehensive indicator of the overall network condition; calculates the comprehensive complexity of the communication protocol, including the average values ​​of communication protocol complexity C, reliability E, and bandwidth utilization B, and subtracts this from 1 to obtain the adaptability of the communication protocol; multiplying the comprehensive indicator of the overall network condition by the adaptability of the communication protocol yields the adaptability score S, expressed by the formula: Where N represents network stability and congestion level, D represents device performance and power consumption, P represents the importance and urgency of communication, C represents protocol processing complexity, E represents protocol signal stability and fault tolerance, and B represents protocol bandwidth utilization efficiency. The data processing module analyzes, processes, and transforms data collected from sensors. The storage module stores the processed data locally and automatically adjusts the data storage strategy based on network connection status and storage space availability. Specifically, it calculates the combined impact of data importance, access frequency, and storage requirements, subtracts the impact of storage resource availability on storage location, and obtains... A base score is calculated by dividing the base score by the product of storage cost, storage utilization, and storage scalability, plus the impact of data complexity on storage requirements and access frequency. A lower base score indicates more complex data, lower storage requirements, and higher access frequency. The impact of data timeliness on storage requirements and access frequency is subtracted; a higher timeliness indicates more timely data and higher access frequency. The impact of data redundancy on storage requirements and access frequency is also added; a higher redundancy indicates higher data redundancy, lower storage requirements, and higher access frequency. This results in the storage priority P, where a higher P indicates higher storage priority. The formula is as follows: Wherein, I represents the importance of the data, F represents the data access efficiency, R represents the storage requirement, A represents the availability of storage resources, S represents the security of the storage location, C represents the storage cost, D represents the complexity of the data, T represents the timeliness of the data, E represents the redundancy of the data, U represents the storage utilization rate, and V represents the scalability of the storage. The remote communication module is used to communicate with a remote server or cloud platform and to remotely view and operate the device through a mobile application or web interface.

2. The gateway data interaction system according to claim 1, characterized in that, The data acquisition module includes an interface management module, a data parsing module, and an acquisition control module. The interface management module is used to manage the communication interfaces with various sensors, devices, or data sources. The data parsing module is used to parse and process the raw data obtained from the sensors, devices, or data sources. The acquisition control module is used to manage the control logic and strategy of data acquisition, and control the start and stop of data acquisition and the adjustment of the sampling frequency according to the preset sampling frequency, triggering conditions, or priority.

3. The gateway data interaction system according to claim 1, characterized in that, The communication module includes a communication protocol processing module and a network communication module. The communication protocol processing module is responsible for processing communication protocols with external devices or systems, and implementing corresponding protocol parsing and processing logic according to different communication protocols. The network communication module is used to manage network connections with remote servers or other devices, and is responsible for establishing, maintaining and closing network connections, including physical layer and transport layer communication, processing data transmission and reception, including network configuration, connection status management, and handling network anomalies.

4. The gateway data interaction system according to claim 1, characterized in that, The data processing module is used to process raw sensor data and effectively identify and process outliers. First, a sliding window average is applied to the raw data D(t) to obtain the average value μ(t): Then, calculate the standard deviation σ(t) of the data within the current window: Where D(t) represents the original sensor data, N represents the size of the sliding window used for averaging, α represents the attenuation factor used for weighted moving average, k represents the threshold used for anomaly detection, T(t) represents the processed data, μ(t) represents the mean of the data within the current window, and σ(t) represents the standard deviation of the data within the current window. Next, a weighted moving average is applied to smooth the data, and anomaly detection is performed in conjunction with the standard deviation. If the deviation of the current data point exceeds the threshold k*σ(t), it is marked as an outlier; otherwise, the original data is retained. Finally, the processed data T(t) is output, where outliers have been marked or removed.

5. The gateway data interaction system according to claim 1, characterized in that, The storage module includes a storage management module and a backup module. The storage management module is used to monitor the usage of storage space, allocate and reclaim storage resources, and manage the lifecycle of data storage. The backup module is used to back up the stored data regularly to ensure the security and integrity of the data; at the same time, it provides data recovery functions to quickly recover data when it is lost or damaged. It also designs intelligent storage management algorithms to dynamically manage data storage locations and strategies based on the importance of the data, access frequency, and storage requirements.

6. A gateway data interaction method, applied to the gateway data interaction system as described in any one of claims 1-5, characterized in that, Includes the following steps: S1: The gateway needs to collect data from various sensors, devices, or data sources; S2: After collecting the data, the gateway performs data processing operations, including data cleaning, noise reduction, and format conversion; S3: The processed data needs to be transmitted to the designated target via the network. The gateway uses the communication module to encapsulate the data and send it out through the network transmission protocol. S4: After receiving the data, the receiving device or server parses and processes the data, and performs corresponding processing and storage according to the data format and content; S5: The collected data needs to be stored and managed long-term for subsequent analysis and application. The data should be stored on local storage devices or uploaded to cloud storage systems for backup and management.

Images