Data transmission method, system and device and storage medium

Abstract

The invention discloses a data transmission method, system and device and a storage medium. The method comprises the following steps: determining a corresponding equipment behavior according to equipment protocol parameters in the south direction of a gateway; determining a first gateway configuration in the south direction of a gateway based on a protocol specification according to the equipment behavior; wherein the first gateway configuration comprises equipment online mode configuration, data acquisition mode configuration, TCP packet processing configuration and multi-round interaction configuration; determining a second gateway configuration of the north direction of the gateway according to a push data format of the north direction of the gateway; according to the first gateway configuration, adjusting the format of the to-be-uploaded data reported by the equipment to be an internal data format, and according to the second gateway configuration, adjusting the internal data format to be a service data format; and uploading the to-be-uploaded data in the business data format to a cloud platform. According to the invention, data configuration of multiple service scenes can be realized, the data transmission efficiency and accuracy can be improved, and the method can be widely applied to the technical field of Internet of Things.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a data transmission method, system, device and storage medium. Background Art

[0002] In IoT applications, migrating terminal devices to the cloud via a gateway is a common method. The vast number of existing and incremental devices varies greatly, using a wide variety of communication protocols. Most are application-layer protocols based on TCP and UDP, including common protocols, industry-standard protocols, and a large number of proprietary protocols. Protocol integration requires extensive development work. When connecting gateways to cloud platforms, the MQTT protocol, with its lightweight, low power consumption, and support for message subscription and publication models, has become a common protocol for communication between IoT devices and the cloud.

[0003] In related technologies, traditional gateway devices often lack flexible configuration capabilities and are unable to configure device data for multiple business requirements, resulting in low efficiency and accuracy of data transmission. Summary of the Invention

[0004] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0005] To this end, an object of the present invention is to provide an efficient data transmission method, system, device and storage medium.

[0006] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include: On the one hand, an embodiment of the present invention provides a data transmission method, comprising the following steps: determining the corresponding device behavior according to the device protocol parameters of the southbound gateway; wherein the device protocol parameters include connection behavior, data interaction behavior, transport layer protocol category and data interaction logic, and the device behavior includes device online, device data acquisition, TCP protocol and interaction behavior; according to the device behavior, based on the protocol specification, determining the first gateway configuration of the southbound gateway; wherein the first gateway configuration includes device online mode configuration, data acquisition mode configuration, TCP packet processing configuration and multi-round interaction configuration; according to the push data format of the northbound gateway, determining the second gateway configuration of the northbound gateway; according to the first gateway configuration, adjusting the format of the data to be uploaded reported by the device to the internal data format, and according to the second gateway configuration, adjusting the internal data format to the business data format; uploading the data to be uploaded in the business data format to the cloud platform. The present invention realizes the configuration of the southbound gateway through the device online mode configuration, data acquisition mode configuration, TCP packet processing configuration and multi-round interaction configuration, and realizes the configuration of the northbound gateway through the second gateway configuration, which can realize data configuration for multiple business scenarios and is conducive to improving the efficiency and accuracy of data transmission.

[0007] In addition, the data transmission method according to the above embodiment of the present invention may also have the following additional technical features: Furthermore, in the data transmission method according to the embodiment of the present invention, the device online mode configuration is determined by the following steps: If the device is of active connection type, open the IP and port corresponding to the device, and establish a connection with the device through identity authentication according to the first connection request of the device to implement the device online mode configuration; If the device is of passive connection type, configure the IP and port corresponding to the device, initiate a second connection request for the device, and determine to establish a connection with the device through the response received from the device to implement the device online mode configuration.

[0008] Furthermore, in the data transmission method of the present application, the TCP packet processing configuration is determined by the following steps: If the data to be uploaded is of a fixed length, the received data to be uploaded of the preset length will be integrated into a complete data packet to implement TCP packet processing configuration; Alternatively, if the data to be uploaded includes a preset end character, the data to be uploaded before receiving the preset end character is integrated into a complete data table to implement TCP packet processing configuration; Alternatively, if the data to be uploaded contains a preset data header, the data to be uploaded is subpacketized according to the subpacketization parameters of the preset data header and integrated into a complete data packet to implement TCP packet processing configuration; wherein the subpacketization parameters include data packet length and data type.

[0009] Furthermore, in the data transmission method of the present application, the multi-round interaction configuration is determined by the following steps: A mapping node, a parsing node, a script node, and an ACK node are set to realize multi-round interaction configuration between the device and the gateway; wherein the mapping node is used to map the key information of the key-value data to realize the mapping of the value information; the parsing node is used to extract data that meets the preset configuration rules from the data to be uploaded; the script node is used to write a script to parse the data to be uploaded; the ACK node is used to determine the confirmation parameters of the other party during the interaction process; wherein the confirmation parameters include message content and message format.

[0010] Furthermore, in the data transmission method of the present application, the second gateway configuration includes a topic format configuration, and the topic format configuration is determined by the following steps: The receiving device customizes the topic format; wherein the topic format is the format in which the uplink data is sent by the gateway to the cloud platform; According to the topic format, the identifier and position of the uplink data to be written are determined to implement the topic format configuration; the identifier is the gateway identifier, and the position is the position of the gateway identifier in the uplink data.

[0011] Furthermore, in the data transmission method of the present application, the second gateway configuration includes a payload format configuration, and the payload format configuration is determined by the following steps: A payload format customized by the receiving device; wherein the payload format is the format of uplink data sent by the gateway to the cloud platform; According to the payload format, determine the gateway ID, first position, sub-device ID, and second position of the uplink data to be written to implement the payload format configuration; the first position is the position of the gateway ID in the uplink data, and the second position is the position of the sub-device ID in the uplink data.

[0012] Furthermore, the data transmission method in the present application further includes: Adjusting the format of the instruction data issued by the cloud platform to a first data format according to the second gateway configuration, and adjusting the first data format to a second data format according to the first gateway configuration; Send the instruction data in the second data format to the device.

[0013] On the other hand, an embodiment of the present invention provides a data transmission system, including: A behavior determination module is used to determine the corresponding device behavior based on the southbound device protocol parameters of the gateway; wherein the device protocol parameters include connection behavior, data interaction behavior, transport layer protocol category and data interaction logic; the device behavior includes device online, device data acquisition, TCP protocol and interaction behavior; A first configuration module is configured to determine a first southbound gateway configuration of the gateway based on the device behavior and protocol specifications; wherein the first gateway configuration includes a device online mode configuration, a data collection mode configuration, a TCP packet processing configuration, and a multi-round interaction configuration; A second configuration module is used to determine a second gateway configuration for the northbound direction of the gateway according to a push data format of the northbound direction of the gateway; a format conversion module, configured to adjust the format of the data to be uploaded reported by the device to an internal data format according to the first gateway configuration, and to adjust the internal data format to a business data format according to the second gateway configuration; The upload module is used to upload the data to be uploaded in the business data format to the cloud platform.

[0014] In another aspect, an embodiment of the present invention provides a data transmission device, including: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor is enabled to implement the above-mentioned data transmission method.

[0015] On the other hand, an embodiment of the present invention provides a storage medium storing a program executable by a processor. When the program is executed by the processor, it is used to implement the above-mentioned data transmission method.

[0016] The method provided by the present invention includes: determining the corresponding device behavior according to the device protocol parameters of the southbound gateway; wherein the device protocol parameters include connection behavior, data interaction behavior, transport layer protocol category and data interaction logic, and the device behavior includes device online, device data acquisition, TCP protocol and interaction behavior; according to the device behavior, based on the protocol specification, determining the first gateway configuration of the southbound gateway; wherein the first gateway configuration includes device online mode configuration, data acquisition mode configuration, TCP packet processing configuration and multi-round interaction configuration; according to the push data format of the northbound gateway, determining the second gateway configuration of the northbound gateway; according to the first gateway configuration, adjusting the format of the data to be uploaded reported by the device to the internal data format, and according to the second gateway configuration, adjusting the internal data format to the business data format; uploading the data to be uploaded in the business data format to the cloud platform. The present invention realizes the configuration of the southbound gateway through the device online mode configuration, data acquisition mode configuration, TCP packet processing configuration and multi-round interaction configuration, and realizes the configuration of the northbound gateway through the second gateway configuration, which can realize data configuration of multiple business scenarios, and is conducive to improving the efficiency and accuracy of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic diagram of a flow chart of an embodiment of a data transmission method provided by the present invention; Figure 2 A schematic diagram of a flow chart of an embodiment of the device online mode configuration provided by the present invention; Figure 3A flowchart of an embodiment of the TCP packet processing configuration provided by the present invention; Figure 4 A schematic diagram of a flow chart of an embodiment of a multi-round interaction configuration provided by the present invention; Figure 5 A schematic diagram of a flow chart of an embodiment of the second gateway configuration provided by the present invention; Figure 6 A schematic diagram of a flow chart of another embodiment of the second gateway configuration provided by the present invention; Figure 7 A flowchart of an embodiment of the data sending process provided by the present invention; Figure 8 A schematic flow chart of another embodiment of the data transmission method provided by the present invention; Figure 9 A schematic structural diagram of an embodiment of a data transmission system provided by the present invention; Figure 10 A schematic structural diagram of an embodiment of the data transmission device provided by the present invention. DETAILED DESCRIPTION

[0019] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0020] First, the terms used in this application are explained: Edge side: also known as the gateway side, in the IoT architecture, is located at the edge of the network close to the device end, responsible for connecting the device to the cloud, and is responsible for data collection, processing, and forwarding.

[0021] Intelligent protocol conversion: Automatically and intelligently converts different communication protocols used by devices, such as the TCP protocol, into the MQTT protocol suitable for cloud interaction, including handling various issues during the protocol conversion process to achieve efficient and stable communication between devices and the cloud.

[0022] TCP protocol unpacking and packet assembly issues: When the TCP protocol transmits data, due to its stream-oriented nature, multiple data packets may be stuck together and need to be disassembled (unpacking) or a data packet may be segmented and transmitted (packet assembly). Improper handling can lead to data reception and parsing errors.

[0023] MQTT protocol: A lightweight message publish / subscribe transmission protocol suitable for data communication between IoT devices and the cloud in low-bandwidth, high-latency or unstable network environments.

[0024] Southbound configuration: refers to the configuration related to the interaction between the gateway and the device side, which is used to control device access to the gateway and data collection functions.

[0025] Northbound configuration: refers to the configuration related to the interaction between the gateway and the cloud, which is used to standardize the format and rules of data transmission between the gateway and the cloud.

[0026] Multi-round interaction configuration nodes: including mapping nodes, parsing nodes, script nodes, and ACK nodes. During the communication process, it supports multiple rounds of interaction between devices and gateways, and can effectively adapt to logically complex IoT protocols.

[0027] In IoT applications, migrating terminal devices to the cloud via a gateway is a common method. The vast number of existing and incremental devices varies greatly, using a wide variety of communication protocols. Most are application-layer protocols based on TCP and UDP, including common protocols, industry-standard protocols, and a large number of proprietary protocols. Protocol integration requires extensive development work. When connecting gateways to cloud platforms, the MQTT protocol, with its lightweight, low power consumption, and support for message subscription and publication models, has become a common protocol for communication between IoT devices and the cloud.

[0028] Currently, edge gateways face numerous challenges when communicating with devices and the cloud. Traditional gateways often lack flexible configuration capabilities, preventing personalized access and data collection settings based on the needs of different devices. For example, online and data collection modes often rely on a single, fixed approach, failing to meet the diverse needs of business scenarios. Existing solutions for TCP packet unpacking and assembly are either overly complex, increasing the processing burden on gateway devices, or inflexible, unable to adapt to diverse application scenarios. Furthermore, existing technologies struggle to flexibly and finely configure the topic and payload formats of uplink and downlink messages between the gateway and the cloud, compromising data transmission accuracy and efficiency. These challenges limit the overall performance and scalability of IoT systems, necessitating new technical solutions.

[0029] The present invention belongs to the field of Internet of Things and communication technology, and in particular to protocol conversion technology for edge-side devices.

[0030] First refer to Figure 1, a data transmission method is provided in an embodiment of the present invention. The data transmission method in the embodiment of the present invention can be applied to a terminal, a server, or software running in a terminal or a server. The terminal can be a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Reference Figure 1 The data transmission method in the embodiment of the present invention mainly includes the following steps: Step S100: Determine the corresponding device behavior based on the southbound device protocol parameters of the gateway; wherein the device protocol parameters include connection behavior, data interaction behavior, transport layer protocol type, and data interaction logic; and the device behavior includes device online, device data acquisition, TCP protocol, and interaction behavior. Step S200: Determine a first southbound gateway configuration of the gateway based on the device behavior and protocol specifications; wherein the first gateway configuration includes a device online mode configuration, a data collection mode configuration, a TCP packet processing configuration, and a multi-round interaction configuration; Step S300: Determine the second gateway configuration for the northbound direction of the gateway according to the push data format of the northbound direction of the gateway; Step S400: adjusting the format of the data to be uploaded reported by the device to an internal data format according to the first gateway configuration, and adjusting the internal data format to a business data format according to the second gateway configuration; Step S500: Upload the data to be uploaded in the business data format to the cloud platform.

[0031] In some possible implementations, the device online mode is determined according to the device protocol connection behavior of the southbound gateway, and the device online mode configuration of the southbound gateway is determined based on the device online mode and protocol specifications; the device data acquisition mode is determined according to the device protocol data interaction behavior of the southbound gateway, and the data acquisition mode configuration of the southbound gateway is determined based on the device data acquisition mode and protocol specifications; if the transport layer protocol category of the southbound gateway device protocol is the TCP protocol, the TCP packet processing configuration of the southbound gateway is determined based on the protocol specifications to unpack and assemble the uploaded data; according to the data interaction logic of the southbound gateway, the multi-round interaction configuration of the southbound gateway is determined based on the interaction parameters of the protocol specifications. The protocol specification can be customized by the device. The first gateway configuration can be used for the uplink upload process from the device to the cloud platform, and can also be used for the downlink distribution process from the cloud platform to the device. It should be noted that the execution subject of steps S100 to S500 in this application can be the gateway.

[0032] The present invention provides a method and system for implementing protocol adaptive conversion and cloud migration for IoT terminal devices on the edge side, solving the problems of protocol access and compatibility with cloud migration for different devices, meeting the diverse business docking scenarios of the IoT, reducing the workload of protocol customization development, and improving the access efficiency of IoT devices and the scalability of the platform. Through flexible configuration, the present invention can combine the southbound device online mode and the gateway data acquisition mode to achieve high configurability of the southbound gateway, thereby improving the access efficiency of complex device protocols; The present invention effectively handles the unpacking and assembly problems of TCP IoT device protocols through flexible configuration, ensuring the accuracy and stability of data transmission between devices and gateways. The present invention provides configurable mapping nodes, parsing nodes, script nodes, and ACK nodes to support multiple rounds of interaction, ensuring adaptation to the interaction behaviors of most complex protocols and improving the degree of protocol conversion adaptability. Through flexible configuration, the present invention achieves high configurability of the topic and payload formats of the gateway's northbound (cloud) uplink and downlink messages based on the MQTT protocol, improving the gateway's ability to adapt to different cloud platform gateway access specifications.

[0033] Further, refer to Figure 2 In the data transmission method according to the embodiment of the present invention, the device online mode configuration is determined by the following steps: Step S210: If the device is an active connection type, open the IP and port corresponding to the device, and establish a connection with the device through identity authentication according to the first connection request of the device to implement the device online mode configuration; Step S220: If the device is a passive connection type, configure the IP and port corresponding to the device, initiate a second connection request for the device, and determine to establish a connection with the device through the response received from the device to implement the device online mode configuration.

[0034] For actively connected devices, the gateway can also perform legitimacy verification on data requests.

[0035] Further, refer to Figure 3 In the data transmission method of this application, the TCP packet processing configuration is determined by the following steps: Step S230: If the data to be uploaded is of a fixed length, the received data to be uploaded of the preset length is integrated into a complete data packet to implement TCP packet processing configuration; Alternatively, in step S240, if the data to be uploaded includes a preset end character, the data to be uploaded before receiving the preset end character is integrated into a complete data table to implement TCP packet processing configuration; Alternatively, in step S250, if the data to be uploaded contains a preset data header, the data to be uploaded is sub-packetized according to the sub-packetization parameters of the preset data header and integrated into a complete data packet to implement TCP packet processing configuration; wherein the sub-packetization parameters include data packet length and data type.

[0036] The fixed length can be the length of the data to be uploaded, and the specific values ​​of the fixed length and the preset length can be adjusted; the preset length is the data length of each complete packet after the TCP protocol is unpacked. The method provided in the present application also includes waiting for subsequent data to be uploaded if the length of the currently received data to be uploaded is less than the preset length. The preset end character can be the last character of each complete data packet of the uploaded data to be unpacked. The preset data header can be the first character of each complete data packet of the uploaded data to be unpacked.

[0037] Further, refer to Figure 4 In the data transmission method of this application, the multi-round interaction configuration is determined by the following steps: Step S260, setting a mapping node, a parsing node, a script node, and an ACK node to implement multi-round interaction configuration between the device and the gateway; wherein the mapping node is used to map the key information of the key-value data to implement the mapping of the value information; the parsing node is used to extract data that meets the preset configuration rules from the data to be uploaded; the script node is used to write a script to parse the data to be uploaded; the ACK node is used to determine the confirmation parameters of the other party during the interaction process; wherein the confirmation parameters include the message content and the message format.

[0038] The value information in this application can be a device identifier. The preset configuration rules determine the behavior and the data parameters corresponding to the behavior. The configuration of the ACK node can confirm the successful data exchange by receiving the other party's confirmation parameters.

[0039] Further, refer to Figure 5 In the data transmission method of this application, the second gateway configuration includes a topic format configuration, and the topic format configuration is determined by the following steps: Step S270: receiving a device-defined topic format; wherein the topic format is the format for sending uplink data from the gateway to the cloud platform; Step S280 , determining the identifier and location of the uplink data to be written according to the topic format to implement the topic format configuration; the identifier is the gateway identifier, and the location is the location of the gateway identifier in the uplink data.

[0040] According to the topic format defined by the device, the gateway identifier of the current gateway is written into the uplink data sent by the gateway to the cloud platform.

[0041] Further, refer to Figure 6 In the data transmission method of this application, the second gateway configuration includes a payload format configuration, and the payload format configuration is determined by the following steps: Step S291: receiving a device-defined payload format; wherein the payload format is the format of uplink data sent by the gateway to the cloud platform; Step S292, according to the payload format, determine the gateway ID, first position and sub-device ID, second position of the uplink data to be written to realize the payload format configuration; the first position is the position of the gateway ID in the uplink data, and the second position is the position of the sub-device ID in the uplink data.

[0042] Further, refer to Figure 7 , the data transmission method in this application further includes: Step S600: adjusting the format of the instruction data issued by the cloud platform to the first data format according to the second gateway configuration, and adjusting the first data format to the second data format according to the first gateway configuration; Step S700: Send the instruction data in the second data format to the device.

[0043] The second data format is a data format that can be recognized by the device.

[0044] Refer to the following Figure 8 The data transmission method provided by this application is described in detail with specific embodiments: The invention is applied on the edge side, that is, the gateway device (here refers to the gateway in a broad sense, including but not limited to the gateway box in a narrow sense, the edge all-in-one machine, the computer and the edge side device that can run the operating system). It comprehensively considers the docking requirements of different types of devices and platforms. By providing highly flexible configuration capabilities and protocol adaptation conversion engines, it quickly builds corresponding protocol configurations for protocols with different business logics, and flexibly and efficiently realizes the access of devices and the docking of platforms. The architecture diagram is shown in the attached figure. Figure 8 As shown, it mainly includes gateway southbound adaptive configuration and gateway northbound adaptive configuration.

[0045] 1. Gateway southbound adaptive configuration.

[0046] Supports southbound gateway, that is, adaptive configuration on the device side, which mainly includes four aspects: one is the device online mode configuration, two is the data collection mode configuration, three is the TCP packet processing configuration, and four is the multi-round interaction configuration.

[0047] 1) Support device online mode configuration.

[0048] Device online modes are generally divided into two categories: active connection and passive connection, that is, the device actively connects to the gateway and the gateway actively connects to the device.

[0049] When a device actively connects, no gateway configuration is required. The gateway simply opens the corresponding IP address and port, waits for the device's connection request, and performs device verification. The device initiates a connection request, sending a connection request message to the gateway according to pre-set rules. Upon receiving the request, the gateway performs a validation check and identity authentication. For example, upon startup, the device sends a connection request packet containing information such as the device ID and device type to the gateway's specific IP address and port based on its stored configuration information. Upon receiving the request packet, the gateway verifies the device's legitimacy by querying its local device information database. If the device is legitimate, it responds with a confirmation message; otherwise, it denies the connection and sends an error message.

[0050] When the device is passively connected, the device's IP and port are configured on the gateway side, and the gateway initiates the connection request. The gateway can also be set to periodically scan the device list. For devices that are not online, a connection request message is sent to them. After receiving the request, the device responds and completes the online process.

[0051] 2) Support data collection mode configuration.

[0052] Data collection modes are generally divided into two categories: active reporting and passive collection, that is, devices actively report data and gateways actively collect data.

[0053] In device-active reporting mode, no configuration is required on the gateway side. Instead, the device proactively sends collected data to the gateway at set intervals or event triggers. For example, an environmental monitoring device collects temperature, humidity, and other data at regular intervals, encapsulates this data into a specific format, and sends it to the gateway. Upon receiving the data packets, the gateway parses and processes them.

[0054] In the gateway's active data collection mode, the gateway supports configuring data collection tasks, including but not limited to point information, data type, data collection instructions, and data collection length, and periodically sends data collection requests to the device. After receiving the instructions, the device returns the collected data to the gateway.

[0055] 3) Support TCP protocol packet processing configuration.

[0056] If the device uses an application layer protocol based on the TCP protocol to report data, there are problems with the processing of TCP packet unpacking and assembly. The present invention supports the following four configurations of TCP packet processing methods.

[0057] Fixed-length packet processing: When receiving fixed-length datagrams, the gateway divides them into packets according to the pre-configured length. For example, if each packet is fixed-length 1024 bytes, the gateway reads 1024 bytes of data from the receive buffer each time it receives data and processes it as a complete packet. If the buffer contains less than 1024 bytes, the gateway waits for more data to be added until the full 1024 bytes are available before processing the packet.

[0058] Special terminator processing: Each TCP packet carries a special terminator, such as "\r\n" or another custom end identifier. Based on the configuration information, the gateway identifies packet boundaries by searching for the special terminator. When the gateway receives data, it searches for the special terminator from the beginning of the receive buffer. Once it finds the special terminator, it processes the data before the terminator as a complete packet and then continues searching for the next packet from the position after the terminator.

[0059] Header-based processing: A header containing information such as the packet length and data type is added to the packet. Upon receiving the data, the gateway first parses the header according to the configuration information to obtain the packet length. It then reads the corresponding length of data from the receive buffer and processes it as a complete packet. For example, if the header specifies a packet length of 4 bytes, the gateway will first read the first 4 bytes to determine the packet length, then read the subsequent data based on that length for processing.

[0060] Customized (dynamic scripting supported) processing: Users can define packet processing rules by writing dynamic scripts. For example, users can write scripts to parse complex packet structures based on specific business needs. Scripts can flexibly judge and process packet content and data characteristics, providing personalized unpacking and assembly solutions. After receiving data, the gateway invokes the corresponding script to process the data and parse it into complete packets.

[0061] 4) Support multi-round interaction configuration.

[0062] By setting up mapping nodes, parsing nodes, script nodes, and ACK nodes, multiple rounds of interaction between the device and the gateway are achieved.

[0063] Mapping node: mainly for data in key:value format, mapping key1 in the data to key2. For example, the internal protocol conversion specification of the gateway stipulates that the device unique identifier is deviceId, while the device-reported data uses deviceCode to identify the device unique identifier. In this case, the mapping node can be used to map deviceCode to deviceId; or the value corresponding to a key can be stored in a global variable for use in the protocol conversion process.

[0064] Parsing node: Extracts data that meets certain configuration rules from message data, such as intercepting strings with indexes 3-9 or extracting the content of the type node in JSON. Configuration rules can be further expanded as needed.

[0065] Script node: supports writing scripts (such as Java, JS, Python, etc.) in the node to parse message data that cannot be parsed by existing node types. The input parameters and output values ​​of the script are fixed types.

[0066] ACK node: Some protocols with complex interaction logic often require confirmation. During communication between two parties, after one party sends a data packet, the data exchange is considered successful only after receiving an ACK confirmation message from the other party. For example, in the device online process, after the device sends an online request packet, the gateway receives and verifies the request packet. Then, according to the pre-configured ACK format in accordance with the protocol specification, it sends an ACK packet containing online information to the device. After receiving the ACK packet, the device confirms the online success. The ACK format is user-definable, allowing users to configure the ACK packet content and format according to business needs, such as adding additional status information and timestamps.

[0067] 2. Gateway northbound adaptive configuration.

[0068] Supports northbound gateways, or adaptive configuration on the cloud platform side. Since gateways and cloud platforms typically interact via the MQTT protocol, this paper, based on the MQTT protocol specification, illustrates the northbound gateway adaptive configuration capabilities. While the communication protocol is established, different cloud platforms have different format specifications for service data. Consequently, gateway data can only be reported transparently without code changes on either side. Otherwise, one side must adapt to the other.

[0069] The present invention supports the standard configuration of business data in Json format based on the MQTT protocol to adapt the gateway's upstream and downstream data messages to different cloud platforms. Its configuration content mainly includes topic configuration and payload configuration.

[0070] Support for uplink message topic format configuration: The gateway supports user-defined topic formats for uplink messages, including the placement of the gateway ID within the topic. For example, based on the cloud platform's topic format requirements, users can configure the topic format to "iot / gw / device / {gateway_id} / data / upload," where "gateway_id" represents the gateway's unique identifier and "{gateway_id}" is a placeholder for the gateway ID. When the gateway sends uplink messages to the cloud, it fills the placeholder with its own gateway ID to complete the topic. This allows the cloud to verify message compliance based on the topic and accurately distinguish uplink data from different gateways.

[0071] Support for uplink message payload format configuration: Users can configure the uplink message payload format, focusing on the placement of the device ID (gateway or sub-device) in the payload. For example, according to the cloud platform's gateway protocol specification, when configuring a JSON payload, the gateway ID is placed in the "gateway_id" field, and the sub-device ID is placed in the "device_list[0].device_id" field (assuming the first device in the device list). When assembling the uplink message payload, the gateway fills the relevant ID information into the corresponding position according to the configured format, ensuring that the cloud can accurately obtain the sub-device and gateway information, thereby accurately placing the business data under the corresponding device.

[0072] Support for configuring the topic format for downlink messages: Similar to uplink messages, the gateway supports configuring the topic format for downlink messages and the location of the gateway ID in the topic. For example, the topic format for downlink messages is configured as "iot / gw / device / {gateway_id} / command / down", where "gateway_id" represents the unique identifier of the gateway and "{gateway_id}" is a placeholder for the gateway ID. When the cloud platform publishes a downlink command, it will carry the corresponding topic. Different gateway devices will subscribe to their own topics to ensure that they receive their own messages.

[0073] Downlink message payload format configuration: Users can configure the payload format of the downlink message, as well as the placement of the device ID (gateway or sub-device) and command ID in the payload. For example, configure the downlink message payload format to JSON format, place the gateway ID in the "gateway_id" field, place the sub-device ID in the "device_list[0].device_id" field (assuming the first device in the device list), and place the command ID in "device_list[0].cmd_id." After receiving the downlink message, the gateway parses the payload according to the configured format, obtains the command and the ID information of the relevant device, and then determines whether the command content belongs to itself, and chooses to localize the command or forward it to the corresponding sub-device.

[0074] In summary, the overall process of migrating edge devices to the cloud using the method and system described in this application is as follows: Step S81: Analyze the device protocol connection behavior, determine the device online mode, and configure the corresponding parameters according to the method described above; Step S82: Analyze the device protocol data interaction behavior, determine the device data acquisition mode, and configure the corresponding parameters according to the method described above; Step S83: Analyze the transport layer protocol type of the device protocol. If it is TCP protocol, perform corresponding unpacking and assembly configuration according to the protocol specification. Step S84: Analyze the device protocol data interaction logic, and perform protocol Payload layer analysis and response based on specific protocol specifications such as whether there is interaction, message type, field type definition, etc., in combination with mapping nodes, parsing nodes, script nodes, and ACK nodes.

[0075] Step S85: The protocol adaptive conversion cloud engine described in this application executes the protocol conversion process configured in steps S81 to S84, converts the device reported data into an internal data format (usually JSON format), and saves it to the local database for persistence; Step S86: Northbound data conversion is configured according to the cloud platform gateway access specification. The protocol adaptive conversion engine described in this application converts the internal data format into a business data format recognizable by the cloud platform according to this configuration, and finally completes the conversion and cloud migration of device data. In step S87, the overall configuration process of instruction downlink and data reporting is similar, but the data flow is opposite. That is, under the drive of the protocol adaptive conversion cloud engine described in this application, the cloud platform instruction is first converted into an internal data format and encoded into a device-recognizable format according to the device protocol specification and sent to the device.

[0076] The present invention provides a method and system for adaptive protocol conversion and cloud migration of IoT terminal devices. By providing flexible configuration capabilities and a protocol adaptation conversion engine, it solves the problem of protocol access for different devices and compatibility with cloud migration. The present invention provides a highly configurable device adaptive access engine that adapts to the online mode and data acquisition mode of IoT terminal devices through flexible configuration and realizes automatic conversion of device protocols, thereby improving the access efficiency of complex device protocols. The present invention provides a device adaptive access engine that effectively handles the unpacking and assembly problems of TCP-based IoT device protocols through flexible configuration, ensuring the accuracy and stability of data transmission between devices and gateways; The present invention provides highly configurable mapping nodes, parsing nodes, script nodes, and ACK nodes, enabling multi-round interaction between devices, gateways, and platforms. It supports the adaptation of interactive behaviors of most complex protocols and improves the degree of protocol conversion adaptability. The present invention provides a method for flexibly configuring the topic and payload formats in MQTT messages, thereby achieving adaptive adaptation of the uplink and downlink specifications of the gateway and northbound (cloud platform) business data, and improving the gateway's ability to adapt to different cloud platforms.

[0077] This invention improves compatibility and flexibility: Through multiple southbound configuration options, the gateway can adapt to the access needs of different device types. Whether the onboarding mode is device-initiated or gateway-initiated, or the data collection mode is device-initiated or gateway-initiated, flexible selection can be made based on device characteristics and business requirements, significantly improving the gateway's compatibility with various devices. Furthermore, the flexible northbound configuration enables communication between the gateway and the cloud to meet diverse business scenarios. Different topic and payload format configurations can adapt to different cloud platforms and application requirements, enhancing system scalability.

[0078] This invention ensures data transmission accuracy: Addressing the TCP protocol's packet unpacking and assembly issues, it provides a variety of flexible configuration options, including fixed-length, special terminator, header-based, and custom (with support for dynamic scripting) processing. These methods allow for optimal processing based on the data characteristics of different devices, effectively preventing errors and loss during data transmission and ensuring accurate and stable data transmission between devices and gateways.

[0079] This invention enhances communication reliability: it supports multiple rounds of interaction and ensures flexibility and reliability of communication between devices and gateways by configuring mapping nodes, parsing nodes, script nodes, and ACK nodes. This mechanism adapts to the interactions of most complex protocols while avoiding data loss and transmission errors, improving protocol conversion adaptability.

[0080] The present invention improves the overall performance of the system: The protocol intelligent conversion method and system of the present invention optimize the communication process between edge-side gateway devices and devices and the cloud, reduce data processing and transmission errors, and improve data transmission efficiency, thereby improving the overall performance of the Internet of Things system and providing more reliable technical support for various Internet of Things application scenarios such as smart homes and industrial Internet of Things.

[0081] Existing technical solutions: Using a customized development method, customized development is carried out for each protocol, which has the disadvantages of large workload and long cycle. Some protocol conversion configuration solutions, on the one hand, do not consider the device's online and offline mode and data acquisition mode factors during southbound configuration, and also lack the processing of TCP-type protocol unpacking and assembly issues. On the other hand, they do not consider the adaptation of the northbound platform.

[0082] This invention comprehensively and in detail considers the adaptability of north-south services on the edge side and provides a complete process method for adaptively converting device protocols to the cloud. It can not only meet the docking requirements of different types of southbound device protocols, but also adapt to the access requirements of northbound platforms, effectively reducing the workload of customized device protocol development. When the southbound device protocol is configured for access, the present invention focuses on solving the differentiated configuration problems brought about by the southbound device online and data acquisition mode, as well as the unpacking and assembly problems faced when configuring TCP-type protocols. At the same time, by adding multiple protocol conversion configuration nodes, the ability to adapt to complex protocols is improved; on the other hand, through the configuration method, the adaptive adaptation problem of the northbound platform is solved, and a fast and flexible communication bridge is built for the device and platform.

[0083] The present invention is particularly suitable for business scenarios in the field of the Internet of Things where the edge side accesses devices and goes to the cloud through a gateway.

[0084] Secondly, refer to the attached Figure 9 A data transmission system proposed in an embodiment of the present invention is described, and the system specifically includes: Behavior determination module 710, configured to determine corresponding device behaviors based on southbound device protocol parameters of the gateway; wherein the device protocol parameters include connection behavior, data interaction behavior, transport layer protocol type, and data interaction logic; and the device behaviors include device online, device data acquisition, TCP protocol, and interaction behavior; A first configuration module 720 is configured to determine a first southbound gateway configuration of the gateway based on the device behavior and protocol specifications; wherein the first gateway configuration includes a device online mode configuration, a data collection mode configuration, a TCP packet processing configuration, and a multi-round interaction configuration; The second configuration module 730 is used to determine the second gateway configuration for the northbound direction of the gateway according to the push data format of the northbound direction of the gateway; a format conversion module 740 configured to adjust the format of the data to be uploaded reported by the device to an internal data format according to the first gateway configuration, and to adjust the internal data format to a service data format according to the second gateway configuration; The uploading module 750 is used to upload the data to be uploaded in the business data format to the cloud platform.

[0085] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0086] Reference Figure 10 , an embodiment of the present invention provides a data transmission device, including: at least one processor 810; at least one memory 820, configured to store at least one program; When the at least one program is executed by the at least one processor 810, the at least one processor 810 implements the data transmission method.

[0087] Similarly, the contents of the above method embodiments are applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0088] An embodiment of the present invention further provides a computer-readable storage medium storing a program executable by a processor. The program executable by the processor is used to execute the above-mentioned data transmission method when executed by the processor.

[0089] Similarly, the contents of the above method embodiments are applicable to the present storage medium embodiment. The functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0090] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0091] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art will be able to implement the present invention set forth in the claims using ordinary skill without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0092] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several programs that enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0093] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable programs for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, a program execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can retrieve and execute a program from a program execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, a program execution system, apparatus, or device.

[0094] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0095] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable program execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0096] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0097] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0098] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A data transmission method, characterized in that: The following steps are involved: Determine the corresponding device behavior based on the southbound device protocol parameters of the gateway; wherein the device protocol parameters include connection behavior, data interaction behavior, transport layer protocol category and data interaction logic; the device behavior includes device online, device data acquisition, TCP protocol and interaction behavior; Determine, based on the device behavior and protocol specifications, a first southbound gateway configuration for the gateway; wherein the first gateway configuration includes a device online mode configuration, a data collection mode configuration, a TCP packet processing configuration, and a multi-round interaction configuration; Determine the second gateway configuration for the northbound direction of the gateway based on the data format pushed by the northbound direction of the gateway; Adjusting the format of the data to be uploaded reported by the device to an internal data format according to the first gateway configuration, and adjusting the internal data format to a service data format according to the second gateway configuration; Upload the data to be uploaded in the business data format to the cloud platform.

2. The data transmission method according to claim 1, wherein: The device online mode configuration is determined by the following steps: If the device is of active connection type, open the IP and port corresponding to the device, and establish a connection with the device through identity authentication according to the first connection request of the device to implement the device online mode configuration; If the device is of passive connection type, configure the IP and port corresponding to the device, initiate a second connection request for the device, and determine to establish a connection with the device through the response received from the device to implement the device online mode configuration.

3. The data transmission method according to claim 1, wherein: The TCP packet processing configuration is determined by the following steps: If the data to be uploaded is of a fixed length, the received data to be uploaded of the preset length will be integrated into a complete data packet to implement TCP packet processing configuration; Alternatively, if the data to be uploaded includes a preset end character, the data to be uploaded before receiving the preset end character is integrated into a complete data table to implement TCP packet processing configuration; Alternatively, if the data to be uploaded contains a preset data header, the data to be uploaded is subpacketized according to the subpacketization parameters of the preset data header and integrated into a complete data packet to implement TCP packet processing configuration; wherein the subpacketization parameters include data packet length and data type.

4. The data transmission method according to claim 1, wherein: The multi-round interaction configuration is determined by the following steps: A mapping node, a parsing node, a script node, and an ACK node are set to realize multi-round interaction configuration between the device and the gateway; wherein the mapping node is used to map the key information of the key-value data to realize the mapping of the value information; the parsing node is used to extract data that meets the preset configuration rules from the data to be uploaded; the script node is used to write a script to parse the data to be uploaded; the ACK node is used to determine the confirmation parameters of the other party during the interaction process; wherein the confirmation parameters include message content and message format.

5. The data transmission method according to claim 1, wherein: The second gateway configuration includes a topic format configuration, and the topic format configuration is determined by the following steps: The receiving device customizes the topic format; wherein the topic format is the format in which the uplink data is sent by the gateway to the cloud platform; According to the topic format, the identifier and position of the uplink data to be written are determined to implement the topic format configuration; the identifier is the gateway identifier, and the position is the position of the gateway identifier in the uplink data.

6. The data transmission method according to claim 1, wherein: The second gateway configuration includes a payload format configuration, and the payload format configuration is determined by the following steps: A payload format customized by the receiving device; wherein the payload format is the format of uplink data sent by the gateway to the cloud platform; According to the payload format, determine the gateway ID, first position, sub-device ID, and second position of the uplink data to be written to implement the payload format configuration; the first position is the position of the gateway ID in the uplink data, and the second position is the position of the sub-device ID in the uplink data.

7. The data transmission method according to claim 1, wherein: The method further comprises: Adjusting the format of the instruction data issued by the cloud platform to a first data format according to the second gateway configuration, and adjusting the first data format to a second data format according to the first gateway configuration; Send the instruction data in the second data format to the device.

8. A data transmission system, characterized in that: include: A behavior determination module is used to determine the corresponding device behavior based on the southbound device protocol parameters of the gateway; wherein the device protocol parameters include connection behavior, data interaction behavior, transport layer protocol category and data interaction logic; the device behavior includes device online, device data acquisition, TCP protocol and interaction behavior; A first configuration module is configured to determine a first southbound gateway configuration of the gateway based on the device behavior and protocol specifications; wherein the first gateway configuration includes a device online mode configuration, a data collection mode configuration, a TCP packet processing configuration, and a multi-round interaction configuration; A second configuration module is used to determine a second gateway configuration for the northbound direction of the gateway according to a push data format of the northbound direction of the gateway; a format conversion module, configured to adjust the format of the data to be uploaded reported by the device to an internal data format according to the first gateway configuration, and to adjust the internal data format to a business data format according to the second gateway configuration; The upload module is used to upload the data to be uploaded in the business data format to the cloud platform.

9. A data transmission device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the data transmission method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to implement the data transmission method according to any one of claims 1 to 7 when executed by the processor.

Images