Data transmission method and device, electronic equipment and readable storage medium
By introducing a broker into the MQTT protocol, calculating a comprehensive score based on network indicator data, and dynamically adjusting the transmission strategy, the problem of unstable data transmission in the MQTT protocol under harsh network environments is solved, and efficient data transmission under harsh network environments is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-22
- Publication Date
- 2026-03-24
AI Technical Summary
The existing MQTT protocol cannot adapt to dynamic network changes in harsh network environments, resulting in unstable data transmission and failing to meet the needs of high-security scenarios such as smart cities and industrial monitoring.
An agent is introduced between the MQTT client and server. It calculates a comprehensive network score by acquiring network indicator data and determines the target transmission strategy from a preset dynamic strategy matrix. The transmission strategy is dynamically adjusted for data transmission, including weighted summation of network indicator data, dynamic strategy matrix matching, and enhanced transmission process.
It improves the stability and efficiency of data transmission in harsh network environments, significantly enhances data transmission performance in scenarios with high latency, packet loss, and intermittent connections, reduces protocol latency and retransmission frequency, and meets the needs of high-security scenarios.
Smart Images
Figure CN121728037A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of data processing technology, specifically relating to a data transmission method, a data transmission device, an electronic device, and a readable storage medium. Background Technology Currently, the data security of IoT terminal devices mainly relies on data verification and storage by centralized servers, which has problems such as high risk of single point of failure, easy data tampering, and difficulty in auditing and tracing, making it difficult to meet the needs of high-security scenarios such as smart cities and industrial monitoring.
[0002] As a mainstream lightweight communication protocol in the Internet of Things (IoT) field, MQTT performs excellently in ideal network environments. However, the standard MQTT protocol faces serious challenges in the following harsh network scenarios: ① High-latency networks (satellite communication, intercontinental links, RTT > 500ms) ② High packet loss environments (wireless mesh networks, mobile scenarios, packet loss rate > 10%) ③ Intermittent connections (outdoor devices, mobile terminals, disconnection frequency > 5 times / minute) ④ Bandwidth-constrained networks (narrowband IoT NB-IoT, LoRaWAN) The main shortcomings of the standard MQTT protocol in these environments include: (1) Fixed retransmission mechanisms (usually using static intervals such as 1 second) cannot adapt to dynamic network conditions.
[0003] (2) QoS switching requires manual configuration and cannot respond to network changes in real time. Summary of the Invention
[0004] The purpose of this application is to provide a data transmission method, apparatus, electronic device, and readable storage medium that can solve the problem that the current standard MQTT protocol cannot adapt to dynamic network changes.
[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a data transmission method applied to a proxy end, wherein a proxy end for network optimization is deployed between an MQTT client and the MQTT server, the method comprising: When the MQTT client, the MQTT server, and the broker are detected to be connected, multiple network metrics data for evaluating the network are obtained in the current network environment. A comprehensive network score for the current network environment is determined based on the aforementioned multiple network indicator data. The target transmission strategy corresponding to the comprehensive network score is determined from the preset dynamic strategy matrix; Data is transmitted between the MQTT client and the MQTT server according to the target transmission strategy.
[0006] Optionally, determining the comprehensive network score of the current network environment based on the multiple network indicator data includes: Determine the weight data corresponding to each network indicator data, and the sum of the weight data of the multiple network indicator data is 1; The network index data is weighted and summed according to the weight data to obtain the comprehensive network score of the current network environment.
[0007] Optionally, the comprehensive network score includes a first network score at a first time point and a second network score at a second time point, and further includes: Determine the score difference between the first network score and the second network score; The sampling frequency of the network indicator data is adjusted based on the score difference.
[0008] Optionally, the network indicator data includes first indicator data at a first time point and second indicator data at a second time point, wherein the first indicator data and the second indicator data are indicator data of the same type, and further includes: Determine the difference between the first indicator data and the second indicator data; The sampling frequency of the network indicator data is adjusted based on the difference in the indicator data.
[0009] Optionally, determining the target transmission strategy corresponding to the comprehensive network score from a preset dynamic strategy matrix includes: The target score range for the comprehensive network score matching is determined in the preset dynamic strategy matrix; Determine the target transmission strategy corresponding to the target score range.
[0010] Optionally, it also includes: In response to receiving MQTT message data sent by the MQTT client at QoS 1 level, a preset enhanced transmission process is initiated.
[0011] Optionally, after initiating the preset enhanced transmission process, it also includes: The target message data is encoded according to a preset format to obtain encoded data; The encoded data is then segmented into data fragments to obtain fragmented data; The fragmented data is sent to a proxy instance of the server using the QUIC protocol, so that the proxy instance generates MQTT message data based on the fragmented data and sends the MQTT message data to the MQTT server at the QoS 1 level.
[0012] Optionally, it also includes: When the connection between the proxy layer and the MQTT client and / or the MQTT server is interrupted, an optimized connection is rebuilt with the MQTT client and / or the MQTT server.
[0013] Optionally, it also includes: When the broker layer reconnects with the MQTT client and / or the MQTT server, it obtains the status of key messages during the connection disconnection period; The status of the key messages is synchronized to the agent.
[0014] Optionally, it also includes: Obtain high-priority messages from the target connected device from the MQTT server; the target connected device is a device with intermittent connections. In response to the connection success command from the target connected device, the high-priority message is pushed to the target connected device.
[0015] Secondly, embodiments of this application provide a data transmission apparatus applied to a proxy terminal, which is deployed between an MQTT client and the MQTT server for network optimization. The apparatus includes: The network metric data acquisition module is used to acquire multiple network metric data for evaluating the network in the current network environment when the MQTT client, the MQTT server, and the agent are detected to be connected. The comprehensive network score determination module is used to determine the comprehensive network score of the current network environment based on the multiple network indicator data; The transmission strategy determination module is used to determine the target transmission strategy corresponding to the comprehensive network score from a preset dynamic strategy matrix; The data transmission module is used to transmit data between the MQTT client and the MQTT server according to the target transmission strategy.
[0016] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.
[0017] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.
[0018] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.
[0019] In this embodiment, when the MQTT client, MQTT server, and broker are detected to be connected, multiple network indicator data for evaluating the network in the current network environment can be acquired. Then, a comprehensive network score for the current network environment is determined based on the multiple network indicator data. A target transmission strategy corresponding to the comprehensive network score is determined from a preset dynamic strategy matrix. Data transmission is performed between the MQTT client and the MQTT server according to the target transmission strategy. Through this embodiment, real-time monitoring of network status can be achieved, and a dynamic transmission strategy can be determined based on the comprehensive network score of the real-time network environment to implement data transmission. Attached Figure Description
[0020] Figure 1 This is a schematic diagram of the structure of a data transmission method according to an embodiment of this application; Figure 2a This is a schematic diagram of another data transmission method in an embodiment of this application; Figure 2b This is a flowchart of the enhanced MQTT transport layer in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of a data transmission device according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0022] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0023] Currently, the data security of IoT terminal devices mainly relies on data verification and storage by centralized servers, which has problems such as high risk of single point of failure, easy data tampering, and difficulty in auditing and tracing, making it difficult to meet the needs of high-security scenarios such as smart cities and industrial monitoring.
[0024] As a mainstream lightweight communication protocol in the Internet of Things (IoT) field, MQTT performs excellently in ideal network environments. However, the standard MQTT protocol faces serious challenges in the following harsh network scenarios: ① High-latency networks (satellite communication, intercontinental links, RTT>500ms) ② High packet loss environments (wireless mesh networks, mobile scenarios, packet loss rate>10%) ③ Intermittent connections (outdoor devices, mobile terminals, disconnection frequency>5 times / minute) ④ Bandwidth-constrained networks (narrowband IoT NB-IoT, LoRaWAN).
[0025] In practical applications, a method and system for resuming interrupted file transfers in a distributed system based on MQTT communication are disclosed. The method includes: storing the file identifier and file path of the file to be sent in Redis; querying the file path from the distributed shared memory Redis using the file identifier, and collecting the file content of the file to be sent from the distributed file system; calculating the number of packets of the file to be sent, and sending the file to be sent from the MQTT server to a preset terminal device; identifying the sent data, and using the sent data to transfer the unsent data in the file to be sent to the preset terminal device to complete the first interrupted file transfer in the distributed file system; storing the target file corresponding to the file download request in the distributed file system, and completing the second interrupted file transfer in the distributed file system after transferring the target file from the distributed file system to the business system. This technical solution can realize interrupted file transfers in a distributed environment.
[0026] Currently, the main shortcomings of the standard MQTT protocol in these environments include: (1) Fixed retransmission mechanisms (usually using static intervals such as 1 second) cannot adapt to dynamic network conditions.
[0027] (2) QoS switching requires manual configuration and cannot respond to network changes in real time.
[0028] In this embodiment, when the MQTT client, MQTT server, and broker are detected to be connected, multiple network indicator data for evaluating the network in the current network environment can be acquired. Then, a comprehensive network score for the current network environment is determined based on the multiple network indicator data. A target transmission strategy corresponding to the comprehensive network score is determined from a preset dynamic strategy matrix. Data transmission is performed between the MQTT client and the MQTT server according to the target transmission strategy. By calculating the comprehensive network score using the collected network indicator data, the network is evaluated, and a dynamic transmission strategy can be executed based on different comprehensive network scores.
[0029] The data transmission method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.
[0030] Reference Figure 1 This diagram illustrates a data transmission method according to an embodiment of this application. Applied to a proxy end, it deploys a proxy end for network optimization between the MQTT client and the MQTT server, and specifically includes the following steps: Step S101: When the MQTT client, the MQTT server, and the broker are detected to be connected, obtain multiple network metric data for evaluating the network in the current network environment; In this embodiment, the MQTT client and MQTT server transmit data based on the MQTT protocol. However, when the network environment is poor, the MQTT protocol will have many defects in the transmission process. In order to solve the defects of the MQTT protocol in the face of poor network environment, this embodiment introduces a transparent poor network optimization proxy layer without modifying the existing MQTT client and server, so as to realize intelligent optimization of the MQTT transmission process.
[0031] On the broker side, both the client and server appear as standard MQTT peers. The broker can intercept all passing MQTT traffic and perform real-time analysis, transformation, and optimization through its internal engine, thereby achieving adaptability to harsh network environments.
[0032] The agent can be materialized as a physical device (such as an industrial gateway), software middleware (such as a Docker container), or cloud service (such as an edge computing node). The agent mainly consists of three core engines: a network state awareness engine, a policy execution engine, and an enhanced MQTT transport layer.
[0033] Among them, the network status awareness engine can be used to sense and monitor the network status in real time, the policy execution engine can be used to execute different policies under different network conditions, and the enhanced MQTT transport layer can be used for data transmission processing under adverse network conditions.
[0034] When an MQTT client initiates a connection to an MQTT server, the client is actually connecting to a local optimized broker. The broker then establishes a connection with the remote MQTT server, posing as the client. This connection between the MQTT server, MQTT client, and broker facilitates subsequent data transmission optimization based on the broker.
[0035] After the connection is established, the network status awareness engine in the agent starts to obtain network indicator data, which may include, but is not limited to, any one or more of the following: round-trip time (RTT), basic packet loss rate, available bandwidth, dynamic RRT, packet loss rate, available bandwidth, and disconnection frequency per unit time.
[0036] Step S102: Determine the comprehensive network score of the current network environment based on the multiple network indicator data; After obtaining network indicator data, multiple network indicator data can be combined to determine the comprehensive network score in the current network environment. This comprehensive network score is used to assess the severity of network degradation. The higher the score, the higher the severity of network degradation, and the lower the score, the lower the severity of network degradation.
[0037] In one embodiment of this application, when an MQTT client initiates a connection, it actually connects to a local optimized broker. The broker then establishes a connection with the remote MQTT server as the client. At the initial stage of connection establishment, the network status awareness engine immediately starts, measuring initial network metrics, including round-trip time (RTT), basic packet loss rate, and available bandwidth, by sending probe packets, and calculating an initial network severity score. The awareness engine continuously monitors the network status, including dynamic RRT, packet loss rate, available bandwidth, and disconnection frequency per unit time. Severity score calculation: The engine calculates a comprehensive severity score based on real-time monitoring data. The comprehensive severity score can change with real-time network environment changes.
[0038] In one embodiment of this application, determining the comprehensive network score of the current network environment based on the plurality of network indicator data includes: determining the weight data corresponding to each network indicator data, wherein the sum of the weight data of the plurality of network indicator data is 1; and performing a weighted summation on each network indicator data according to the weight data to obtain the comprehensive network score of the current network environment.
[0039] The overall network score is calculated as follows: Weight data 1 * Network indicator data 1 + Weight data 2 * Network indicator data 2 + ... + Weight data N * Network indicator data N.
[0040] For example, the score = 0.4 * min(RTT / 2000ms, 1.0) + 0.3 * min(packet loss rate / 30%, 1.0) + 0.2 * min(disconnection frequency / 10 times, 1.0) + 0.1 * min(10kbps / available bandwidth, 1.0). The score result is normalized to represent the degree of network degradation.
[0041] Step S103: Determine the target transmission strategy corresponding to the comprehensive network score from the preset dynamic strategy matrix; A dynamic policy matrix can be pre-configured on the proxy side, containing the correspondence between network scores and specific implementation policies. Therefore, after determining the overall network score, the target transmission policy corresponding to that score can be found based on the dynamic policy matrix. The proxy's policy execution engine can then select the optimal transmission policy from the dynamic policy matrix based on the overall network score.
[0042] Table 1 shows a transmission strategy setting table in an embodiment of this application.
[0043]
[0044] The following are examples of data transfers performed based on Table 1: Assuming the initial network severity score is 25 (mainly contributed by high latency, but packet loss and disconnection metrics are good), the policy execution engine matches the 25 score with the dynamic policy matrix, falling into the "0-30 (good)" range.
[0045] The proxy layer uses the most stable standard MQTT over TCP connection to communicate with the remote cloud platform. To save bandwidth with limited satellite bandwidth, the proxy layer compresses the MQTT protocol message headers before sending. This ensures reliable basic data transmission over a relatively stable connection, while optimizing bandwidth through compression.
[0046] Assuming an initial network severity score of 45, the engine detects a change in score and immediately switches to a strategy within the "31-60 (moderate)" range. The proxy layer proactively establishes an accelerated UDP-based channel with the remote proxy. Compared to TCP, UDP has no congestion control overhead and offers lower latency even with some packet loss. To prevent message loss due to 8% packet loss, the engine applies forward error correction coding (FEC) to every MQTT message sent. For example, a message is divided into 5 data packets, encoded into one redundant packet (6 packets in total, with a redundancy rate of 20%). As long as no more than one packet is lost, the receiver can recover the data automatically without retransmission. If a sensor suddenly needs to upload a 2KB firmware log, the proxy layer automatically fragments it into two 1KB segments for separate transmission, preventing large messages from failing to transmit as a whole in a network prone to packet loss. Switching from TCP to UDP reduces protocol latency, and the 20% FEC effectively combats moderate packet loss, avoiding frequent retransmissions and maintaining smooth transmission.
[0047] When the network score spiked to 80, the engine immediately activated the highest-level "61-100 (Severe)" range strategy. The proxy layer upgraded the transport protocol from basic UDP to QUIC. QUIC implements reliable transmission on top of UDP and incorporates multiplexing (parallel transmission of multiple data streams over a single connection, avoiding head-of-line congestion) and 0-RTT fast reconnection (extremely fast recovery after connection interruption). To combat the 25% packet loss, the FEC redundancy rate was increased to 50%. For example, a critical temperature alarm message (such as "engine overheating") is encoded as 4 data packets + 2 redundant packets (RS(6,4)). Even if any two packets are lost, the message can still be completely recovered.
[0048] Step S104: Data transmission is performed between the MQTT client and the MQTT server according to the target transmission strategy.
[0049] In one embodiment of this application, the sampling frequency of the network indicator data can be adaptively changed. The comprehensive network score includes a first network score at a first time point and a second network score at a second time point. The method further includes: determining the score difference between the first network score and the second network score; and adjusting the sampling frequency of the network indicator data according to the score difference.
[0050] If the score difference is less than the first preset difference, the original sampling frequency is maintained; if the score difference is greater than or equal to the preset difference, the sampling frequency is increased.
[0051] In another embodiment of this application, the network indicator data includes first indicator data at a first time point and second indicator data at a second time point, wherein the first indicator data and the second indicator data are indicator data of the same type, and further includes: determining the indicator data difference between the first indicator data and the second indicator data; and adjusting the sampling frequency of the network indicator data according to the indicator data difference.
[0052] If the difference in the indicator data is less than the second preset difference, the original sampling frequency is maintained; if the difference in the indicator data is greater than or equal to the preset difference, the sampling frequency is increased.
[0053] In one embodiment of this application, determining the target transmission strategy corresponding to the comprehensive network score from a preset dynamic strategy matrix includes: determining the target transmission strategy corresponding to the comprehensive network score from the preset dynamic strategy matrix; determining the target score interval matched by the comprehensive network score in the preset dynamic strategy matrix; and determining the target transmission strategy corresponding to the target score interval.
[0054] In this embodiment, under scenarios such as satellite links (RTT>1s) and high packet loss wireless networks (packet loss rate>25%), the severity scoring error rate is <5%, significantly better than the traditional fixed threshold detection method. During network mutations (such as sudden disconnection), the policy adjustment latency is reduced from seconds in traditional solutions to within 200 milliseconds; through dynamic sampling, proxy layer power consumption is reduced by 40%, adapting to the low power consumption requirements of LPWAN devices.
[0055] The embodiments described above in this application have improved performance in harsh environments. For example, in high-latency satellite links (RTT=1200ms): message delivery success rate is increased from 68% to 95%, and end-to-end latency is reduced by 40%; in high-packet-loss wireless networks (packet loss rate 25%): effective throughput is increased by 3 times, and retransmissions are reduced by 75%; in frequent disconnection scenarios (5 disconnections per minute): connection recovery time is shortened from 8s to 1.2s, and message loss rate is reduced from 45% to 3%; in narrowband networks (bandwidth 10kbps): protocol overhead is reduced from 30% to 8%, and message throughput is increased by 5 times under the same bandwidth.
[0056] In this embodiment, when the MQTT client, the MQTT server, and the agent are detected to be connected, multiple network indicator data for evaluating the network in the current network environment can be obtained; then, a comprehensive network score for the current network environment can be determined based on the multiple network indicator data; a target transmission strategy corresponding to the comprehensive network score can be determined from a preset dynamic strategy matrix; and data transmission can be performed between the MQTT client and the MQTT server according to the target transmission strategy. This enables real-time monitoring of network status and allows data transmission to be performed by determining a dynamic transmission strategy based on the comprehensive network score of the real-time network environment.
[0057] Reference Figure 2a This diagram illustrates another data transmission method according to an embodiment of this application, applied to a proxy end. A proxy end for network optimization is deployed between the MQTT client and the MQTT server, and may specifically include the following steps: Step S201: When it is detected that the MQTT client, the MQTT server, and the broker are connected, obtain multiple network metric data for evaluating the network in the current network environment; In this embodiment, the MQTT client and MQTT server transmit data based on the MQTT protocol. However, when the network environment is poor, the MQTT protocol will have many defects in the transmission process. In order to solve the defects of the MQTT protocol in the face of poor network environment, this embodiment introduces a transparent poor network optimization proxy layer without modifying the existing MQTT client and server to realize intelligent optimization of the MQTT transmission process.
[0058] On the broker side, both the client and server appear as standard MQTT peers. The broker can intercept all passing MQTT traffic and perform real-time analysis, transformation, and optimization through its internal engine, thereby achieving adaptability to harsh network environments.
[0059] The agent can be materialized as a physical device (such as an industrial gateway), software middleware (such as a Docker container), or cloud service (such as an edge computing node). The agent mainly consists of three core engines: a network state awareness engine, a policy execution engine, and an enhanced MQTT transport layer.
[0060] Among them, the network status awareness engine can be used to sense and monitor the network status in real time, the policy execution engine can be used to execute different policies under different network conditions, and the enhanced MQTT transport layer can be used for data transmission processing under adverse network conditions.
[0061] When an MQTT client initiates a connection to an MQTT server, the client is actually connecting to a local optimized broker. The broker then establishes a connection with the remote MQTT server, posing as the client. This connection between the MQTT server, MQTT client, and broker facilitates subsequent data transmission optimization based on the broker.
[0062] After the connection is established, the network status awareness engine in the agent starts to obtain network indicator data, which may include, but is not limited to, any one or more of the following: round-trip time (RTT), basic packet loss rate, available bandwidth, dynamic RRT, packet loss rate, available bandwidth, and disconnection frequency per unit time.
[0063] Step S202: Determine the comprehensive network score of the current network environment based on the multiple network indicator data; After obtaining network indicator data, multiple network indicator data can be combined to determine the comprehensive network score in the current network environment. This comprehensive network score is used to assess the severity of network degradation. The higher the score, the higher the severity of network degradation, and the lower the score, the lower the severity of network degradation.
[0064] In one embodiment of this application, when an MQTT client initiates a connection, it actually connects to a local optimized broker. The broker then establishes a connection with the remote MQTT server as the client. At the initial stage of connection establishment, the network status awareness engine immediately starts, measuring initial network metrics, including round-trip time (RTT), basic packet loss rate, and available bandwidth, by sending probe packets, and calculating an initial network severity score. The awareness engine continuously monitors the network status, including dynamic RRT, packet loss rate, available bandwidth, and disconnection frequency per unit time. Severity score calculation: The engine calculates a comprehensive severity score based on real-time monitoring data. The comprehensive severity score can change with real-time network environment changes.
[0065] Step S203: Determine the target transmission strategy corresponding to the comprehensive network score from the preset dynamic strategy matrix; A dynamic policy matrix can be pre-configured on the proxy side, containing the correspondence between network scores and specific implementation policies. Therefore, after determining the overall network score, the target transmission policy corresponding to that score can be found based on the dynamic policy matrix. The proxy's policy execution engine can then select the optimal transmission policy from the dynamic policy matrix based on the overall network score.
[0066] Step S204: Data transmission is performed between the MQTT client and the MQTT server according to the target transmission strategy; Step S205: In response to receiving MQTT message data sent by the MQTT client at QoS 1 level, a preset enhanced transmission process is initiated.
[0067] QoS level 1, or Quality of Service Level 1, is known as "at least once" delivery in the MQTT protocol. Its core guarantee is that a message will be received by the receiver at least once. The sender will make every effort to ensure the receiver receives the message, and retries may be necessary in this process, so the receiver may receive duplicate messages.
[0068] In one embodiment of this application, after initiating a preset enhanced transmission process, the target message data can be encoded according to a preset format to obtain encoded data; the encoded data can be fragmented to obtain fragmented data; the fragmented data can be sent to a proxy instance of the server using the QUIC protocol, so that MQTT message data can be generated in the proxy instance based on the fragmented data, and the MQTT message data can be sent to the MQTT server at the QoS 1 level.
[0069] The preset format can be RS(10,7) encoding format.
[0070] In practical applications, the enhanced transmission process is as follows: a. Protocol Conversion and Upgrade: The client sends a QoS 1 PUBLISH message to the proxy layer. Upon receiving the message, the proxy layer parses and buffers it. Based on the policy, the proxy layer does not simply forward the message to the server via the original TCP connection. Instead, it initiates an enhanced transport process.
[0071] b. Application Encoding and Fragmentation: The proxy layer applies RS(10,7) encoding to the message content (i.e., the original data is divided into 7 packets, and 3 redundant packets are generated through encoding, for a total of 10 packets, which can tolerate a maximum of 3 packet loss). Subsequently, these 10 data packets are fragmented and sent to the remote proxy instance (deployed on the server side) through more efficient QUIC protocol stream multiplexing.
[0072] c. Reception and Reassembly: The proxy instance on the server side receives data packets. As long as the number of received packets is greater than or equal to 7, the original message can be completely decoded. The proxy instance then delivers the reassembled original MQTT message to the actual MQTT server at the standard QoS 1 level.
[0073] d. Acknowledgment Mechanism: An enhanced acknowledgment mechanism is maintained internally within the proxy layer to ensure reliable message delivery. For the client, it receives an acknowledgment based on the original QoS 1, completely unaware of its status.
[0074] In one embodiment of this application, when the connection between the proxy layer and the MQTT client and / or the MQTT server is detected to be interrupted, an optimized connection with the MQTT client and / or the MQTT server is rebuilt.
[0075] In practical applications, when the perception engine detects a connection interruption (especially in scenarios with frequent disconnections), it can perform a fast reconnection, that is, rebuild an optimized connection with the MQTT client and / or the MQTT server. Specifically, the 0-RTT fast reconnection method in QUIC can be used to achieve reconnection.
[0076] In one embodiment of this application, when the proxy layer reconnects with the MQTT client and / or the MQTT server, it acquires the key message status during the connection disconnection period and synchronizes the key message status to the proxy.
[0077] In practical applications, when the perception engine detects a connection recovery, it can achieve incremental data state synchronization. Specifically, it can obtain the key message status during the connection break based on the connection break time and the connection recovery time, and then synchronize the key message status to the agent.
[0078] In one embodiment of this application, a high-priority message for a target connected device is obtained from the MQTT server. The target connected device is an intermittently connected device. In response to a connection success command from the target connected device, the high-priority message is pushed to the target connected device. This reduces latency.
[0079] Reference Figure 2b This document illustrates an enhanced MQTT transport layer workflow diagram in an embodiment of this application. The workflow mainly includes the following steps: determining the priority of the MQTT message; when the priority is high, forced QoS2 can be performed; when the priority is normal, the original QoS is maintained; after transmitting the data to the peer, the receiving end can determine whether reassembly or FEC decoding is required; when reassembly or FEC decoding is required, fragment reassembly or error correction is performed; after restoring the MQTT message, it is delivered to the MQTT server.
[0080] In the embodiments of this application, under a 25% packet loss rate, the message delivery success rate can be increased from 72% to 98% and the number of retransmissions can be reduced by 75% through RS encoding + QUIC hybrid transmission; in a 10kbps narrowband network, the protocol overhead is reduced from 30% to 8% and the effective throughput is increased by 5 times; in frequent disconnection scenarios (5 times / minute), the session recovery time is shortened from 8s to 1.2s, and the client is unaware of it.
[0081] In this embodiment, when the MQTT client, the MQTT server, and the agent are detected to be connected, multiple network indicator data for evaluating the network in the current network environment can be obtained; then, a comprehensive network score for the current network environment can be determined based on the multiple network indicator data; a target transmission strategy corresponding to the comprehensive network score can be determined from a preset dynamic strategy matrix; and data transmission can be performed between the MQTT client and the MQTT server according to the target transmission strategy. This enables real-time monitoring of network status and allows data transmission to be performed by determining a dynamic transmission strategy based on the comprehensive network score of the real-time network environment.
[0082] It should be noted that the data transmission method provided in this application can be executed by a data transmission device, or a control module within that data transmission device for executing the loading data transmission method. This application uses the execution of the loading data transmission method by a data transmission device as an example to illustrate the data transmission method provided in this application.
[0083] Reference Figure 3 This illustration shows a schematic diagram of a data transmission device according to an embodiment of this application. Applied to a proxy end, it is deployed between the MQTT client and the MQTT server for network optimization. Specifically, it may include the following modules: The network metric data acquisition module 301 is used to acquire multiple network metric data for evaluating the network in the current network environment when the MQTT client, the MQTT server and the agent are detected to be connected. The comprehensive network score determination module 302 is used to determine the comprehensive network score of the current network environment based on the multiple network indicator data; The transmission strategy determination module 303 is used to determine the target transmission strategy corresponding to the comprehensive network score from a preset dynamic strategy matrix; The data transmission module 304 is used to transmit data between the MQTT client and the MQTT server according to the target transmission strategy.
[0084] In one embodiment of this application, the comprehensive network scoring determination module 302 may include: The weight data determination submodule is used to determine the weight data corresponding to each network indicator data, and the sum of the weight data of the multiple network indicator data is 1; The comprehensive network score determination submodule is used to perform weighted summation on each network indicator data according to the weight data to obtain the comprehensive network score of the current network environment.
[0085] In one embodiment of this application, the comprehensive network score includes a first network score at a first time point and a second network score at a second time point, and the apparatus further includes: The scoring interpolation determination module is used to determine the score difference between the first network score and the second network score; The first sampling frequency adjustment module is used to adjust the sampling frequency of the network indicator data according to the score difference.
[0086] In one embodiment of this application, the network indicator data includes first indicator data at a first time point and second indicator data at a second time point, wherein the first indicator data and the second indicator data are indicator data of the same type, and the device further includes: The indicator data difference determination module is used to determine the indicator data difference between the first indicator data and the second indicator data. The second sampling frequency determination module is used to adjust the sampling frequency of the network indicator data based on the difference in the indicator data.
[0087] In one embodiment of this application, the transmission strategy determination module 303 may include: The target transmission strategy determination submodule is used to determine the target transmission strategy corresponding to the comprehensive network score from a preset dynamic strategy matrix. The target score differentiation determination submodule is used to determine the target score range that matches the comprehensive network score in a preset dynamic strategy matrix; The target transmission strategy determination submodule is used to determine the target transmission strategy corresponding to the target scoring interval.
[0088] In one embodiment of this application, the apparatus may further include: The enhanced transmission process determination module is used to initiate a preset enhanced transmission process in response to receiving MQTT message data sent by the MQTT client at QoS level 1.
[0089] In one embodiment of this application, the apparatus may further include: The data encoding module is used to encode the target message data according to a preset format to obtain encoded data; The data fragmentation module is used to fragment the encoded data to obtain fragmented data. The fragmented data sending module is used to send the fragmented data to a proxy instance of the server using the QUIC protocol, so that the proxy instance generates MQTT message data based on the fragmented data and sends the MQTT message data to the MQTT server at the QoS 1 level.
[0090] In one embodiment of this application, the apparatus further includes: The reconnection module is used to rebuild an optimized connection with the MQTT client and / or the MQTT server when the connection between the broker layer and the MQTT client and / or the MQTT server is interrupted.
[0091] In one embodiment of this application, the apparatus may further include: The critical message status acquisition module is used to acquire the critical message status during the connection disconnection period when the broker layer resumes connection with the MQTT client and / or the MQTT server. The critical message status synchronization module is used to synchronize the critical message status to the agent.
[0092] In one embodiment of this application, the apparatus may further include: The high-priority message acquisition module is used to acquire high-priority messages from the target connected device from the MQTT server. The target connected device is a device with intermittent connections. The message push module is used to push the high-priority message to the target connected device in response to the connection success command of the target connected device.
[0093] In this embodiment, when the MQTT client, MQTT server, and broker are detected to be connected, multiple network indicator data for evaluating the network in the current network environment can be acquired. Then, a comprehensive network score for the current network environment is determined based on the multiple network indicator data. A target transmission strategy corresponding to the comprehensive network score is determined from a preset dynamic strategy matrix. Data transmission is performed between the MQTT client and the MQTT server according to the target transmission strategy. Through this embodiment, real-time monitoring of network status can be achieved, and a dynamic transmission strategy can be determined based on the comprehensive network score of the real-time network environment to implement data transmission.
[0094] The data transmission device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.
[0095] The data transmission device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.
[0096] The data transmission device provided in this application embodiment can achieve... Figures 1 to 2b The various processes implemented by the data transmission device in the method embodiment will not be described again here to avoid repetition.
[0097] Optionally, this application embodiment also provides an electronic device, including a processor 1010, a memory 1009, and a program or instructions stored in the memory 1009 and executable on the processor 1010. When the program or instructions are executed by the processor 1010, they implement the various processes of the above-described data transmission method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0098] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.
[0099] Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application. The electronic device 1000 includes, but is not limited to, the following components: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, and processor 1010.
[0100] The memory 1009 includes applications and an operating system; the user input unit 1007 may include a touch panel 10071 and other input devices 100072; the input unit 1004 may include an image processor 10041 and a microphone 10042; and the display unit 1006 may include a display panel 10061.
[0101] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here. This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described data transmission method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0102] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0103] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above data transmission method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0104] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.
[0105] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0107] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A data transmission method, characterized in that, The method, applied to the proxy side, involves deploying a proxy for network optimization between the MQTT client and the MQTT server. When the MQTT client, the MQTT server, and the broker are detected to be connected, multiple network metrics data for evaluating the network are obtained in the current network environment. A comprehensive network score for the current network environment is determined based on the aforementioned multiple network indicator data. The target transmission strategy corresponding to the comprehensive network score is determined from the preset dynamic strategy matrix; Data is transmitted between the MQTT client and the MQTT server according to the target transmission strategy.
2. The method according to claim 1, characterized in that, The determination of the comprehensive network score for the current network environment based on the multiple network indicator data includes: Determine the weight data corresponding to each network indicator data, and the sum of the weight data of the multiple network indicator data is 1; The network index data is weighted and summed according to the weight data to obtain the comprehensive network score of the current network environment.
3. The method according to claim 1, characterized in that, The comprehensive network score includes a first network score at a first time point and a second network score at a second time point, and also includes: Determine the score difference between the first network score and the second network score; The sampling frequency of the network indicator data is adjusted based on the score difference.
4. The method according to claim 1, characterized in that, The network indicator data includes first indicator data at a first time point and second indicator data at a second time point, wherein the first indicator data and the second indicator data are indicator data of the same type, and further includes: Determine the difference between the first indicator data and the second indicator data; The sampling frequency of the network indicator data is adjusted based on the difference in the indicator data.
5. The method according to claim 1, characterized in that, Determining the target transmission strategy corresponding to the comprehensive network score from the preset dynamic strategy matrix includes: The target score range for the comprehensive network score matching is determined in the preset dynamic strategy matrix; Determine the target transmission strategy corresponding to the target score range.
6. The method according to any one of claims 1 to 5, characterized in that, Also includes: In response to receiving MQTT message data sent by the MQTT client at QoS 1 level, a preset enhanced transmission process is initiated.
7. The method according to claim 6, characterized in that, After initiating the preset enhanced transmission process, it also includes: The target message data is encoded according to a preset format to obtain encoded data; The encoded data is then segmented into data fragments to obtain fragmented data; The fragmented data is sent to a proxy instance of the server using the QUIC protocol, so that the proxy instance generates MQTT message data based on the fragmented data and sends the MQTT message data to the MQTT server at the QoS 1 level.
8. The method according to claim 1, characterized in that, Also includes: When the connection between the proxy layer and the MQTT client and / or the MQTT server is interrupted, an optimized connection is rebuilt with the MQTT client and / or the MQTT server.
9. The method according to claim 8, characterized in that, Also includes: When the broker layer reconnects with the MQTT client and / or the MQTT server, it obtains the status of key messages during the connection disconnection period; The status of the key messages is synchronized to the agent.
10. The method according to claim 9, characterized in that, Also includes: Obtain high-priority messages from the target connected device from the MQTT server; the target connected device is a device with intermittent connections. In response to the connection success command from the target connected device, the high-priority message is pushed to the target connected device.
11. A data transmission device, characterized in that, An application on the proxy side, deploying a proxy for network optimization between the MQTT client and the MQTT server, the device includes: The network metric data acquisition module is used to acquire multiple network metric data for evaluating the network in the current network environment when the MQTT client, the MQTT server, and the agent are detected to be connected. The comprehensive network score determination module is used to determine the comprehensive network score of the current network environment based on the multiple network indicator data; The transmission strategy determination module is used to determine the target transmission strategy corresponding to the comprehensive network score from a preset dynamic strategy matrix; The data transmission module is used to transmit data between the MQTT client and the MQTT server according to the target transmission strategy.
12. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the data transmission method as described in any one of claims 1-10.
13. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the data transmission method as described in any one of claims 1-10.