A multi-path based large-scale sensor data aggregation method

Abstract

The application discloses a kind of large-scale sensor data gathering methods based on multi-path, wherein method includes, deployment sensor acquisition environment data, sensor node random clustering, establish the communication link between sensor source node, cluster head node and gathering node;Utilize multi-path selection algorithm, select the best transmission path, realize system total benefit maximization;Cluster head node utilizes dynamic data packet binding (DPB) algorithm, after data packet is bound processing, send to gathering node again;After gathering node receives data, data packet reconstruction is carried out, and original data information is obtained;The application based on multi-path wireless sensor network data gathering method and system can effectively carry out the collection and gathering of data, accelerate the transmission rate and delivery rate of data through multi-path transmission protocol, ensure the delay constraint of data, reduce system energy demand, greatly improve the resource utilization rate of system and the reliability of system.

Description

Technical Field

[0001] This invention relates to the field of large-scale sensor data aggregation technology, and in particular to a method for large-scale sensor data aggregation based on multiple paths. Background Technology

[0002] In large-scale wireless sensor networks, data aggregation solutions involve data sensing at the sensor level, with the central base station acting as the aggregation node and cluster head nodes serving as relay nodes for communication between multiple terminals. This means that sensor nodes sense and collect a large amount of data from their surroundings and transmit it to the aggregation node. Depending on the transmission method or other factors, existing data aggregation methods mainly fall into the following categories:

[0003] Data aggregation methods based on communication protocols: For situations where the sensor node and the aggregation node are close to each other, protocols such as ZigBee are used, while for situations where the distance is far, protocols such as LoRa and NB-IoT are used. In addition, the SigFox protocol is suitable for scenarios with low data transmission rate requirements, while for scenarios with high data transmission rate requirements, protocols such as WiFi and 5G are used.

[0004] Data aggregation methods based on data processing techniques can be further divided into real-time data aggregation and non-real-time data aggregation. In real-time data aggregation, sensor nodes collect data in real time, preprocess it, and immediately send it to the aggregation node. In non-real-time data aggregation, sensor nodes store a certain amount of data locally and send it to the aggregation node when conditions are suitable, such as through timed transmission or event-triggered transmission.

[0005] Network topology-based data aggregation methods include planar network and hierarchical network data aggregation. In planar network data aggregation, all sensor nodes are at the same level, and data is transmitted to the aggregation node through neighboring nodes. In hierarchical network data aggregation, sensor nodes are divided into one or more aggregation layers and a normal layer, and data is aggregated layer by layer to the aggregation node.

[0006] Existing large-scale wireless sensor aggregation methods have the following drawbacks: 1) Communication protocol-based data aggregation methods: Due to the varying coverage and capabilities of different communication protocols, they may not function effectively in specific environments. Furthermore, wireless communication modules are typically the most energy-intensive parts of sensor nodes, and some protocols may consume excessive energy during data transmission, affecting node lifespan. 2) Data aggregation methods based on data processing methods: Real-time data aggregation methods require high-speed data processing and transmission capabilities, but in resource-constrained sensor networks, this can make it difficult to guarantee real-time performance, especially under heavy network loads. Non-real-time data aggregation methods require local data storage at nodes, increasing the risk of data leakage and corruption. Additionally, limited storage space may prevent the storage of long-term data. 3) Network topology-based data aggregation methods: Planar network structures experience a significant performance degradation as the number of nodes increases. While hierarchical network structures can improve network scalability, the complexity of maintaining the network structure increases with high node density or dynamic network changes.

[0007] Another drawback of existing technologies is that when the number of sensor nodes is very large, the amount of data transmitted will be extremely large, causing network congestion, data loss, and system lifecycle crises.

[0008] Invention application No. 202411222399.8 discloses a method and device for data acquisition and transmission from multiple types of sensors. This solution uses an MCU to control hardware channel switching, ensuring the matching of access channels with sensor types. This enables real-time online remote monitoring of disasters in rugged and complex mountainous areas, ensuring the safe operation of railways along the line. However, this solution also suffers from the problem of excessive energy consumption during data transmission, affecting the lifespan of the nodes.

[0009] Therefore, there is a need for a method that can increase data aggregation rate while reducing the energy consumption of sensor networks, extending the lifespan of sensor networks, and enhancing the user experience. Summary of the Invention

[0010] The purpose of this invention is to provide a method for large-scale sensor data aggregation based on multipath transmission. This method achieves data aggregation in large-scale wireless sensor networks through multipath transmission technology, and can extend the life cycle of sensor networks and achieve load balancing.

[0011] This invention provides a method for large-scale sensor data aggregation based on multi-path, comprising the following steps:

[0012] S1. Deploy sensor nodes in the monitoring area to collect environmental data and assign a unique identifier to each sensor node;

[0013] S2. Randomly cluster the sensor nodes and assign a unique identifier to each cluster;

[0014] S3. Based on the identifiers, establish communication links between sensor source nodes, cluster head nodes, and aggregation nodes, and construct a multipath transmission network based on the MPQUIC protocol;

[0015] S4. Utilize a multi-path selection algorithm to select the optimal transmission path and maximize the overall system efficiency of data transmission from the source node to the sink node.

[0016] S5. The cluster head node uses the Dynamic Packet Bundling (DPB) algorithm to bundle data packets before sending them to the aggregation node.

[0017] S6. After receiving the data, the aggregation node reconstructs the data packet to obtain the original data information.

[0018] Furthermore, the set of environmental data collected in S1 is represented as J = {j 1,1 ...j 1,i ...j m,i Each data feature is represented by a quadruple as follows:

[0019]

[0020] in, For the time of collection, For data size, For data latency constraints, This represents the number of CPU cycles required to process each KB of data.

[0021] Furthermore, sensor nodes are classified into sleep, standby, and running states based on their current remaining energy and storage capacity, and whether a sensor node is a source node is determined based on its current state.

[0022] Furthermore, the communication links between the source node, cluster head node, and aggregation node include both 5G and WiFi paths.

[0023] Furthermore, the optimal transmission path in S4 is selected based on a path weight function, expressed by the following formula:

[0024]

[0025] Where B is the bandwidth, T RTT For the delay of each path, P loss For jitter and packet loss rate.

[0026] Furthermore, the Dynamic Data Packet Bundling (DPB) algorithm in S5 includes the following steps:

[0027] S51, Create an empty data fragment p binding ;

[0028] S52, Receive Buffer D pq Data packet p in m,i ;

[0029] S53, based on data packet p m,i The latency constraints and the set latency threshold determine whether data needs to be bundled, as expressed by the formula:

[0030]

[0031] in, For delay constraints, T max This is the time delay threshold;

[0032] S54. When the delay constraint is greater than the delay threshold, the data packet p... m,i Assemble into empty data fragment p binding Middle binding;

[0033] S55. If the delay constraint is less than the delay threshold, then add it directly to the sending buffer.

[0034] S56. Repeat S12-S15 for iteration, when empty data is partitioned into p... binding When full, insert into the send buffer. And create a new empty data fragment p binding .

[0035] Furthermore, the empty data fragment p binding For data packet p m,i When bundling, add header information, including source address, destination address, packet type, and packet quantity.

[0036] Furthermore, the formula for maximizing the total system benefit in S4 is as follows:

[0037]

[0038] Where, j m,i For data, For the total transmission delay, The total energy consumption of the system. For the weighting factor;

[0039]

[0040]

[0041] in, For transmission delay, To bundle latency, For transmission energy consumption, This is to bundle energy consumption.

[0042] Further, in step S6, data packet reconstruction is performed to obtain the original data information, including the following steps:

[0043] S61. The aggregation node establishes a mapping table Y(PSN,SSN) to associate the fragment sequence number PSN with the original data stream SSN;

[0044] S62. Check if all data fragments have arrived. If a data fragment has not arrived but the current time has exceeded the data delay constraint, do not retransmit it. If the delay constraint has not been exceeded, retransmit the data from the cluster head node's data buffer.

[0045] S63. Reassemble the data fragments into original data packets according to the fragment sequence number, and check the data integrity using CRC.

[0046] Furthermore, in step S61, a mapping table Y(PSN,SSN) is established to associate the fragment sequence number PSN with the original data stream SSN;

[0047] Based on the header information of each data segment, the sensor identifier, timestamp, and segment sequence number are extracted to determine its position in the original data.

[0048] The beneficial effects of this invention are:

[0049] 1. The present invention provides a multi-path wireless sensor network data aggregation method and system, which can effectively collect and aggregate data. Through multi-path transmission protocols, it accelerates the data transmission rate and delivery rate, ensures data latency constraints, reduces system energy requirements, and greatly improves system resource utilization and system reliability.

[0050] 2. The present invention is based on a multi-path wireless sensor network data aggregation method and system. It also utilizes a data bundling (DPB) algorithm to adaptively bundle data, thereby reducing the amount of data during transmission and increasing the upper limit of the number of sensors that the system can accommodate. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the structure of the large-scale sensor data aggregation method based on multiple paths according to the present invention;

[0052] Figure 2 This is a schematic diagram of the process of the large-scale sensor data aggregation device based on multiple paths according to the present invention;

[0053] Figure 3 This is a flowchart illustrating the dynamic data packet bundling algorithm of the present invention. Detailed Implementation

[0054] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0055] Existing large-scale wireless sensor aggregation methods suffer from problems such as high node energy consumption, which affects node lifespan, limited storage space, and the inability to store long-term data. When the scale of sensor nodes is very large, the amount of data transmitted will be extremely large, which will also cause network congestion, data loss, and system life cycle crisis.

[0056] To address the aforementioned problems, this invention provides a method for large-scale sensor data aggregation based on multi-path methods. Figure 2 This is a flowchart illustrating the aggregation method provided in an embodiment of the present invention. The method includes the following steps:

[0057] S1. Deploy sensor nodes in the monitoring area to collect environmental data, ensuring that the nodes cover the monitoring area, and assign a unique identifier to each sensor node.

[0058] like Figure 1 As shown, the monitoring area is divided into grids of equal size, and each sensor node is randomly distributed in a grid. Each node can collect environmental data of the monitoring area, including one or more attributes, such as temperature, humidity, light intensity, decibels, etc. A unique identifier is assigned to each sensor so that each node can be identified and tracked in the subsequent data collection and transmission process.

[0059] When collecting environmental data, use S = {S1, S2, S3, ..., S} M} represents a sensor node in a cluster, and the collected data set is represented as J = {j 1,1 ...j 1,i ...j m,i Each data feature can be represented by a quadruple as follows:

[0060]

[0061] in, For the time of collection, For data size, For data latency constraints, This represents the number of CPU cycles required to process each KB of data.

[0062] S2. Randomly cluster the sensor nodes and assign a unique identifier to each cluster. Data transmission is divided into intra-cluster transmission and inter-cluster transmission.

[0063] All sensor nodes can be randomly divided into P clusters. Based on the current remaining energy and storage size of the sensor nodes, they can be set to sleep, standby, or running states.

[0064] Within each cluster, the current state of each sensor node is monitored, such as energy level and storage size, to determine its operating state: sleep, standby, or running. Based on set parameter thresholds, the sensor node with the best state is selected as the source node for data collection; data is collected from the source node, and non-source nodes do not collect data.

[0065] The data aggregation process is modeled as a two-hop transmission: intra-cluster transmission and inter-cluster transmission. In intra-cluster transmission, data is transmitted step by step from within the cluster to the cluster head node, while in inter-cluster transmission, the cluster head node sends the data to the aggregation node.

[0066] S3. Establish communication links between sensor source nodes, cluster head nodes, and aggregation nodes to construct a multipath transmission network based on the MPQUIC protocol.

[0067] End-to-end transmission occurs between the source node, cluster head node, and aggregation node. Each transmission link has at least two available paths, including 5G and WiFi.

[0068] The MPQUIC protocol stack is implemented on each node to support multipath data transmission over MPQUIC. MPQUIC-related flow multiplexing, congestion control, and connection migration are configured. Connections are established via MPQUIC, and data transmission is performed on each path. After a series of negotiations, an end-to-end connection is established between the aggregation node and the sensor nodes. This connection enables data transmission during the data aggregation process through multiple paths. The MPQUIC protocol is deployed on the sensor nodes and cluster head nodes, congestion control is configured as CUBIC, and connection migration is enabled.

[0069] S4. Utilize a multi-path selection algorithm to select the optimal transmission path and maximize the overall efficiency of the data transmission system between the source node and the sink node.

[0070] Sensor nodes perceive their surroundings and collect environmental data such as temperature, humidity, and light intensity. This data is then divided into multiple data packets and sent to the aggregation node via different paths. Data transmission along each path is encapsulated using the MPQUIC protocol.

[0071] The optimal transmission path is selected based on the latency, bandwidth, packet loss rate, and jitter metrics of each path. Let X represent the path selection scheme for all nodes. A value of 0 or 1 indicates that data j is being transmitted. m,iThe selected path is 0 for the first path and 1 for the second path. The node segments the data, and each data packet is sent to the aggregation node through a different path. When data is lost, the data packet is retransmitted according to the urgency of the data. If the data delay constraint has not been exceeded, a new path can be selected for transmission. If the data has timed out, no retransmission is required.

[0072] The optimal transmission path is selected based on the path weight function, expressed by the formula:

[0073]

[0074] Where B is the bandwidth, T RTT为 Delay for each path, P loss For jitter and packet loss rate.

[0075] Continuously monitor network status to maximize overall system benefits, including link quality and node energy consumption, dynamically adjust MPQUIC connection path selection, and optimize transmission performance.

[0076] The formula for maximizing the overall system benefit is as follows:

[0077]

[0078] Where, j m,i For data, For the total transmission delay, The total energy consumption of the system. For the weighting factor;

[0079]

[0080] in, For transmission delay, To bundle latency, For transmission energy consumption, This is to bundle energy consumption.

[0081] By balancing the minimization of total system energy consumption and total transmission latency, the system's overall benefits are maximized, and the optimal transmission path is selected.

[0082] S5. The cluster head node uses the Dynamic Packet Bundling (DPB) algorithm to bundle data packets before sending them to the aggregation node.

[0083] The cluster head node collects data from each sensor node, compresses and preprocesses the collected raw data, and sets up a receive buffer at the cluster head node to temporarily store the preprocessed data packets. The cluster head node uses the Dynamic Packet Bundling (DPB) algorithm to process the data before sending it to the sink node.

[0084] This includes setting a latency threshold based on the latency constraints of data packets. For data packets with latency constraints greater than this threshold, a new data fragment is created, and data packets that meet the requirements are bundled and inserted as payloads. Necessary header information, including source address, destination address, data packet type, and number of data packets, is added to the bundled new data packets, and the transmission order of the bundled data packets is ensured. For data packets with latency constraints less than the latency threshold, they are directly sent to the aggregation node through the send buffer.

[0085] Specifically, such as Figure 3 As shown, the steps include:

[0086] S11, Create empty data partition p binding ;

[0087] S12, Receive Buffer D pq Data packet p in m,i ;

[0088] S13, Based on data packet p m,i The latency constraints and the set latency threshold determine whether data needs to be bundled, as expressed by the formula:

[0089]

[0090] in, For delay constraints, T max This is the time delay threshold;

[0091] S14. When the delay constraint is greater than the delay threshold, the data packet p... m,i Assemble into empty data fragment p binding Middle binding;

[0092] S15. If the delay constraint is less than the delay threshold, then add it directly to the sending buffer.

[0093] S16. Repeat S12-S15 for iteration. When empty data is partitioned into p... binding When full, insert into the send buffer. And create a new empty data fragment p binding .

[0094] S6. After receiving the data, the aggregation node reconstructs the data packet to obtain the original data information.

[0095] The aggregation node receives data packets from the cluster head node, parses the header information of the data packets, including source address, destination address, data packet type, data packet quantity, etc., and extracts sensor identifiers, timestamps, and fragment sequence number information.

[0096] The aggregation node determines the position of each data shard in the original data based on the header information of each data shard, establishes a mapping table Y(PSN,SSN), and associates the shard sequence number PSN with the original data stream SSN.

[0097] Check if all data fragments have arrived. If a data fragment has not arrived but the current time has exceeded the data delay constraint, do not retransmit it. If the delay constraint has not been exceeded, retransmit the data from the cluster head node's data buffer.

[0098] The bundled data packet is unpacked into multiple original data packets. The unpacked data is then verified for integrity using Cyclic Redundancy Check (CRC) to ensure that the data packets have not been tampered with during transmission.

[0099] Then, the data is sorted and reassembled based on the metadata (serial number, timestamp) in the data packet to restore the original order of the data. The reassembled data packet is then decompressed to parse the data into specific sensor readings or values.

[0100] This invention combines multipath transmission and large-scale sensor data aggregation, which can increase the data aggregation rate while reducing the energy consumption of the sensor network, extending the life cycle of the sensor network, and enhancing the user experience, thus possessing high innovation and practical value.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for large-scale sensor data aggregation based on multi-path, characterized in that, include: S1. Deploy sensor nodes in the monitoring area to collect environmental data and assign a unique identifier to each sensor node; S2. Randomly cluster the sensor nodes and assign a unique identifier to each cluster; S3. Based on the identifiers, establish communication links between sensor source nodes, cluster head nodes, and aggregation nodes, and construct a multipath transmission network based on the MPQUIC protocol; S4. Utilize a multi-path selection algorithm to select the optimal transmission path and maximize the overall system efficiency of data transmission from the source node to the sink node. S5. The cluster head node uses the Dynamic Packet Bundling (DPB) algorithm to bundle data packets before sending them to the aggregation node. S6. After receiving the data, the aggregation node reconstructs the data packet to obtain the original data information; The communication links between the source node, cluster head node, and aggregation node include two paths: 5G and WiFi. The optimal transmission path in S4 is selected based on a path weight function, expressed by the following formula: Where B is the bandwidth, T RTT For the delay of each path, P loss For jitter and packet loss rate; The dynamic data packet bundling DPB algorithm in S5 includes the following steps: S51. Create empty data fragments ; S52, Receive Buffer Data packet p in m,i ; S53, based on data packet p m,i The latency constraints and the set latency threshold determine whether data needs to be bundled, as expressed by the formula: in, For delay constraints, This is the time delay threshold; S54. When the delay constraint is greater than the delay threshold, the data packet p... m,i Assemble into empty data fragments Middle binding; S55. If the delay constraint is less than the delay threshold, then add it directly to the transmission buffer. ; S56. Repeat S52-S55 iteratively, when empty data is fragmented. When full, insert into the send buffer. and create new empty data shards. ; The formula for maximizing the total system benefit in S4 is as follows: Where, j m,i For data, For the total transmission delay, The total energy consumption of the system. , For the weighting factor; in, For transmission delay, To bundle latency, For transmission energy consumption, For bundled energy consumption; In step S6, data packet reconstruction is performed to obtain the original data information, including the following steps: S61. Establish a mapping table at the aggregation node. Associate the fragment sequence number (PSN) with the original data stream SSN; S62. Check if all data fragments have arrived. If a data fragment has not arrived but the current time has exceeded the data delay constraint, do not retransmit it. If the delay constraint has not been exceeded, retransmit the data from the cluster head node's data buffer. S63. Reassemble the data fragments into original data packets according to the fragment sequence number, and check the data integrity using CRC. S61 establishes a mapping table When associating the fragment sequence number (PSN) with the original data stream SSN; Based on the header information of each data segment, the sensor identifier, timestamp, and segment sequence number are extracted to determine its position in the original data.

2. The convergence method according to claim 1, characterized in that, The environmental data set collected in S1 is represented as follows: Each data feature is represented by a quadruple as follows: in, For the time of collection, For data size, For data latency constraints, This represents the number of CPU cycles required to process each KB of data.

3. The convergence method according to claim 1, characterized in that, Sensor nodes are classified into sleep, standby, and running states based on their current remaining energy and storage capacity. Whether a sensor node is a source node is determined based on its current state.

4. The convergence method according to claim 1, characterized in that, The empty data fragmentation For data packet p m,i When bundling, add header information, including source address, destination address, packet type, and packet quantity.

Images