Lightweight network small sample data automatic extraction method

Abstract

The invention relates to a lightweight network small sample data automatic extraction method. In a network data acquisition stage, an acquisition information transmission method based on compressed sensing is designed, and the method can efficiently compress original sparse data into small data packets, so that the data transmission quantity is remarkably reduced, and the integrity of the data in a transmission and reconstruction process is ensured. Meanwhile, a relay multi-hop forwarding mechanism based on a graph model is designed for special environments such as high mountains and canyons. According to the mechanism, by optimizing a data transmission path, the energy overhead in a data transmission and collection process is effectively reduced, so that the efficiency and reliability of data collection are improved. In a network data information extraction stage, the invention provides a long and short-term memory network method based on data enhancement. The method can effectively cope with the challenge of small scale of network samples, expands the data set through the data enhancement technology, and improves the generalization ability of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Things and artificial intelligence, and specifically relates to a method for automatically extracting lightweight network small sample data. Background Art

[0002] In today's era of digitalization and the intelligent connection of all things, the collection, processing and analysis of large-scale network data plays a vital role in solving various practical problems. However, in areas with complex geographical conditions and weak communication infrastructure such as high mountains and valleys, the acquisition and processing of network data becomes particularly difficult. High mountains and valleys are known for their steep peaks, deep canyons and varied terrain. These special landforms lead to frequent problems such as incomplete network signal coverage, unstable signal strength and difficulty in powering equipment. Traditional sensor network data extraction methods are difficult to use in such environments.

[0003] In order to cope with the difficulty of data collection of IoT nodes in mountainous and canyon areas, automatic data extraction technology has emerged. Through automated and lightweight means, this technology can remotely complete the data extraction of sensor nodes with almost no human involvement, thereby ensuring the real-time and accuracy of data extraction and reducing the energy consumption of IoT nodes. Therefore, solving the problem of automatic data extraction in mountainous and canyon areas is crucial to promoting applications such as resource development, environmental protection, and natural disaster monitoring in the region. This has also been the focus of attention of academia and industry in recent years.

[0004] Although the existing IoT data extraction technology has made some progress in improving the data extraction capabilities in mountainous and canyon areas, it still faces many challenges. On the one hand, due to the harsh geographical environment and high labor costs, IoT nodes in the region are usually designed as low-power sensors, which have small data collection volume and slow transmission speed, and can only provide limited data samples, which undoubtedly increases the difficulty of extracting effective information. On the other hand, factors such as complex terrain and incomplete signal coverage lead to problems such as incomplete and small-scale collected data samples. In addition, the network signal in the mountainous and canyon areas fluctuates greatly and the strength is unstable, which puts higher requirements on the real-time performance of data extraction.

[0005] In response to these challenges, the patent of this invention is dedicated to designing a lightweight network small sample data automatic extraction method. This method fully considers the variability of network status in alpine valley areas, the redundant heterogeneity of original collected data, and the small size of data samples, aiming to achieve low-latency, low-power data collection and high-accuracy data feature extraction, providing strong technical support for data processing and application in alpine valley areas. Summary of the invention

[0006] The present invention provides a lightweight network small sample data automatic extraction method, which specifically includes the design of two parts: data collection process and data processing process. In the data collection process, a data collection scheme based on compressed sensing is designed for each Internet of Things node, the original data collected by the node is sparsely expressed, and the sparse data is compressed into a smaller data packet for transmission through a random measurement matrix; a relay node multi-hop forwarding mechanism is introduced into the transmission process of multi-node compressed data packets, and the power consumption of the Internet of Things nodes deployed in the high mountain canyon area is reduced by sending the data packet to the relay node with a closer distance, and the node's continuous working ability is enhanced, so as to realize the efficient collection of data collected by the Internet of Things nodes in the high mountain canyon area. Data processing process. In the data extraction and processing process, the present invention proposes a long short-term memory network (Long Short-Term Memory, LSTM) based on data enhancement, processes the collected small sample time series data, and captures the time feature information of the data; the knowledge distillation technology is used to compress the constructed LSTM network model, so that the model can be deployed in low-power Internet of Things nodes with limited resources in high mountain canyons.

[0007] Specifically, the present invention is achieved through the following technical solutions:

[0008] A lightweight network small sample data automatic extraction method comprises the following steps: step S1: collecting sensor data of low-power Internet of Things nodes, and reducing the data transmission volume based on compressed sensing technology while ensuring that key information is not lost; step S2: designing a relay node selection mechanism based on the Steiner tree to achieve multi-hop forwarding and aggregation of data with minimized transmission power consumption; step S3: training and deploying a long short-term memory network based on data enhancement, extracting feature information of small sample sensor data, and combining knowledge distillation technology to achieve lightweight deep learning models so that they can be deployed on resource-constrained devices.

[0009] In one embodiment, a data collection mechanism is designed for low-power IoT nodes, the original collected data is sparsely represented based on compressed sensing, and the sparse signal is compressed into a smaller data packet through a random measurement matrix.

[0010] In a certain embodiment, the data receiving node decompresses and reconstructs the received compressed sensing data packet, decodes the transmission data into sparse data based on the compressed sensing matrix, and further uses the least squares data reconstruction algorithm to restore the data from the sparse data to the original data.

[0011] In a certain implementation, after the transmission data of each IoT node is compressed, the node transmission route is further planned, thereby realizing automatic data aggregation of large-scale IoT nodes.

[0012] In a certain implementation, the node transmission route is planned, the transmission relay node is selected, the transmission process of the sensor network in the mountain valley is constructed as a directed graph model, and the minimum Steiner tree in the graph model is found to determine the relay node selection scheme with the best transmission power consumption.

[0013] In a certain embodiment, the nodes in the graph model are sensor nodes in a high mountain valley, the directed edges indicate that the sensor nodes can communicate wirelessly, and the node weights are the static energy costs of the sensor nodes. The static energy cost here refers to the energy consumption per unit time to maintain the survival of the node when the sensor is not communicating data. The edge weight of the graph model is the energy consumption per unit time of the communication between two nodes. Based on the constructed directed graph model, the Prim algorithm is used to find the minimum Steiner tree to determine the relay node selection and multi-hop transmission routing strategy of the high mountain valley sensor network.

[0014] In one embodiment, data enhancement is performed on the small sample sensor data collected in the high mountain valley to expand the data set. Sensor data at different time points and different scales are simulated by random translation and scaling to increase the diversity and richness of the data.

[0015] In one embodiment, based on the enhanced sensor data set of the alpine canyon area, an LSTM network for time series data processing is constructed to capture the long-term temporal correlation between the data. The training process of the LSTM network adopts an Adam-based adaptive learning rate training method to update and tune the parameters to minimize the loss function and improve the performance of the model.

[0016] In one embodiment, knowledge distillation technology is used on the trained LSTM model to lightweight the model, and the knowledge of the original complex model is transferred to the lightweight model to achieve model compression. The network model after lightweight operation can be deployed to low-power IoT nodes in mountainous and canyon areas, thereby realizing efficient data feature extraction and analysis of the model in low-power IoT nodes.

[0017] The present invention has the following advantages and beneficial effects:

[0018] 1. Aiming at the challenges of limited energy consumption of IoT devices in high mountain valleys, unevenly dispersed device locations, and complex network communication environment, the present invention designs a data transmission scheme for IoT devices based on compressed sensing and a relay forwarding routing technology based on Steiner tree to reduce node transmission energy consumption, enhance node continuous working capability, and achieve efficient collection of sensor data.

[0019] 2. In view of the characteristics of small data samples and limited device resources collected by IoT devices in high mountain canyons, the present invention designs a long short-term memory network based on data enhancement to extract the features of collected data, and uses knowledge distillation technology to lightweight the constructed deep learning model, so that the model can be deployed in resource-constrained IoT nodes.

[0020] The technical solutions / features disclosed above are intended to summarize the technical solutions and technical features described in the specific implementation method section, so the scope of the records may not be exactly the same. However, the new technical solutions disclosed in this section are also part of the many technical solutions disclosed in the invention document. The technical features disclosed in this section, the technical features disclosed in the subsequent specific implementation method section, and some contents in the drawings that are not clearly described in the specification, disclose more technical solutions in a reasonable combination with each other.

[0021] The technical solution formed by combining all the technical features disclosed at any position of the present invention is used to support the generalization of the technical solution, the modification of the patent document, and the disclosure of the technical solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is an example diagram illustrating the lightweight network small sample data automatic extraction method for high mountain valley areas proposed by the present invention;

[0023] Figure 2 It is a schematic diagram illustrating the automatic extraction of features of small sample data in a lightweight network proposed in the present invention. DETAILED DESCRIPTION

[0024] Since it is impossible to describe all alternatives, the following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the key points of the technical solutions in the embodiments of the present invention. For other technical solutions and details not disclosed in detail below, they generally belong to technical goals or technical features that can be achieved by conventional means in the art. Due to space limitations, the present invention will not introduce them in detail.

[0025] Unless it means division, " / " in any position in the present invention means logical "or". The serial numbers such as "first" and "second" in any position in the present invention are only used as distinguishing marks for description, and do not imply an absolute order in time or space, nor do they imply that the terms prefixed with such serial numbers and the same terms prefixed with other attributives must have different references.

[0026] The present invention will describe various key points for combining into various specific embodiments, and these key points will be combined into various methods and products. In the present invention, even if the key points are only described when introducing the method / product solution, it means that the corresponding product / method solution also explicitly includes the technical features.

[0027] When describing the existence or inclusion of a certain step, module, or feature at any position in the present invention, it does not imply that such existence is exclusive and unique. Those skilled in the art can obtain other embodiments based on the technical solution disclosed in the present invention and supplemented by other technical means. Based on the key points described in the specific embodiments of the present invention, those skilled in the art can completely replace, delete, add, combine, and change the order of certain technical features to obtain a technical solution that still follows the concept of the present invention. These solutions that do not deviate from the technical concept of the present invention are also within the scope of protection of the present invention.

[0028] like Figure 1 As shown in the figure, this example architecture carefully designs a lightweight network small sample data automatic extraction method suitable for high mountain valley areas. The method mainly includes the following three key links: First, we adopt a data collection scheme based on compressed sensing to realize the lightweight transmission of IoT node collected data. The key to this step is to reduce the amount of data in the transmission process through efficient data compression technology to adapt to the characteristics of limited communication infrastructure and unstable signals in high mountain valley areas.

[0029] Secondly, in order to achieve efficient collection of collected data in the high mountain valley area, we combined the relay multi-hop forwarding mechanism. By building a network graph model and calculating the minimum Steiner tree, we optimized the data transmission path and reduced the loss and delay during the transmission process, thus ensuring that the data can be accurately and quickly collected to the data processing center.

[0030] Finally, in the data processing stage, we used a long short-term memory network based on data enhancement to extract data feature information. This method can not only make full use of limited data samples, expand the data set through data enhancement technology, and improve the generalization ability of the model; but also use the long short-term memory network to effectively capture the temporal dependencies in the data and extract valuable information for practical applications. At the same time, in order to better adapt to the automatic data extraction of low-power IoT devices in high mountain valleys, we also combined knowledge distillation technology to achieve lightweight processing of the model. Through this step, we can reduce the complexity of the model while ensuring the performance of the model, reduce the consumption of computing resources, and further improve the efficiency and accuracy of data extraction.

[0031] The following will provide a detailed description of these three stages so that readers can have a deeper understanding of the principles and implementation process of this method.

[0032] In the first phase, we are committed to designing a data collection mechanism for low-power IoT nodes, with the core goal of reducing the amount of data transmission while ensuring that key information is fully preserved. To achieve this goal, we plan to use a mechanism that combines compressed sensing with the least squares method.

[0033] At the node of the data sender, we select a suitable measurement matrix A and use the random measurement matrix to compress the original data. Specifically, let the sensor data be vector x, and its sparse representation be vector s, that is, x can be converted to s through the sparse representation. Then, we use the random measurement matrix A to compress the sensor data x to generate compressed data y with a small volume but rich information content. This process can be expressed as a mathematical formula y=Ax, where y is the transmitted data, which has a much smaller data volume than the original data x, but contains key information components.

[0034] Through this mechanism, we can ensure the integrity and accuracy of key information while ensuring data transmission efficiency, laying a solid foundation for subsequent data processing and analysis.

[0035] The data receiving node decompresses and reconstructs the received compressed data y. First, the compressed sensing measurement matrix A is used to decode the compressed data y into sparse data s, that is, y = As. Then, the least squares method is used to reconstruct the data, and the original data x is solved by minimizing the sum of squares of the reconstruction error, and the sparse data s is restored to the original data x. This process can be symbolically expressed as:

[0036]

[0037] in, is the reconstructed sensor data.

[0038] In the second phase, we are committed to designing a multi-hop forwarding mechanism for node data in mountainous and canyon areas, aiming to achieve efficient aggregation of collected data in the area. After completing the compression of IoT node transmission data, we need to carefully plan the node transmission route to ensure that large-scale IoT nodes can automatically complete the data aggregation task.

[0039] Specifically, we first construct a directed graph model of sensor nodes and relay nodes in the alpine valley. In this model, the vertices of the graph represent sensor nodes in the alpine valley area, and the directed edges between nodes represent the communication links between them. The weights of these edges are set as the energy cost per unit of communication between nodes, while the weight of the point reflects the static energy cost per unit time of the IoT node.

[0040] Next, based on this constructed directed graph structure, we look for the minimum Steiner tree in the graph. This step is crucial because it can effectively reduce the total energy consumption of regional IoT nodes while ensuring that all node communication tasks are completed. In the process of building the Steiner tree, we first select a starting node as the root node of the tree, and then gradually add nodes according to the minimum weight rule until all terminal nodes are included in the tree.

[0041] This process of finding the minimum Steiner tree can be formalized as an optimization problem. We use Prim's algorithm to efficiently solve this problem, ensuring that the data collection process is both efficient and energy-efficient. Through this design, we can achieve efficient collection of IoT node data in the complex terrain environment of high mountains and valleys, providing reliable support for subsequent data analysis and applications.

[0042]

[0043]

[0044]

[0045]

[0046] Where E represents the directed edge set of the directed graph, V represents the vertex set of the directed graph, and x ij Indicates whether the edge from node i to node j is selected, c ij represents the communication energy cost from node i to node j.

[0047] After completing the construction of the Steiner tree, we further determined the selection scheme of relay nodes and formulated a multi-hop transmission routing strategy for data. This strategy aims to minimize the power consumption of data transmission and effectively aggregate data. By carefully selecting relay nodes and planning transmission paths, we can reduce the energy consumption of IoT nodes while ensuring data transmission efficiency, ensuring that data can be accurately and efficiently aggregated to the data center.

[0048] Entering the third stage, we designed a lightweight long short-term memory network based on data enhancement to better adapt to the automatic data extraction of low-power IoT devices in high mountain valleys. This stage has the following significant features:

[0049] First, we performed a series of data augmentation operations on the small sample sensor data collected in the high mountain valley to expand the dataset. These operations include random translation and random scaling, which are designed to simulate sensor data at different time points and different scales, thereby increasing the diversity and richness of the data. Through data augmentation, we can make full use of limited sample data and improve the generalization ability and accuracy of the model.

[0050] Secondly, based on the enhanced sensor dataset of the alpine valley area, we built an LSTM network for time series data processing. The LSTM network has the ability to capture long-term temporal correlations between data and is very suitable for processing IoT sensor data. During the training process, we adopted an adaptive learning rate training method based on Adam to minimize the loss function and improve the performance of the model by continuously updating and tuning the network parameters.

[0051] Specifically, the LSTM network model includes an input gate, a forget gate, an output gate, and a memory unit. The input gate is responsible for controlling the input of new information, the forget gate determines the degree of retention of past information, and the output gate controls the selection of output information. The memory unit is the core of the LSTM network, which is used to store and update information. Through the collaborative work of these gates, the LSTM network can effectively process time series data and extract key feature information.

[0052] Through the design and implementation of this stage, we are able to achieve automatic extraction and efficient processing of IoT sensor data in mountainous canyon areas, providing strong support for subsequent applications.

[0053] f t =σ g (W f ·[h t-1 ,x t ]+b f )

[0054] Among them, f t is the output of the forget gate, W f is the weight of the forget gate, h t-1 is the hidden state at the previous moment, x t is the input at the current moment, b f is the bias term.

[0055] After completing the training of the LSTM network, we further applied knowledge distillation technology to achieve the lightweight model. The core idea of ​​this technology is to effectively transfer the knowledge of the original complex model to the lightweight model, so as to achieve model compression without sacrificing too much performance. Through knowledge distillation, we can significantly reduce the number of model parameters and computational complexity while maintaining the model's ability to extract data features.

[0056] Specifically, the temperature parameter and distillation loss function are introduced in the knowledge distillation process. The temperature parameter is used to control the degree of dependence of the student model (i.e. the lightweight model) on the output of the teacher model (i.e. the original complex model). By adjusting the temperature parameter, we can make the lightweight model more dependent on the output of the teacher model during the learning process, so as to learn the characteristic information of the data more effectively. The distillation loss function is used to measure the difference between the student model and the teacher model. By minimizing this difference, we can achieve effective knowledge transfer.

[0057] After the lightweight operation, the network model has a smaller size and lower computing requirements, so it can be easily deployed on low-power IoT nodes in mountainous and canyon areas. This enables our model to achieve efficient data feature extraction and analysis in these resource-constrained environments, providing strong support for practical applications.

[0058] In summary, by combining LSTM networks and knowledge distillation technology, we have successfully designed a data automatic extraction method suitable for low-power IoT nodes in mountainous canyon areas. This method not only has efficient data processing capabilities, but also can achieve lightweight models while maintaining performance, injecting new vitality into the development of the IoT field. The distillation loss L of knowledge distillation in this process is KD It can be calculated by the following formula:

[0059]

[0060] Among them, α is the factor that weighs the distillation loss and the original loss, T is the temperature parameter, and p i and q i are the prediction results of the teacher model and the student model respectively.

Claims

1. A lightweight network small sample data automatic extraction method, characterized in that: Includes the following steps: Step S1: Collect sensor data from low-power IoT nodes, and reduce the amount of data transmission based on compressed sensing technology while ensuring that key information is not lost; Step S2: Design a relay node selection mechanism based on the Steiner tree to achieve data multi-hop forwarding aggregation with minimized transmission power consumption; Step S3: Train and deploy a long short-term memory network based on data enhancement, extract feature information of small sample sensor data, and combine knowledge distillation technology to achieve lightweight deep learning models so that they can be deployed on resource-constrained devices.

2. The method for automatically extracting small sample data from a lightweight network according to claim 1 is characterized by: In the step S1, a data collection mechanism for low-power IoT nodes is designed, the original collected data is sparsely represented based on compressed sensing, and the sparse signal is compressed into a smaller data packet through a random measurement matrix.

3. The method for automatically extracting small sample data from a lightweight network according to claim 1 is characterized by: In step S1, the data receiving node decompresses and reconstructs the received compressed sensing data packet, decodes the transmission data into sparse data based on the compressed sensing matrix, and further uses the least squares data reconstruction algorithm to restore the data from the sparse data to the original data.

4. The method for automatically extracting small sample data from a lightweight network according to claim 1, characterized in that: In the step S2, after compressing the transmission data of each IoT node, the node transmission route is further planned, thereby realizing automatic data collection of large-scale IoT nodes.

5. The method for automatically extracting small sample data from a lightweight network according to claim 1, characterized in that: In step S2, the node transmission route is planned, the transmission relay node is selected, the transmission process of the sensor network in the mountain valley is constructed as a directed graph model, and the minimum Steiner tree in the graph model is found to determine the relay node selection scheme with the best transmission power consumption.

6. The method for automatically extracting small sample data from a lightweight network according to claim 1, characterized in that: In step S2, the nodes in the graph model are sensor nodes, the directed edges indicate that the sensor nodes can perform wireless communication, and the node weights are the static energy costs of the sensor nodes; the static energy costs here refer to the energy consumption of maintaining the survival of the nodes per unit time when the sensors are not performing data communication; the edge weights of the graph model are the energy consumption of the communication between the two nodes per unit time; based on the constructed directed graph model, the Prim algorithm is used to find the minimum Steiner tree to determine the relay node selection and multi-hop transmission routing strategy of the high mountain canyon sensor network.

7. The method for automatically extracting small sample data from a lightweight network according to claim 1, characterized in that: In step S2, data enhancement is performed on the small sample sensor data to expand the data set; sensor data at different time points and different scales are simulated by random translation and scaling to increase the diversity and richness of the data.

8. The method for automatically extracting small sample data from a lightweight network according to claim 1, characterized in that: In step S3, based on the enhanced sensor data set, an LSTM network for time series data processing is constructed to capture the long-term temporal correlation between data. The training process of the LSTM network adopts an Adam-based adaptive learning rate training method to update and tune parameters to minimize the loss function and improve the performance of the model.

9. The method for automatically extracting small sample data from a lightweight network according to claim 1, characterized in that: In step S3, knowledge distillation technology is used on the trained LSTM model to lightweight the model, and the knowledge of the original complex model is transferred to the lightweight model to achieve model compression; the network model after the lightweight operation can be deployed to a low-power IoT node, thereby achieving efficient data feature extraction and analysis of the model in the low-power IoT node.

Images