A method and system for processing Internet of Things data based on edge computing

Abstract

The present invention discloses an Internet of Things data processing method and system based on edge computing, which relates to the field of data processing technology, including using a multimodal sensor network to dynamically collect data, calculating the lowest common multiple of the data upload time of each Internet of Things device for cross-device collaboration, receiving data through an edge server and performing deduplication and time series optimization on the data; performing event identification and classification on the optimized time series data, and performing data aggregation and compression according to the classification results. The present invention dynamically collects data and synchronizes the data upload time of different Internet of Things devices, performs deduplication and time series optimization on the collected data, performs event identification and classification on the data, and then performs data aggregation and compression according to the classification results, effectively reducing network congestion and data processing delays and data upload costs, and can effectively manage and adjust priorities according to the actual importance of the data, significantly improving the performance and efficiency of data processing of Internet of Things devices.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an Internet of Things data processing method and system based on edge computing. Background Art

[0002] With the rapid development of Internet of Things (IoT) technology, more and more devices and sensors are connected to the Internet, generating a large amount of data. Edge computing, as a distributed processing technology, has emerged, aiming to transfer data processing tasks from the central cloud to the edge of the network, thereby reducing latency and improving data processing efficiency. This method allows data to be processed closer to the data source, thereby responding to changes in the external environment and updates to device status in real time. However, data uploads from IoT devices are frequent and scattered. Existing technologies cannot dynamically adjust the data collection frequency according to changes in the environment in which the devices are located. When processing data uploads from a large number of devices, it is impossible to effectively coordinate the upload time of each device, and it is impossible to optimize the cost of device data uploads, resulting in network congestion and data processing delays. Existing data processing methods cannot effectively combine event recognition and data classification, resulting in insufficient intelligence in data processing, and cannot effectively manage and adjust priorities based on the actual importance of the data. Summary of the invention

[0003] In view of the problems existing in the above-mentioned existing edge computing-based IoT data processing methods and systems, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is that the existing technology cannot dynamically adjust the data collection frequency according to changes in the environment in which the device is located, cannot effectively coordinate the upload time of each device when processing data uploads from a large number of devices, and cannot optimize the cost of device data uploads, resulting in network congestion and data processing delays. The existing data processing method cannot effectively combine event identification and data classification, resulting in insufficient intelligence in data processing, and cannot effectively manage and adjust priorities based on the actual importance of the data.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: an Internet of Things data processing method based on edge computing, which includes: using a multimodal sensor network to dynamically collect data, calculating the lowest common multiple of the data upload time of each Internet of Things device for cross-device collaboration, receiving data through an edge server and deduplicating and optimizing the time series of the data; performing event identification and classification on the optimized time series data, and aggregating and compressing the data according to the classification results; uploading the aggregated and compressed data to a cloud platform for decompression and storage.

[0006] As a preferred solution of the IoT data processing method based on edge computing described in the present invention, wherein: the dynamic data collection using a multimodal sensor network refers to establishing a connection between the sensor in the IoT device and the edge server, configuring the connection parameters of the sensor network, including the IP address, network frequency band and connection protocol, setting the initial collection frequency according to the sensor type and monitoring requirements, and using the data collected by the sensor according to the initial collection frequency to establish a baseline;

[0007] In the process of sensor data collection, each collected data is compared with the baseline data, and the difference between each collected data and the baseline data is taken as the change. If the change exceeds the threshold E t , then record the change;

[0008] Calculate the new data collection frequency based on the detected environmental changes:

[0009]

[0010] In the formula, F(t) is the acquisition frequency at time t, F0 is the initial acquisition frequency, α is the adjustment coefficient, which reflects the influence of environmental changes on the acquisition frequency, ΔE(t) is the change, and E t is the environmental change threshold;

[0011] Sensor data is collected according to the adjusted parameters, and the new data collection situation is monitored in real time. If the environmental changes continue, the dynamic parameter adjustment process is repeated, and the timestamps, sensor identifiers and measurement values ​​of the collected data are integrated to form an IoT device data set.

[0012] As a preferred solution of the IoT data processing method based on edge computing described in the present invention, wherein: the calculation of the lowest common multiple of the data upload time of each IoT device for cross-device collaboration refers to collecting the time interval for uploading data from each IoT device, and the time interval for uploading data by each IoT device is expressed as:

[0013] [T1,T2,…T w ],

[0014] The formula for calculating the least common multiple of time intervals is expressed as:

[0015]

[0016] In the formula, a and b are the time intervals between two IoT devices uploading data, and GCD is the greatest common factor;

[0017] Set the lowest common multiple of the initial time interval to the upload time interval of the first IoT device LCM = T1;

[0018] For each IoT device Tw , calculate the current LCM and T in turn w LCM:

[0019]

[0020] Set the upload time intervals of all devices to T1, T2, ...T q Sort from small to large, and use a dynamic programming algorithm to divide devices with similar time intervals into multiple preliminary groups;

[0021] For each preliminary combination, the least common multiple time interval within the combination is calculated using the least common multiple algorithm;

[0022] Define the cost function C for each combination q :

[0023]

[0024] In the formula, E w is the energy consumption of the wth IoT device, T w is the upload time interval of the wth IoT device, n q is the number of IoT devices in the qth combination;

[0025] Traverse all combinations, calculate the overall cost and time interval, select the IoT device combination with the lowest cost, and calculate the total cost C of all combinations:

[0026]

[0027] Where M is the number of IoT device combinations;

[0028] The optimization goal is to minimize the total cost of uploading data from IoT devices:

[0029] minC,

[0030] According to the optimized combination, the time interval for uploading data of the IoT device with the lowest cost is selected as the new data upload time. The time interval for uploading data of the new IoT device is set to update the data upload time setting of each IoT device through the network management system, and the collected IoT device data set is uploaded to the edge server according to the set upload time.

[0031] As a preferred solution of the IoT data processing method based on edge computing of the present invention, wherein: receiving data through an edge server and performing deduplication and time series optimization on the data refers to using an edge server to receive data transmitted from various IoT devices in real time;

[0032] Apply a hash function to each data point received to generate a unique hash value, and compare the generated hash value with the hash table stored in the edge server;

[0033] If the hash value of the data already exists, it means that the data is duplicated and is discarded;

[0034] If the hash value of the data does not exist, keep the data and add the hash value to the hash table;

[0035] Sort all non-repeated data according to the timestamp of the data to generate time series data, check for missing values ​​in the time series, and use linear interpolation method to fill in the missing points;

[0036] Compute the time intervals between adjacent data points in a time series and detect anomalous time intervals:

[0037] ΔT l =T′[i+1]-T′[i],

[0038] Where, ΔT l is the lth time interval, T'[i+1] is the i+1th timestamp after sorting, and T'[i] is the ith timestamp after sorting;

[0039] The abnormal time interval is expressed as:

[0040] |ΔT l -μ|>2σ,

[0041] In the formula, μ is the mean of the time interval, σ is the standard deviation of the time interval, and the corresponding data is determined as the abnormal data value through the abnormal time interval;

[0042] Abnormal data values ​​are eliminated based on the detection results, the time series is smoothed using sliding window technology, and the processed data is centrally stored in the database of the edge server.

[0043] As a preferred solution of the IoT data processing method based on edge computing described in the present invention, wherein: the event identification and classification of the optimized time series data includes:

[0044] Use convolutional neural networks to build event recognition models, including input layers, convolution layers, pooling layers, and output layers;

[0045] Use historical IoT device data to input the event recognition model for iterative training, define the loss function and Adam optimizer to iteratively optimize the model parameters. When the loss of the event recognition model no longer decreases significantly during continuous iterations, stop iterating and output the model parameters to update the event recognition model.

[0046] Input the real-time IoT device data set in the database into the event recognition model to obtain the event label corresponding to each IoT device data;

[0047] Integrate event tags with IoT device performance logs, merge event tags and IoT device performance data, and use the Pearson correlation coefficient to analyze the correlation between event tags and IoT device performance indicators;

[0048] According to the impact of historical events in the event processing log, the urgency and priority of different events are evaluated, and the urgency level is set for each event based on the analysis results of data update frequency and historical impact.

[0049] As a preferred solution of the IoT data processing method based on edge computing described in the present invention, wherein: the data aggregation and compression according to the classification results refers to classifying and sorting the IoT device data according to the emergency level output by the event classification model, marking each data point according to the emergency level of the associated event, performing real-time data aggregation processing for high-emergency events, and setting an aggregation cycle of 15 to 30 minutes for medium-emergency events and low-emergency events;

[0050] Use low compression for data with high urgency, and high compression for data with medium and low urgency.

[0051] As a preferred solution of the IoT data processing method based on edge computing described in the present invention, the uploading of the aggregated and compressed data to the cloud platform for decompression and storage refers to uploading the aggregated and compressed data to the cloud platform through the edge server, and after the cloud platform receives the data, it decompresses the data and stores it in the cloud database, implements security protection measures and backups for the stored data, regularly performs integrity checks on the stored data and backup data, and stores the test results synchronously.

[0052] Another object of the present invention is to provide an Internet of Things data processing system based on edge computing, which comprises:

[0053] The data collection module is used to configure the connection parameters between the IoT device and the edge server, set the initial collection frequency and data resolution, and dynamically adjust the data collection frequency and data resolution based on the baseline data;

[0054] The synchronization module is used to collect the data upload time intervals from each IoT device and calculate the lowest common multiple to achieve synchronous data upload between IoT devices;

[0055] The data deduplication module is used to receive data and perform data deduplication through a hash function, and sort the non-duplicate data in time series;

[0056] The event recognition module is used to use convolutional neural networks to identify and classify events after time series optimization, and set emergency levels for classified events;

[0057] A data aggregation module, which is used to classify, sort, label, aggregate and compress IoT device data according to the emergency level of the event;

[0058] The data storage module is used to upload the aggregated and compressed data to the cloud platform for decompression, storage and backup.

[0059] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of an Internet of Things data processing method based on edge computing when executing the computer program.

[0060] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of an Internet of Things data processing method based on edge computing.

[0061] The beneficial effects of the present invention are as follows: the present invention dynamically collects data and synchronizes the data upload time of different IoT devices, deduplicates and optimizes the time series of the collected data, identifies and classifies events on the data, and then aggregates and compresses the data according to the classification results, thereby effectively reducing network congestion and data processing delays. Effective management and priority adjustment can be performed based on the actual importance of the data, significantly improving the performance and efficiency of data processing of IoT devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0063] Figure 1 The figure is a flowchart of the IoT data processing method based on edge computing.

[0064] Figure 2 This is a structural diagram of the Internet of Things data processing system based on edge computing. DETAILED DESCRIPTION

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0066] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0067] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0068] Example 1

[0069] Reference Figure 1 , which is the first embodiment of the present invention, and this embodiment provides an Internet of Things data processing method based on edge computing. The Internet of Things data processing method based on edge computing includes:

[0070] S1. Use a multimodal sensor network to dynamically collect data, calculate the lowest common multiple of the data upload time of each IoT device for cross-device collaboration, receive data through the edge server, and perform deduplication and time series optimization on the data;

[0071] Specifically, using a multimodal sensor network to dynamically collect data means establishing a connection between the sensor in the IoT device and the edge server, configuring the connection parameters of the sensor network, including the IP address, network frequency band, and connection protocol, setting the initial collection frequency according to the sensor type and monitoring requirements, and using the data collected by the sensor at the initial collection frequency to establish a baseline;

[0072] In the process of sensor data collection, each collected data is compared with the baseline data, and the difference between each collected data and the baseline data is taken as the change. If the change exceeds the threshold E t , then record the change;

[0073] Calculate the new data collection frequency based on the detected environmental changes:

[0074]

[0075] In the formula, F(t) is the acquisition frequency at time t, F0 is the initial acquisition frequency, α is the adjustment coefficient, which reflects the influence of environmental changes on the acquisition frequency, ΔE(t) is the change, and E t is the environmental change threshold;

[0076] Sensor data is collected according to the adjusted parameters, and the new data collection situation is monitored in real time. If the environmental changes continue, the dynamic parameter adjustment process is repeated, and the timestamps, sensor identifiers and measurement values ​​of the collected data are integrated to form an IoT device data set.

[0077] The environmental change threshold is set according to the actual application scenario and the experience of professionals. It establishes a connection with the edge server through a multimodal sensor network. According to the set initial collection frequency and monitoring requirements, it automatically adjusts network parameters such as IP address and connection protocol to achieve real-time data collection, ensure the flexibility and accuracy of data collection, enable the system to quickly adapt to environmental changes, improve the real-time and reliability of data collection, and reduce the risk of data loss due to improper configuration. By comparing the difference between the collected data and the baseline data in real time, if the change exceeds the preset threshold, the system automatically calculates the new data collection frequency to adapt to environmental changes, making data collection more refined and targeted, optimizing resource allocation, reducing the storage of invalid data, improving data processing efficiency and reducing energy consumption.

[0078] Further, calculating the least common multiple of the data upload time of each IoT device for cross-device collaboration refers to collecting the time interval for uploading data from each IoT device, and the time interval for uploading data by each IoT device is expressed as:

[0079] [T1,T2,…T w ],

[0080] The formula for calculating the least common multiple of time intervals is expressed as:

[0081]

[0082] In the formula, a and b are the time intervals between two IoT devices uploading data, and GCD is the greatest common factor;

[0083] Set the lowest common multiple of the initial time interval to the upload time interval of the first IoT device LCM = T1;

[0084] For each IoT device T w , calculate the current LCM and T in turn w LCM:

[0085]

[0086] Set the upload time intervals of all devices to T1, T2, ...T q Sort from small to large, and use a dynamic programming algorithm to divide devices with similar time intervals into multiple preliminary groups;

[0087] For each preliminary combination, the least common multiple time interval within the combination is calculated using the least common multiple algorithm;

[0088] Define the cost function C for each combination q :

[0089]

[0090] In the formula, E w is the energy consumption of the wth IoT device, T w is the upload time interval of the wth IoT device, n q is the number of IoT devices in the qth combination;

[0091] Traverse all combinations, calculate the overall cost and time interval, select the IoT device combination with the lowest cost, and calculate the total cost C of all combinations:

[0092]

[0093] Where M is the number of IoT device combinations;

[0094] The optimization goal is to minimize the total cost of uploading data from IoT devices:

[0095] minC,

[0096] According to the optimized combination, the time interval for uploading data of the IoT device with the lowest cost is selected as the new data upload time. The time interval for uploading data of the new IoT device is set to update the data upload time setting of each IoT device through the network management system, and the collected IoT device data set is uploaded to the edge server according to the set upload time.

[0097] The greatest common divisor is the largest positive integer factor shared by two or more integers. By determining the optimal time point for each device to upload data, it significantly reduces conflicts and network congestion during data upload, improves network communication efficiency, reduces processing time and computing resource consumption through intelligent grouping, and ensures that the data upload time of each device in the group is optimized, further enhancing the overall performance of the system. Through precise cost evaluation, the economy and efficiency of the entire system are ensured, the energy consumption and upload efficiency of the equipment are optimized, and the system operating costs are reduced. The introduction of the cost function makes equipment operation more economical, providing an efficient and economical data management strategy for IoT devices.

[0098] Furthermore, receiving data through an edge server and performing deduplication and time series optimization on the data refers to using an edge server to receive data transmitted from various IoT devices in real time;

[0099] Apply a hash function to each data point received to generate a unique hash value, and compare the generated hash value with the hash table stored in the edge server;

[0100] If the hash value of the data already exists, it means that the data is duplicated and is discarded;

[0101] If the hash value of the data does not exist, keep the data and add the hash value to the hash table;

[0102] Generate time series data by sorting all non-repeated data according to their timestamps, check for missing values ​​in the time series, and use linear interpolation to fill in the missing points:

[0103]

[0104] In the formula, y is the missing data value, t1 and t2 are timestamps, y1 and y2 are the data values ​​at the timestamps, and t is the data point corresponding to the missing data value;

[0105] Compute the time intervals between adjacent data points in a time series and detect anomalous time intervals:

[0106] ΔT l =T′[i+1]-T′[i],

[0107] Where, ΔT l is the lth time interval, T'[i+1] is the i+1th timestamp after sorting, and T'[i] is the ith timestamp after sorting;

[0108] The abnormal time interval is expressed as:

[0109] |ΔT l -μ|>2σ,

[0110] In the formula, μ is the mean of the time interval, σ is the standard deviation of the time interval, and the corresponding data is determined as the abnormal data value through the abnormal time interval;

[0111] Abnormal data values ​​are eliminated based on the detection results, and the sliding window technology is used to smooth the time series to reduce the noise in the time series. The processed data is stored in the database of the edge server.

[0112] By comparing existing hash tables, the system can instantly identify and discard duplicate data, retaining only unique items, significantly reducing the waste of resources in storing and processing invalid and redundant data, improving the speed of data processing and the overall performance of the system, and sorting all non-duplicate data according to timestamps to form a time series. The system automatically checks missing values ​​in the sequence and fills them with linear interpolation methods. By calculating the time intervals between adjacent data points, the system can identify and eliminate data with abnormal time intervals. Data smoothing improves data resolution and availability, providing high-quality input for complex data analysis and machine learning. Centralized storage optimizes data management, speeds up data access, and reduces the risk of data corruption and loss.

[0113] S2. Perform event recognition and classification on the optimized time series data, and perform data aggregation and compression based on the classification results;

[0114] Specifically, event recognition and classification of optimized time series data include:

[0115] Use convolutional neural networks to build event recognition models, including input layers, convolution layers, pooling layers, and output layers;

[0116] Use historical IoT device data to input the event recognition model for iterative training, define the loss function and Adam optimizer to iteratively optimize the model parameters. When the loss of the event recognition model no longer decreases significantly during continuous iterations, stop iterating and output the model parameters to update the event recognition model.

[0117] Input the real-time IoT device data set in the database into the event recognition model to obtain the event label corresponding to each IoT device data;

[0118] Integrate event tags with IoT device performance logs, merge event tags and IoT device performance data, and use the Pearson correlation coefficient to analyze the correlation between event tags and IoT device performance indicators:

[0119]

[0120] Where r is the correlation between the event label and the IoT device performance indicator, X j and Y j is the j-th observation value of the event label and IoT device performance indicators X and Y, and is the average of event labels and IoT device performance indicators X and Y;

[0121] Evaluate the urgency and priority of different events based on the impact of historical events in the event processing log, and set an emergency level for each event based on the analysis results of data update frequency and historical impact;

[0122] Create an emergency level framework to categorize incidents into high, medium, and low emergency levels;

[0123] For high-emergency events, the alarm system is automatically triggered and real-time notification is sent to the system administrator via the network. For medium- and low-emergency events, they are reviewed and processed during routine maintenance.

[0124] By leveraging the powerful feature extraction capabilities of CNN, complex event patterns are automatically identified, the accuracy and efficiency of event identification are improved, and the reliance on manually set features is reduced. Iterative training ensures that the model can adapt to changing data features. The Adam optimizer accelerates the training process and improves training stability, making the model more robust and reliable. Real-time processing allows the system to respond quickly to important events. Urgency assessment helps prioritize events with the greatest impact and ensures effective allocation of resources. In addition, correlation analysis reveals the relationship between events and equipment performance, providing data support for subsequent maintenance and optimization.

[0125] Furthermore, data aggregation and compression according to the classification results refers to classifying and sorting the IoT device data according to the emergency level output by the event classification model, marking each data point according to the emergency level of the associated event, performing real-time data aggregation processing for high-emergency events, and setting an aggregation cycle of 15 to 30 minutes for medium-emergency events and low-emergency events;

[0126] Use low compression for data with high urgency, and high compression for data with medium and low urgency.

[0127] Based on the output from the event classification model, classifying and sorting IoT device data by urgency level allows the system to prioritize the most urgent events, ensuring that critical situations receive an immediate response. By distinguishing data of different urgency levels, the system can manage resources more effectively. Real-time processing of high-urgency data can significantly reduce response time to critical operations and increase system responsiveness and efficiency. By setting regular aggregation cycles for events of lower urgency levels, processing capacity can be optimized, system overload can be prevented, and stable system operation can be ensured.

[0128] S3, upload the aggregated and compressed data to the cloud platform for decompression and storage;

[0129] Specifically, uploading the aggregated and compressed data to the cloud platform for decompression and storage means uploading the aggregated and compressed data to the cloud platform through the edge server, and after receiving the data, the cloud platform decompresses the data and stores it in the cloud database, implements security protection measures and backups for the stored data, regularly performs integrity checks on the stored data and backup data, and synchronously stores the test results.

[0130] The data generated by IoT devices is initially aggregated and compressed on the edge server to reduce the amount of data transmitted to the cloud platform and improve transmission efficiency. This not only reduces network congestion and bandwidth requirements, but also optimizes the data processing process and reduces the load on the cloud server, thereby reducing overall operating costs. Uploading compressed data reduces transmission time and risk, while decompression in the cloud utilizes the powerful computing power of the cloud platform to improve data processing efficiency and response speed. The data stored in the cloud database is not only subject to security protection measures such as encryption and access control, but also regularly undergoes integrity checks to ensure that the data has not been illegally modified or damaged.

[0131] Example 2

[0132] Reference Figure 2 , which is the second embodiment of the present invention, which is different from the previous embodiment and provides an Internet of Things data processing system based on edge computing, comprising:

[0133] The data collection module is used to configure the connection parameters between the IoT device and the edge server, set the initial collection frequency and data resolution, and dynamically adjust the data collection frequency and data resolution based on the baseline data;

[0134] The synchronization module is used to collect the data upload time intervals from each IoT device and calculate the lowest common multiple to achieve synchronous data upload between IoT devices;

[0135] The data deduplication module is used to receive data and perform data deduplication through a hash function, and sort the non-duplicate data in time series;

[0136] The event recognition module is used to use convolutional neural networks to identify and classify events after time series optimization, and set emergency levels for classified events;

[0137] A data aggregation module, which is used to classify, sort, label, aggregate and compress IoT device data according to the emergency level of the event;

[0138] The data storage module is used to upload the aggregated and compressed data to the cloud platform for decompression, storage and backup.

[0139] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0140] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0141] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0142] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A method for processing Internet of Things data based on edge computing, characterized in that: include, Use multimodal sensor networks to dynamically collect data, calculate the lowest common multiple of data upload time of each IoT device for cross-device collaboration, receive data through edge servers, deduplicate data, and optimize time series; Identify and classify events on the optimized time series data, and aggregate and compress data based on the classification results; Upload the aggregated and compressed data to the cloud platform for decompression and storage; The dynamic data collection using a multimodal sensor network refers to establishing a connection between the sensor in the IoT device and the edge server, configuring the connection parameters of the sensor network, including the IP address, network frequency band, and connection protocol, setting the initial collection frequency according to the sensor type and monitoring requirements, and using the data collected by the sensor according to the initial collection frequency to establish a baseline; In the process of sensor data collection, each collected data is compared with the baseline data, and the difference between each collected data and the baseline data is taken as the change. If the change exceeds the threshold , then record the change; Calculate the new data collection frequency based on the detected environmental changes: ， Where F(t) is the acquisition frequency at time t, is the initial acquisition frequency, is the adjustment coefficient, which reflects the influence of environmental changes on the acquisition frequency. ΔE(t) is the amount of environmental change. is the environmental change threshold; Collect sensor data according to the adjusted parameters, monitor the new data collection in real time, and repeat the dynamic parameter adjustment process if the environmental changes continue. Integrate the timestamp, sensor identification and measurement value of the collected data to form an IoT device data set; The calculation of the lowest common multiple of the data upload time of each IoT device for cross-device collaboration refers to collecting the time interval for uploading data from each IoT device. The time interval for uploading data by each IoT device is expressed as: ， The formula for calculating the least common multiple of time intervals is expressed as: ， In the formula, a and b are the time intervals between two IoT devices uploading data, and GCD is the greatest common factor; Set the lowest common multiple of the initial time interval as the upload time interval of the first IoT device ; For each IoT device , calculate the current LCM and LCM: ， Set the upload interval for all devices Sort from small to large, and use a dynamic programming algorithm to divide devices with similar time intervals into multiple preliminary groups; For each preliminary combination, the least common multiple time interval within the combination is calculated using the least common multiple algorithm; Define the cost function for each combination : ， In the formula, is the energy consumption of the wth IoT device, is the upload time interval of the wth IoT device, is the number of IoT devices in the qth combination; Traverse all combinations, calculate the overall cost and time interval, select the IoT device combination with the lowest cost, and calculate the total cost C of all combinations: ， Where M is the number of IoT device combinations; The optimization goal is to minimize the total cost of uploading data from IoT devices: ， According to the optimized combination, the time interval for uploading data of the IoT device with the lowest cost is selected as the new data upload time. The time interval for uploading data of the new IoT device is set to update the data upload time setting of each IoT device through the network management system, and the collected IoT device data set is uploaded to the edge server according to the set upload time.

2. The method for processing IoT data based on edge computing according to claim 1, characterized in that: Receiving data through the edge server and performing deduplication and time series optimization on the data means using the edge server to receive data transmitted from various IoT devices in real time; Apply a hash function to each data point received to generate a unique hash value, and compare the generated hash value with the hash table stored in the edge server; If the hash value of the data already exists, it means that the data is duplicated and is discarded; If the hash value of the data does not exist, keep the data and add the hash value to the hash table; Sort all non-repeated data according to the timestamp of the data to generate time series data, check for missing values ​​in the time series, and use linear interpolation method to fill in the missing points; Compute the time intervals between adjacent data points in a time series and detect anomalous time intervals: ， In the formula, is the lth time interval, is the i+1th timestamp after sorting, is the i-th timestamp after sorting; The abnormal time interval is expressed as: ， In the formula, is the time interval mean, is the standard deviation of the time interval, and the corresponding data is determined as the abnormal data value through the abnormal time interval; Abnormal data values ​​are eliminated based on the detection results, the time series is smoothed using sliding window technology, and the processed data is centrally stored in the database of the edge server.

3. The method for processing IoT data based on edge computing according to claim 2, characterized in that: The event identification and classification of the optimized time series data includes: Use convolutional neural networks to build event recognition models, including input layers, convolution layers, pooling layers, and output layers; Use historical IoT device data to input the event recognition model for iterative training, define the loss function and Adam optimizer to iteratively optimize the model parameters. When the loss of the event recognition model no longer decreases significantly during continuous iterations, stop iterating and output the model parameters to update the event recognition model. Input the real-time IoT device data set in the database into the event recognition model to obtain the event label corresponding to each IoT device data; Integrate event tags with IoT device performance logs, merge event tag data and IoT device performance data, and use the Pearson correlation coefficient to analyze the correlation between event tags and IoT device performance indicators; According to the impact of historical events in the event processing log, the urgency and priority of different events are evaluated, and the urgency level is set for each event based on the analysis results of data update frequency and historical impact.

4. The method for processing Internet of Things data based on edge computing according to claim 3, characterized in that: The data aggregation and compression according to the classification results refers to classifying and sorting the IoT device data according to the emergency level output by the event classification model, marking each data point according to the emergency level of the associated event, performing real-time data aggregation processing for high-emergency events, and setting an aggregation cycle of 15 to 30 minutes for medium-emergency events and low-emergency events; Use low compression for data with high urgency, and high compression for data with medium and low urgency.

5. The method for processing Internet of Things data based on edge computing according to claim 4, characterized in that: The uploading of the aggregated and compressed data to the cloud platform for decompression and storage refers to uploading the aggregated and compressed data to the cloud platform through the edge server. After receiving the data, the cloud platform decompresses the data and stores it in the cloud database, implements security protection measures and backups on the stored data, regularly performs integrity checks on the stored data and backup data, and synchronously stores the test results.

6. An edge computing-based Internet of Things data processing system based on the edge computing-based Internet of Things data processing method according to any one of claims 1 to 5, characterized in that: include, The data collection module is used to configure the connection parameters between the IoT device and the edge server, set the initial collection frequency and data resolution, and dynamically adjust the data collection frequency and data resolution based on the baseline data; The synchronization module is used to collect the data upload time intervals from each IoT device and calculate the lowest common multiple to achieve synchronous data upload between IoT devices; The data deduplication module is used to receive data and perform data deduplication through a hash function, and sort the non-duplicate data in time series; The event recognition module is used to use convolutional neural networks to identify and classify events after time series optimization, and set emergency levels for classified events; A data aggregation module, which is used to classify, sort, label, aggregate and compress IoT device data according to the emergency level of the event; The data storage module is used to upload the aggregated and compressed data to the cloud platform for decompression, storage and backup.

7. A computer device comprising: Memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the Internet of Things data processing method based on edge computing described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the Internet of Things data processing method based on edge computing described in any one of claims 1 to 5 are implemented.

Images