Internet of Things data processing method, system and device based on edge computing and medium

Abstract

The invention relates to an internet-of-things data processing method, system and device based on edge computing and a medium. The method comprises the following steps: acquiring original data acquired by an internet-of-things sensor, and acquiring resource state data and network topology information of an internet-of-things edge computing environment; preprocessing the original data to obtain preprocessed data, and performing transmission strategy analysis on the preprocessed data to obtain transmitted data; mapping the transmitted data into an edge node, and performing distributed analysis on the transmitted data to obtain a corresponding edge analysis result; performing multi-source data fusion according to the edge analysis result to obtain corresponding global data, and performing deep analysis based on the global data to obtain a corresponding intelligent analysis result; and carrying out resource scheduling analysis according to the resource state data and the intelligent analysis result to obtain edge node scheduling strategies, and carrying out comprehensive analysis on all the edge node scheduling strategies to obtain a global resource optimization strategy. According to the invention, dynamic allocation and optimal utilization of resources can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things data processing, and in particular to an Internet of Things data processing method, system, device and medium based on edge computing. Background Art

[0002] With the rapid development of IoT technology, IoT data processing methods based on edge computing have been widely used in smart cities, smart transportation, smart homes and other fields. Edge computing can effectively reduce latency, save bandwidth and improve data processing efficiency by sinking computing resources to the edge of the network. Therefore, how to efficiently process the data collected by IoT sensors to achieve real-time response and intelligent analysis has become one of the research focuses.

[0003] However, traditional IoT data processing methods have limitations to a certain extent. Traditional methods usually only focus on a single dimension of data, such as preprocessing of sensor data or simple edge computing, while ignoring the impact of network topology on data transmission strategies. Secondly, when performing data analysis, the resource status in the edge computing environment is not fully considered, which may lead to uneven resource allocation and low processing efficiency. In addition, multi-source data fusion and deep analysis are often carried out separately, failing to form a coordinated whole, affecting the accuracy and real-time performance of global intelligent analysis results. Summary of the invention

[0004] The main purpose of the present invention is to provide an Internet of Things data processing method, system, device and medium based on edge computing, which can realize dynamic allocation and optimal utilization of resources and improve overall operating efficiency.

[0005] To achieve the above object, the present invention provides an IoT data processing method based on edge computing, comprising: Obtain the raw data collected by IoT sensors, and obtain resource status data and network topology information of the IoT edge computing environment; Preprocessing the original data to obtain preprocessed data, and performing transmission strategy analysis on the preprocessed data based on the network topology information to obtain transmitted data; Mapping the transmitted data to the edge node, and performing distributed analysis on the transmitted data according to the resource status data to obtain corresponding edge analysis results; Perform multi-source data fusion according to the edge analysis results to obtain corresponding global data, and perform in-depth analysis based on the global data to obtain corresponding intelligent analysis results; A resource scheduling analysis is performed based on the resource status data and the intelligent analysis results to obtain an edge node scheduling strategy, and a comprehensive analysis is performed on all the edge node scheduling strategies to obtain a global resource optimization strategy.

[0006] Furthermore, the obtaining of raw data collected by IoT sensors and the obtaining of resource status data and network topology information of the IoT edge computing environment include: Adjusting the data collection frequency of the Internet of Things sensor to obtain an optimized collection frequency, and collecting data from the Internet of Things sensor according to the optimized collection frequency to obtain the original data; Monitor the resource utilization of computing nodes in the IoT edge computing environment to obtain the CPU utilization, memory utilization, and storage space utilization of each node, and calculate the status data based on the CPU utilization, memory utilization, and storage space utilization of each node; Performing link quality detection on the network connection in the IoT edge computing environment to obtain the bandwidth, delay and packet loss rate of each link, and constructing a network topology map according to the bandwidth, delay and packet loss rate of each link to obtain network topology data; Time synchronization is performed on the status data and the network topology data according to the original data to obtain the resource status data and the network topology information that are consistent with the time of the original data.

[0007] Further, the preprocessing of the original data to obtain preprocessed data, and performing transmission strategy analysis on the preprocessed data based on the network topology information to obtain transmitted data, include: Performing data cleaning on the original data to remove outliers and redundant data to obtain cleaned data; The principal component analysis method is used to perform dimension reduction and feature extraction on the cleaned data to obtain feature data; Performing data segmentation on the characteristic data based on the network topology information to obtain segmented data, and performing routing planning on the segmented data according to the network topology information to obtain routing data; Compress and encode the routing data to obtain compressed data, allocate network resources to the compressed data based on the network topology information and determine the transmission order to obtain scheduling data; The transmission quality of the scheduled data is evaluated to obtain a quality evaluation result, and the scheduled data is adaptively adjusted according to the quality evaluation result to obtain the transmitted data.

[0008] Further, mapping the transmitted data to the edge node, and performing distributed analysis on the transmitted data according to the resource status data to obtain corresponding edge analysis results, includes: Performing data structure analysis on the transmitted data to obtain data feature vectors, and performing cluster analysis on the transmitted data according to the data feature vectors to obtain data clusters; Calculate the node matching degree of the data cluster to obtain the edge node matching degree matrix; According to the edge node matching matrix, nodes are allocated to the data cluster to obtain the data node mapping relationship; The transmitted data is distributed to the corresponding edge nodes according to the data node mapping relationship, local features are extracted from the data in each edge node to obtain a local feature set, and distributed iterative calculation is performed on the local feature set to obtain a global feature set; Perform multi-dimensional prediction analysis and anomaly detection on the global feature set to obtain edge analysis results.

[0009] Furthermore, the multi-source data fusion is performed according to the edge analysis result to obtain corresponding global data, and the in-depth analysis is performed based on the global data to obtain corresponding intelligent analysis results, including: Performing data type identification on the edge analysis result to obtain a data type identifier; Performing time series alignment on the edge analysis results according to the data type identifier to obtain aligned time series data; Perform multi-dimensional feature extraction on time series data to obtain multi-dimensional feature representation, and perform heterogeneous fusion on the multi-dimensional feature representation to obtain fusion features; Performing anomaly detection on the fused features to obtain an abnormal data identifier, and performing causal reasoning analysis on the abnormal data identifier to obtain a causal relationship graph; A strategic analysis is performed on the cause-effect relationship diagram according to the decision rules to obtain a preliminary decision strategy, the preliminary decision strategy is optimized to obtain a target decision strategy, and an in-depth analysis is performed on the target decision strategy to obtain the intelligent analysis result.

[0010] Furthermore, the resource scheduling analysis is performed based on the resource status data and the intelligent analysis results to obtain the edge node scheduling strategy, and all the edge node scheduling strategies are comprehensively analyzed to obtain the global resource optimization strategy, including: Performing a quantitative evaluation on the edge node according to the resource status data to obtain an edge node resource score; Analyze the complexity and priority of various data processing tasks of the edge node according to the intelligent analysis results to obtain a task requirement score; Conduct a preliminary comprehensive analysis of the edge node resource scores and task requirement scores to obtain the initial node scheduling plan, conduct a load balancing analysis on the initial node scheduling plan, and obtain the load balancing coefficient; Optimizing and adjusting the initial node scheduling scheme according to the load balancing coefficient to obtain the edge node scheduling strategy; Based on the edge node scheduling strategy, an edge node collaboration graph is constructed, and graph theory analysis is performed on the edge node collaboration graph to obtain an inter-node collaboration efficiency index; Optimizing the data transmission path between the edge nodes according to the inter-node collaboration efficiency index to obtain a data transmission strategy; Integrate the edge node scheduling strategy and the data transmission strategy to obtain an initial resource plan, perform simulation testing on the initial resource plan, and obtain simulation performance indicators; The initial resource solution is iteratively optimized based on the simulation performance index to obtain an iterative resource solution, and when the iterative resource solution meets a preset performance threshold, the global resource optimization strategy is output.

[0011] The present invention also provides an Internet of Things data processing system based on edge computing, which is applied to any one of the above-mentioned Internet of Things data processing methods based on edge computing, including: The acquisition module is used to obtain the raw data collected by the IoT sensors, and obtain the resource status data and network topology information of the IoT edge computing environment; An analysis module, the analysis module is used to pre-process the original data to obtain pre-processed data, and perform a transmission strategy analysis on the pre-processed data based on the network topology information to obtain transmitted data; An association module, the association module is used to map the transmitted data to an edge node, and perform distributed analysis on the transmitted data according to the resource status data to obtain a corresponding edge analysis result; A processing module, the processing module is used to perform multi-source data fusion according to the edge analysis result to obtain corresponding global data, and perform in-depth analysis based on the global data to obtain corresponding intelligent analysis results; A control module is used to perform resource scheduling analysis based on the resource status data and the intelligent analysis results to obtain an edge node scheduling strategy, and to perform a comprehensive analysis on all the edge node scheduling strategies to obtain a global resource optimization strategy.

[0012] The present invention also provides an Internet of Things data processing device based on edge computing, comprising: Memory, used to store programs; The processor is used to execute the program to implement each step of the Internet of Things data processing method based on edge computing described in any one of the above.

[0013] The present invention also provides a storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute any of the above methods.

[0014] The present invention provides the following beneficial effects: By preprocessing the raw data collected by IoT sensors and analyzing the transmission strategy in combination with network topology information, the data transmission path can be managed more effectively, the data transmission delay can be reduced, and the efficiency and reliability of data transmission can be improved. By mapping the transmitted data to the edge node and performing distributed analysis based on the resource status data, the edge computing resources can be fully utilized, the real-time and accuracy of data processing can be improved, and the efficient management of edge nodes can be achieved. By fusing multi-source data on the edge analysis results to obtain global data and conducting in-depth analysis based on this, the comprehensive utilization of multi-source data can be achieved, and the accuracy and reliability of intelligent analysis results can be improved. At the same time, resource scheduling analysis based on resource status data and intelligent analysis results, and the scheduling strategy of edge nodes can be formulated, which is conducive to the dynamic allocation and optimal utilization of resources and improves the overall operation efficiency of the system. By comprehensively analyzing the scheduling strategies of all edge nodes and formulating a global resource optimization strategy, the global optimization management of system resources can be achieved to ensure that the system can operate efficiently under different environments and load conditions. This not only improves the resource utilization of the system, but also reduces unnecessary resource consumption and improves the overall performance and response speed of the system. In summary, the present invention solves the limitations of existing IoT data processing methods by comprehensively considering network topology information, resource status data and intelligent analysis results, improves the efficiency of data transmission and processing, optimizes resource scheduling and system operation, can better meet diverse application needs, and enhances the overall performance and sustainable development capabilities of the IoT system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of an Internet of Things data processing method based on edge computing provided by the present invention; Figure 2 This is a structural diagram of an Internet of Things data processing system based on edge computing provided by the present invention; Figure 3 This is a structural diagram of an Internet of Things data processing device based on edge computing provided by the present invention.

[0016] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0019] Reference Figure 1 As shown, the present invention provides an Internet of Things data processing method based on edge computing, comprising: Step S1: Obtain the original data collected by the IoT sensor, and obtain the resource status data and network topology information of the IoT edge computing environment; Step S2: preprocessing the original data to obtain preprocessed data, and performing transmission strategy analysis on the preprocessed data based on network topology information to obtain transmitted data; Step S3: Map the transmitted data to the edge node, and perform distributed analysis on the transmitted data according to the resource status data to obtain the corresponding edge analysis results; Step S4: perform multi-source data fusion according to the edge analysis results to obtain corresponding global data, perform in-depth analysis based on the global data, and obtain corresponding intelligent analysis results; Step S5: Perform resource scheduling analysis based on resource status data and intelligent analysis results to obtain edge node scheduling strategies, and perform comprehensive analysis on all edge node scheduling strategies to obtain a global resource optimization strategy.

[0020] Based on the above steps, the detailed process is as follows: Step S1: In the IoT environment, sensors are usually deployed at different physical locations. These sensors are responsible for collecting various types of data, such as temperature, humidity, pressure, light intensity, etc. Sensors collect data periodically or in real time and transmit these raw data to edge devices through wireless or wired networks.

[0021] The original data can be transmitted through a variety of communication protocols, such as Wi-Fi, Zigbee, LoRa, NB-IoT, etc., and the appropriate communication method can be selected according to the sensor and application scenario.

[0022] Resource status data in edge computing environments include CPU utilization, memory usage, storage space, network bandwidth, etc. These data can be collected in real time through the resource monitoring module of edge devices. Edge devices will summarize the monitored resource status data and report them to the central management platform regularly or in real time for global resource scheduling and optimization.

[0023] Through network topology discovery protocols (such as LLDP, CDP, etc.), edge devices automatically discover and identify adjacent devices and their connection relationships. Based on the information discovered by topology, the network management system or SDN controller can maintain a complete network topology map and record the connection relationship and link status between devices.

[0024] Step S2: Clean the raw data to remove noise data and invalid data. For example, improve data quality by filtering out outliers and missing values.

[0025] Convert raw data into a unified format and unit for subsequent processing and analysis. For example, convert data collected by different sensors into the same timestamp format and unit.

[0026] Compress a large amount of raw data to reduce storage and transmission costs. For example, a lossless compression algorithm is used to compress the data.

[0027] Using the acquired network topology information, we analyze the connection relationship and bandwidth between edge devices and select the optimal transmission path. We set different data transmission priorities based on the importance and real-time requirements of the data. For example, important real-time data is transmitted first, while less important historical data can be transmitted later.

[0028] Based on the resource status data of the edge device, load balancing analysis is performed to reasonably distribute the data transmission load to different edge nodes to avoid single-point overload.

[0029] Step S3: Develop a data allocation strategy based on the characteristics of the transmitted data and the resource status data of the edge nodes. Ensure that the data is evenly distributed on each edge node to fully utilize computing resources. Store the transmitted data in the corresponding edge node. Edge nodes usually have local storage capabilities that can temporarily save data for rapid processing and analysis.

[0030] Decompose data analysis tasks into multiple subtasks and assign them to different edge nodes for parallel processing. For example, a large-scale data analysis task can be divided into multiple small tasks, each of which processes a portion of the data.

[0031] According to the resource status data of the edge nodes (such as CPU utilization, memory usage, etc.), the task allocation and execution order are dynamically adjusted to ensure efficient use of edge computing resources.

[0032] Data processing and analysis: Edge nodes use local computing resources to process and analyze the assigned data. For example, they perform operations such as data aggregation, statistical analysis, and pattern recognition to generate preliminary analysis results.

[0033] Summarize the analysis results generated by each edge node to obtain the overall edge analysis results. Verify the edge analysis results to ensure the accuracy and reliability of data analysis. For example, verify the validity of the analysis results through cross-validation, multiple sampling and other methods.

[0034] Step S4: Integrate the analysis results from different edge nodes to form a complete global data set. The integration process needs to consider issues such as data format consistency and timestamp alignment. Use data fusion techniques such as weighted averaging, Bayesian reasoning, and Kalman filtering to fuse data from different sources to improve the overall accuracy and reliability of the data.

[0035] Conduct in-depth analysis based on the integrated global data. For example, use machine learning algorithms, deep learning models, etc. to model global data and discover potential rules and patterns. Build and train intelligent analysis models, such as classification models, regression models, clustering models, etc., for prediction, classification, and decision support.

[0036] Generate intelligent analysis results, including prediction results, classification labels, anomaly detection results, etc. These results can be used in application scenarios such as decision support, risk warning, and optimization control.

[0037] Step S5: Based on the intelligent analysis results, the resource requirements of each edge node are evaluated, for example, the computing requirements, storage requirements and network requirements in the future are predicted.

[0038] Based on the resource status data of the edge nodes, evaluate the currently available computing resources, storage resources, and network resources. For example, check the CPU utilization, memory usage, bandwidth usage, etc. of each edge node. Combine resource demand and resource supply to generate a scheduling strategy for the edge node. The scheduling strategy includes task allocation, resource allocation, load balancing, etc.

[0039] The scheduling strategy is adjusted dynamically based on real-time resource status data and intelligent analysis results. For example, when resources at an edge node are tight, some tasks can be migrated to other nodes or resource supply can be increased.

[0040] Comprehensively analyze the scheduling strategies of all edge nodes to form a global resource optimization strategy. The comprehensive analysis takes into account factors such as global resource utilization, task completion time, and network latency. Apply the global resource optimization strategy to the edge computing environment to perform resource scheduling and optimization. For example, adjust the allocation and execution order of tasks, optimize the data transmission path, and improve the overall system performance.

[0041] The Internet of Things data processing method based on edge computing provided by the present invention can manage the data transmission path more effectively, reduce data transmission delay, and improve the efficiency and reliability of data transmission by preprocessing the raw data collected by the Internet of Things sensor and analyzing the transmission strategy in combination with the network topology information. By mapping the transmitted data to the edge node and performing distributed analysis based on the resource status data, the edge computing resources can be fully utilized, the real-time and accuracy of data processing can be improved, and efficient management of the edge nodes can be achieved. By fusing the edge analysis results with multi-source data, global data is obtained, and in-depth analysis is performed based on this, the comprehensive utilization of multi-source data can be achieved, and the accuracy and reliability of the intelligent analysis results can be improved. At the same time, resource scheduling analysis is performed based on the resource status data and the intelligent analysis results, and a scheduling strategy for the edge node is formulated, which helps to realize the dynamic allocation and optimal utilization of resources and improve the overall operation efficiency of the system. By comprehensively analyzing the scheduling strategies of all edge nodes and formulating a global resource optimization strategy, global optimization management of system resources can be achieved, ensuring that the system can operate efficiently under different environments and load conditions. This not only improves the resource utilization of the system, but also reduces unnecessary resource consumption and improves the overall performance and response speed of the system. In summary, the present invention solves the limitations of existing IoT data processing methods by comprehensively considering network topology information, resource status data and intelligent analysis results, improves the efficiency of data transmission and processing, optimizes resource scheduling and system operation, can better meet diverse application needs, and enhances the overall performance and sustainable development capabilities of the IoT system.

[0042] In one embodiment, raw data collected by IoT sensors is obtained, and resource status data and network topology information of an IoT edge computing environment are obtained, including: Adjust the data collection frequency of the IoT sensor and determine an optimized collection frequency based on the specific application scenario and requirements of the sensor. Collect data from the IoT sensor according to the optimized collection frequency to obtain the original data.

[0043] Monitor the resource utilization of computing nodes in the IoT edge computing environment to obtain the CPU utilization, memory utilization, and storage space utilization of each node.

[0044] The status data is calculated based on the CPU usage, memory usage, and storage space usage of each node.

[0045] Perform link quality detection on network connections in the IoT edge computing environment to obtain the bandwidth, delay, and packet loss rate of each link.

[0046] According to the bandwidth, delay and packet loss rate of each link, a network topology map is constructed to obtain network topology data.

[0047] The state data and network topology data are synchronized according to the original data, so that the timestamps of the state data and network topology data are consistent with the timestamps of the original data, and resource state data and network topology information that are consistent with the original data time are obtained.

[0048] This embodiment reduces unnecessary data collection and improves data collection efficiency by optimizing and adjusting the data collection frequency while ensuring data integrity and accuracy. The optimized collection frequency helps to reduce the burden of data transmission and reduce network bandwidth occupancy. Monitoring the resource utilization of computing nodes in the edge computing environment can provide real-time information about the CPU, memory, and storage space usage of each node, which helps to optimize resource allocation. Real-time status data calculation can promptly identify resource bottlenecks, take measures to make adjustments, and improve the overall performance of the system. Through link quality detection, the bandwidth, latency, and packet loss rate of network connections can be monitored in real time, which helps to promptly identify and resolve network problems. Building a network topology diagram can better understand the network structure and facilitate network optimization and troubleshooting. Time synchronization processing ensures the time consistency of original data, resource status data, and network topology information, which helps to accurately analyze and compare data.

[0049] In one embodiment, the original data is preprocessed to obtain preprocessed data, and a transmission strategy analysis is performed on the preprocessed data based on network topology information to obtain transmitted data, including: The original data is cleaned to remove outliers and redundant data to obtain cleaned data.

[0050] The principal component analysis (PCA) method is used to reduce the dimension and extract features of the cleaned data to obtain feature data.

[0051] The characteristic data is segmented based on the network topology information to obtain segmented data, and routing planning is performed on the segmented data according to the network topology information to obtain routing data.

[0052] The routing data is compressed and encoded to reduce the bandwidth required for data transmission and obtain compressed data. Compression encoding can use a variety of algorithms, such as Huffman encoding, LZ77 / LZ78, etc.

[0053] Based on the network topology information, network resources are allocated to the compressed data and the transmission order is determined to obtain scheduling data.

[0054] The transmission quality of the scheduling data is evaluated to obtain a quality evaluation result, and the scheduling data is adaptively adjusted according to the quality evaluation result to obtain the transmitted data.

[0055] This embodiment removes outliers and redundant data through data cleaning, improves the consistency and reliability of the data, and reduces the impact of noise on subsequent analysis. Principal component analysis (PCA) is used for dimensionality reduction and feature extraction, which reduces the data dimension, retains the most important features, and improves the efficiency of data processing. Data segmentation and route planning are performed based on network topology information to ensure the efficiency and reliability of data transmission. Data segmentation makes data more suitable for transmission in the network, while route planning selects the optimal path, reducing transmission delay and packet loss rate. Data is compressed and encoded, reducing the bandwidth required for transmission and improving transmission efficiency. Compression coding technology can significantly reduce the amount of data, thereby reducing transmission costs and time.

[0056] In one embodiment, the transmitted data is mapped to the edge node, and the transmitted data is subjected to distributed analysis according to the resource status data to obtain corresponding edge analysis results, including: The data structure of the transmitted data is analyzed to obtain a data feature vector, and the data after transmission is clustered according to the data feature vector to obtain a data cluster.

[0057] The node matching degree of the data cluster is calculated to obtain the edge node matching degree matrix.

[0058] The nodes of the data cluster are allocated according to the edge node matching matrix to obtain the data node mapping relationship.

[0059] The transmitted data is distributed to the corresponding edge nodes according to the data node mapping relationship, and local features are extracted from the data in each edge node to obtain a local feature set. The local feature set is then subjected to distributed iterative calculation to obtain a global feature set.

[0060] Perform multi-dimensional prediction analysis and anomaly detection on the global feature set to obtain edge analysis results.

[0061] This embodiment can reduce the burden of data transmission and improve data processing efficiency by performing local feature extraction on edge nodes. Distributed iterative computing can make full use of the computing resources of edge nodes and speed up data processing. By calculating the node matching degree of the data cluster, the data can be allocated to the most suitable edge node to avoid waste of resources. According to the computing power and current load of the edge node, ensure that each node can efficiently process the data assigned to it. The reliability of data processing can be ensured by allocating the transmitted data to the corresponding edge node through the data node mapping relationship. Even if an edge node fails, other nodes can still continue to process data, improving the fault tolerance of the system.

[0062] In one embodiment, multi-source data fusion is performed according to the edge analysis results to obtain corresponding global data, and in-depth analysis is performed based on the global data to obtain corresponding intelligent analysis results, including: The data type of the edge analysis results is identified to obtain a data type identifier. The identified data types include numeric, text, time series, and other types for subsequent processing.

[0063] The edge analysis results are aligned in time series according to the data type identifier to obtain aligned time series data. Multidimensional feature extraction is performed on the time series data to obtain multidimensional feature representation, including statistical features, periodic features, trend features, etc.

[0064] Heterogeneous fusion is performed on multi-dimensional feature representations to obtain fused features, anomaly detection is performed on the fused features to obtain abnormal data identification, and causal reasoning analysis is performed on the abnormal data identification to obtain a causal relationship diagram.

[0065] Conduct strategic analysis on the cause-effect relationship diagram according to the decision rules to obtain a preliminary decision strategy, optimize the preliminary decision strategy to obtain a target decision strategy, conduct in-depth analysis on the target decision strategy to obtain intelligent analysis results.

[0066] This embodiment ensures that different types of data are processed correctly by identifying the data type of the edge analysis results. By aligning the time series, all data is ensured to be on the same timeline, improving the consistency and comparability of the data. Multidimensional feature extraction can capture multiple features in the data and improve the accuracy and comprehensiveness of subsequent analysis. Heterogeneous fusion combines different types of data features to form richer fusion features, which helps to improve the depth and breadth of analysis. Anomaly detection can promptly detect anomalies in the data and improve the sensitivity and response speed of the system. Through causal reasoning analysis, the causal relationship between abnormal data is identified to provide a basis for subsequent decision-making.

[0067] In one embodiment, a resource scheduling analysis is performed based on resource status data and intelligent analysis results to obtain an edge node scheduling strategy, and a comprehensive analysis is performed on all edge node scheduling strategies to obtain a global resource optimization strategy, including: Quantitatively evaluate edge nodes based on resource status data to obtain edge node resource scores; Based on the intelligent analysis results, the complexity and priority of various data processing tasks of edge nodes are analyzed to obtain the task requirement score.

[0068] A preliminary comprehensive analysis is performed on the edge node resource scores and task requirement scores to obtain the initial node scheduling plan. A load balancing analysis is performed on the initial node scheduling plan to obtain the load balancing coefficient.

[0069] The initial node scheduling scheme is optimized and adjusted according to the load balancing coefficient to obtain the edge node scheduling strategy.

[0070] An edge node collaboration graph is constructed based on the edge node scheduling strategy, and a graph theory analysis is performed on the edge node collaboration graph to obtain the inter-node collaboration efficiency index.

[0071] The data transmission path between edge nodes is optimized according to the inter-node collaboration efficiency index to obtain the data transmission strategy.

[0072] The edge node scheduling strategy and data transmission strategy are integrated to obtain an initial resource plan, and the initial resource plan is simulated and tested to obtain simulation performance indicators.

[0073] The initial resource plan is iteratively optimized based on the simulation performance index to obtain an iterative resource plan. When the iterative resource plan meets the preset performance threshold, a global resource optimization strategy is output.

[0074] This embodiment quantitatively evaluates the edge nodes based on the resource status data to obtain the edge node resource score, thereby ensuring the objectivity and accuracy of the resource evaluation. The complexity and priority of various data processing tasks are analyzed based on the intelligent analysis results to obtain the task requirement score, thereby ensuring the rationality and priority of task allocation. A preliminary comprehensive analysis is performed on the edge node resource score and the task requirement score to obtain the initial node scheduling scheme to ensure the preliminary match between resources and tasks. A load balancing analysis is performed on the initial node scheduling scheme to obtain the load balancing coefficient to ensure that the load of each edge node is reasonably distributed to avoid overload or waste of resources. The initial node scheduling scheme is optimized and adjusted according to the load balancing coefficient to obtain a more reasonable edge node scheduling strategy to ensure the effective use of resources. An edge node collaboration graph is constructed based on the edge node scheduling strategy, and a graph theory analysis is performed to obtain an inter-node collaboration efficiency index to ensure that the collaboration between nodes is efficient and orderly. The data transmission path between edge nodes is optimized based on the inter-node collaboration efficiency index to obtain a data transmission strategy to ensure the efficiency and reliability of data transmission.

[0075] Reference Figure 2 As shown, the present invention also provides an Internet of Things data processing system based on edge computing, which is applied to any one of the above-mentioned Internet of Things data processing methods based on edge computing, including: The acquisition module is used to obtain the raw data collected by the IoT sensors and obtain the resource status data and network topology information of the IoT edge computing environment; An analysis module is used to pre-process the original data to obtain pre-processed data, and to perform transmission strategy analysis on the pre-processed data based on network topology information to obtain transmitted data; The association module is used to map the transmitted data to the edge node, and perform distributed analysis on the transmitted data according to the resource status data to obtain the corresponding edge analysis results; The processing module is used to fuse multi-source data according to the edge analysis results to obtain the corresponding global data, and to perform in-depth analysis based on the global data to obtain the corresponding intelligent analysis results; The control module is used to perform resource scheduling analysis based on resource status data and intelligent analysis results, obtain edge node scheduling strategies, conduct comprehensive analysis on all edge node scheduling strategies, and obtain global resource optimization strategies.

[0076] The Internet of Things data processing system based on edge computing provided by the present invention can manage data transmission paths more effectively, reduce data transmission delays, and improve the efficiency and reliability of data transmission by preprocessing the raw data collected by the Internet of Things sensor and analyzing the transmission strategy in combination with the network topology information. By mapping the transmitted data to the edge node and performing distributed analysis based on the resource status data, the edge computing resources can be fully utilized, the real-time and accuracy of data processing can be improved, and efficient management of the edge nodes can be achieved. By fusing the edge analysis results with multi-source data, global data is obtained, and in-depth analysis is performed based on this, the comprehensive utilization of multi-source data can be achieved, and the accuracy and reliability of the intelligent analysis results can be improved. At the same time, resource scheduling analysis is performed based on the resource status data and the intelligent analysis results, and a scheduling strategy for the edge node is formulated, which helps to realize the dynamic allocation and optimal utilization of resources and improve the overall operation efficiency of the system. By comprehensively analyzing the scheduling strategies of all edge nodes and formulating a global resource optimization strategy, global optimization management of system resources can be achieved, ensuring that the system can operate efficiently under different environments and load conditions. This not only improves the resource utilization of the system, but also reduces unnecessary resource consumption and improves the overall performance and response speed of the system. In summary, the present invention solves the limitations of existing IoT data processing methods by comprehensively considering network topology information, resource status data and intelligent analysis results, improves the efficiency of data transmission and processing, optimizes resource scheduling and system operation, can better meet diverse application needs, and enhances the overall performance and sustainable development capabilities of the IoT system.

[0077] Reference Figure 3 As shown, the present invention also provides an Internet of Things data processing device based on edge computing, comprising: Memory, used to store programs; The processor is used to execute the program to implement each step of any one of the above-mentioned Internet of Things data processing methods based on edge computing.

[0078] In this embodiment, the processor and the memory may be connected via a bus or other means. The memory may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk, or a solid-state drive. The processor may be a general-purpose processor, such as a central processing unit, a digital signal processor, an application-specific integrated circuit, or one or more integrated circuits configured to implement the embodiments of the present invention.

[0079] The present invention also provides a storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute any of the above methods.

[0080] It should be noted that technicians in the relevant technical field can clearly understand that for the convenience and conciseness of description, the specific working process of the system and each module described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0081] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for processing Internet of Things data based on edge computing, characterized in that: include: Obtain the raw data collected by IoT sensors, and obtain resource status data and network topology information of the IoT edge computing environment; Preprocessing the original data to obtain preprocessed data, and performing transmission strategy analysis on the preprocessed data based on the network topology information to obtain transmitted data; Mapping the transmitted data to the edge node, and performing distributed analysis on the transmitted data according to the resource status data to obtain corresponding edge analysis results; Perform multi-source data fusion according to the edge analysis results to obtain corresponding global data, and perform in-depth analysis based on the global data to obtain corresponding intelligent analysis results; A resource scheduling analysis is performed based on the resource status data and the intelligent analysis results to obtain an edge node scheduling strategy, and a comprehensive analysis is performed on all the edge node scheduling strategies to obtain a global resource optimization strategy.

2. The method for processing Internet of Things data based on edge computing according to claim 1, characterized in that: The method of obtaining the raw data collected by the IoT sensor and obtaining the resource status data and network topology information of the IoT edge computing environment includes: Adjusting the data collection frequency of the Internet of Things sensor to obtain an optimized collection frequency, and collecting data from the Internet of Things sensor according to the optimized collection frequency to obtain the original data; Monitor the resource utilization of computing nodes in the IoT edge computing environment to obtain the CPU utilization, memory utilization, and storage space utilization of each node, and calculate the status data based on the CPU utilization, memory utilization, and storage space utilization of each node; Performing link quality detection on the network connection in the IoT edge computing environment to obtain the bandwidth, delay and packet loss rate of each link, and constructing a network topology map according to the bandwidth, delay and packet loss rate of each link to obtain network topology data; Time synchronization is performed on the status data and the network topology data according to the original data to obtain the resource status data and the network topology information that are consistent with the time of the original data.

3. The method for processing IoT data based on edge computing according to claim 1, characterized in that: The preprocessing of the original data to obtain preprocessed data, and performing transmission strategy analysis on the preprocessed data based on the network topology information to obtain transmitted data, include: Performing data cleaning on the original data to remove outliers and redundant data to obtain cleaned data; The principal component analysis method is used to perform dimension reduction and feature extraction on the cleaned data to obtain feature data; Performing data segmentation on the characteristic data based on the network topology information to obtain segmented data, and performing routing planning on the segmented data according to the network topology information to obtain routing data; Compress and encode the routing data to obtain compressed data, allocate network resources to the compressed data based on the network topology information and determine the transmission order to obtain scheduling data; The transmission quality of the scheduled data is evaluated to obtain a quality evaluation result, and the scheduled data is adaptively adjusted according to the quality evaluation result to obtain the transmitted data.

4. The method for processing Internet of Things data based on edge computing according to claim 1, characterized in that: Mapping the transmitted data to the edge node, and performing distributed analysis on the transmitted data according to the resource status data to obtain corresponding edge analysis results, including: Performing data structure analysis on the transmitted data to obtain data feature vectors, and performing cluster analysis on the transmitted data according to the data feature vectors to obtain data clusters; Calculate the node matching degree of the data cluster to obtain the edge node matching degree matrix; According to the edge node matching matrix, nodes are allocated to the data cluster to obtain the data node mapping relationship; The transmitted data is distributed to the corresponding edge nodes according to the data node mapping relationship, local features are extracted from the data in each edge node to obtain a local feature set, and distributed iterative calculation is performed on the local feature set to obtain a global feature set; Perform multi-dimensional prediction analysis and anomaly detection on the global feature set to obtain edge analysis results.

5. The method for processing Internet of Things data based on edge computing according to claim 1, characterized in that: The multi-source data fusion is performed according to the edge analysis result to obtain corresponding global data, and the in-depth analysis is performed based on the global data to obtain corresponding intelligent analysis results, including: Performing data type identification on the edge analysis result to obtain a data type identifier; Performing time series alignment on the edge analysis results according to the data type identifier to obtain aligned time series data; Perform multi-dimensional feature extraction on time series data to obtain multi-dimensional feature representation, and perform heterogeneous fusion on the multi-dimensional feature representation to obtain fusion features; Performing anomaly detection on the fused features to obtain an abnormal data identifier, and performing causal reasoning analysis on the abnormal data identifier to obtain a causal relationship graph; A strategic analysis is performed on the cause-effect relationship diagram according to the decision rules to obtain a preliminary decision strategy, the preliminary decision strategy is optimized to obtain a target decision strategy, and an in-depth analysis is performed on the target decision strategy to obtain the intelligent analysis result.

6. The method for processing Internet of Things data based on edge computing according to claim 1, characterized in that: The resource scheduling analysis is performed based on the resource status data and the intelligent analysis result to obtain the edge node scheduling strategy, and all the edge node scheduling strategies are comprehensively analyzed to obtain the global resource optimization strategy, including: Performing a quantitative evaluation on the edge node according to the resource status data to obtain an edge node resource score; Analyze the complexity and priority of various data processing tasks of the edge node according to the intelligent analysis results to obtain a task requirement score; Conduct a preliminary comprehensive analysis of the edge node resource scores and task requirement scores to obtain the initial node scheduling plan, conduct a load balancing analysis on the initial node scheduling plan, and obtain the load balancing coefficient; Optimizing and adjusting the initial node scheduling scheme according to the load balancing coefficient to obtain the edge node scheduling strategy; Based on the edge node scheduling strategy, an edge node collaboration graph is constructed, and graph theory analysis is performed on the edge node collaboration graph to obtain an inter-node collaboration efficiency index; Optimizing the data transmission path between the edge nodes according to the inter-node collaboration efficiency index to obtain a data transmission strategy; Integrate the edge node scheduling strategy and the data transmission strategy to obtain an initial resource plan, perform simulation testing on the initial resource plan, and obtain simulation performance indicators; The initial resource solution is iteratively optimized based on the simulation performance index to obtain an iterative resource solution, and when the iterative resource solution meets a preset performance threshold, the global resource optimization strategy is output.

7. An Internet of Things data processing system based on edge computing, characterized in that: The IoT data processing method based on edge computing applied to any one of claims 1 to 6 above comprises: The acquisition module is used to obtain the raw data collected by the IoT sensors, and obtain the resource status data and network topology information of the IoT edge computing environment; An analysis module, the analysis module is used to pre-process the original data to obtain pre-processed data, and perform a transmission strategy analysis on the pre-processed data based on the network topology information to obtain transmitted data; An association module, the association module is used to map the transmitted data to an edge node, and perform distributed analysis on the transmitted data according to the resource status data to obtain a corresponding edge analysis result; A processing module, the processing module is used to perform multi-source data fusion according to the edge analysis result to obtain corresponding global data, and perform in-depth analysis based on the global data to obtain corresponding intelligent analysis results; A control module is used to perform resource scheduling analysis based on the resource status data and the intelligent analysis results to obtain an edge node scheduling strategy, and to perform a comprehensive analysis on all the edge node scheduling strategies to obtain a global resource optimization strategy.

8. An IoT data processing device based on edge computing, characterized in that: include: Memory, used to store programs; A processor is used to execute the program to implement the various steps of the Internet of Things data processing method based on edge computing as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: Computer instructions are stored, and the computer instructions are used to make a computer execute the method according to any one of claims 1 to 6.

Images