Internet of Things operation management scheduling system based on cloud edge collaboration

Abstract

The invention discloses an Internet of Things operation management scheduling system based on cloud edge collaboration, relates to the technical field of task scheduling optimization, and aims to solve the problem of adaptability of edge computing and cloud edge collaboration architecture, the Internet of Things perception access technology is adapted and fused with heterogeneous equipment through an architecture adaptation module, various IOT data are stably acquired, and the cloud edge collaboration is realized. The task execution position is dynamically adjusted in combination with the task distribution unit; real-time stream calculation and an ETL processing mechanism are combined to fit a comprehensive scheduling index Zdzh, the utilization rate of calculation resources is increased, and task scheduling is optimized; internet of Things equipment data, internet data and government affair data are analyzed based on the data lake and warehouse architecture, a data access strategy is optimized, and the data fusion efficiency is improved; and calculating and evaluating a sudden anomaly prediction coefficient Tycs, triggering an anomaly early warning mechanism or a sudden anomaly emergency mechanism, optimizing task migration, calculating node load balancing and data traceability analysis, so as to improve the stability and intelligent level of an Internet of Things operation management scheduling system.

Description

Technical Field

[0001] The present invention relates to the technical field of task scheduling optimization, and specifically to an Internet of Things operation management and scheduling system based on cloud-edge collaboration. Background Art

[0002] The concept of the IoT operation management and dispatching system originated from the development of IoT technology and was first applied to industrial automation and smart city management. Its core goal is to enhance the intelligence level of group-level and multi-level business management. Early enterprise dispatching systems mainly relied on traditional ERP, SCADA and other management platforms. However, due to data silos, lack of real-time performance and limited remote control capabilities, they could not meet the operation and maintenance needs of complex projects. With the development of IoT technology, smart sensors, 5G communications and edge computing have been gradually applied to enterprise management, enabling the IoT operation management and dispatching system to achieve integrated management from data collection, transmission, analysis to remote decision-making.

[0003] However, the existing IoT operation management and dispatching system needs to address the following three major technical shortcomings: The adaptability issue of edge computing and cloud-edge collaborative architecture: The existing cloud computing architecture is difficult to meet the low-latency, high-throughput, and distributed computing requirements required for ultra-large-scale device access, resulting in limited computing power on the device side, reduced command response speed, and increased data transmission pressure.

[0004] The problem of integrating and efficiently analyzing massive heterogeneous data: The IoT operation and management scheduling system needs to process massive data from multiple sources, formats, and dimensions, including IoT device data, Internet data, behavioral data, and government data. These data have different formats and update frequencies. Existing data fusion and analysis technologies are difficult to efficiently perform data cleaning, feature extraction, and association modeling, resulting in the data value not being fully tapped.

[0005] Issues with the real-time and accuracy of remote linkage and intelligent scheduling: The current group operation management and scheduling methods have problems such as delayed information transmission, insufficient instruction execution accuracy, and difficulty in cross-regional remote collaboration. Especially in complex scheduling scenarios with multiple levels and multiple businesses running in parallel, it is difficult for the group headquarters to achieve efficient management of subordinate units and projects. Summary of the Invention

[0006] In response to the shortcomings of the existing technology, the present invention provides an Internet of Things operation management and scheduling system based on cloud-edge collaboration, which solves the technical shortcomings mentioned in the background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: an IoT operation management and dispatching system based on cloud-edge collaboration, including an architecture adaptation module, a data fusion module, an intelligent dispatching module, an optimization control module and an early warning response module; The architecture adaptation module adapts and integrates heterogeneous devices through IoT sensing access technology to stably collect various types of IoT data. After receiving the scheduling task, it uses the task dispatcher to assign the task to the cloud-edge collaborative computing architecture and dynamically adjusts the task execution location based on device access status, computing load, and data priority. The data fusion module, based on the data lake warehouse architecture, converts, cleans, and integrates IoT device data, Internet data, and government data for storage; and finally constructs a standardized data set. The intelligent scheduling module calculates the edge computing adaptability coefficient Byx, the data fusion effectiveness coefficient Srx, and the remote scheduling accuracy coefficient Ldz based on the standardized data set; it also fits the comprehensive scheduling index Zdzh by combining real-time stream computing and ETL processing mechanisms; The optimization control module calculates the scheduling error threshold W based on AI image recognition, semantic analysis, and neural network algorithms, and evaluates it with the comprehensive scheduling index Zdzh; dynamically adjusts scheduling paths, resource allocation, and task priorities; The early warning response module calculates and evaluates the sudden anomaly prediction coefficient Tycs, triggers the anomaly early warning mechanism and sudden anomaly emergency mechanism, adjusts tasks and performs data traceability analysis.

[0008] Preferably, the architecture adaptation module includes a device access unit and a task distribution unit; When the device access unit is deployed in the IoT system, it first adapts heterogeneous devices through IoT sensing access technology, analyzes the communication protocols of various IoT devices, and establishes device access associations based on standardized data interfaces. Next, it registers the access device with a unique identifier and synchronously stores the device's basic information, communication parameters, and real-time status. During the device access process, it collects the data transmission rate, computing power, and operating status of each device, and dynamically classifies and manages the devices based on the set access rules. Once the device access is complete, the system regularly updates the status of online devices, marks abnormal devices based on real-time monitoring results, and records communication anomaly information. At the same time, it verifies device access permissions in conjunction with the set security authentication mechanism. During the data collection process, the preset data collection strategy is used to extract the original data from the access device according to the set collection cycle and data type, and preliminary data format conversion and integrity verification are performed; the collected data is cached in a distributed storage manner and indexed and managed according to timestamps and device identification.

[0009] Preferably, after receiving the scheduled task, the task dispatching unit parses the task content by calling the task dispatcher and determines the task execution mode based on the task type, computing requirements and real-time scheduling strategy; the task dispatcher reads the device access status set, extracts the available computing resources, data transmission capacity and network stability parameters of the current online device, and synchronously obtains the computing load information of the cloud and edge computing nodes; According to the task execution requirements, the computing power, storage capacity and task processing delay of the cloud and edge computing nodes are compared and analyzed; after the task adaptability calculation is completed, the system will prioritize the high-priority tasks to the computing nodes with lower computing loads based on the set task scheduling priority, and adjust the task execution path and data call method after the task execution location is determined; the task dispatcher sends the task execution instruction to the target computing node, and records the task allocation results and execution logs, and at the same time triggers the task status monitoring mechanism to collect the computing status, data flow and node load changes during the task execution process in real time, and dynamically adjust the task execution location according to the computing load changes and task urgency during the task execution process.

[0010] Preferably, the data fusion module is based on a data lake warehouse architecture. When accessing data, it first performs source data analysis on IoT device data, Internet data, and government data, extracts data format, field structure, and data association features, and performs format conversion according to the data storage strategy. During the format conversion process, it performs field standardization mapping on structured data including IoT device data, analyzes nested fields on semi-structured data including Internet data, and generates a unified data index. Unstructured data including government data is classified and stored, and metadata tags are added. During the data cleaning phase, missing values, outliers, and redundant data are detected, and missing values ​​are filled, outliers are eliminated, and duplicate data are merged according to data integrity rules. During the data fusion process, a data association model is constructed and based on it, data entities from different data sources are matched, and multi-dimensional associations are performed in combination with timestamps, geographic location information, and device identification. After data fusion is completed, the system divides the data into the original data layer, the cleaned data layer, and the application data layer according to the layered storage strategy of the data lake warehouse architecture, and sets different levels of retrieval indexes based on the data access strategy.

[0011] Preferably, the intelligent scheduling module includes an edge computing adaptation analysis unit and a scheduling index calculation unit; The edge computing adaptation analysis unit analyzes the device access status, computing node load and task execution requirements based on the standardized data set output by the data fusion module, and obtains the computing node execution capability-related data of each computing node based on the task allocation record; including the node load peak ratio Clp, power consumption change rate Pcv, cache hit rate Cmr and task execution interruption rate Tex; according to the set edge computing adaptability evaluation rules, the computing node execution capability is normalized and calculated in combination with the following formula to obtain the edge computing adaptation coefficient Byx;

[0012] Then, based on the multi-task execution data, the task response time, load changes and task scheduling frequency of the computing nodes are counted, and based on the convolutional neural network, a computing capacity trend model is constructed to dynamically update the data of the edge computing adaptation coefficient Byx; finally, the edge computing adaptation coefficient Byx is stored in the scheduling optimization database and synchronized to the scheduling index calculation unit.

[0013] Preferably, after receiving the edge computing adaptation coefficient Byx, the scheduling index calculation unit extracts fusion validity related data based on the standardized data set, including the update drift rate Dup, the feature information loss rate Ftl, the data alignment error Dal and the cross-source data redundancy value Rdx, and calculates the data fusion validity coefficient Srx using the following formula:

[0014] Then, based on the standardized data set, we further extract the scheduling record-related data of the remote tasks and perform dimensionless processing, including the task execution dependency dispersion Ted, the number of scheduling path switching times Spt, and the multi-task interference index Mti. Then, we correlate the scheduling record-related data and calculate the remote scheduling accuracy coefficient Ldz. The specific calculation formula is as follows:

[0015] Based on the calculation results of the edge computing adaptation coefficient Byx, the data fusion effectiveness coefficient Srx, and the remote scheduling accuracy coefficient Ldz, linear normalization processing is performed. Combined with real-time stream computing and ETL processing mechanisms, the comprehensive scheduling index Zdzh of task scheduling is calculated using the following formula;

[0016] Where n represents the number of scheduled tasks, i represents the index of the iiith task; Byx represents the edge computing adaptation coefficient of the i-th task, represents the data fusion effectiveness coefficient Srx of the i-th task, represents the remote scheduling accuracy coefficient Ldz of the i-th task; After the calculation is completed, the comprehensive scheduling index Zdzh is stored in the scheduling optimization database and synchronized to the optimization control module.

[0017] Preferably, the optimization control module includes a threshold setting unit and a scheduling evaluation unit; The threshold preset unit first collects a historical data set of task scheduling, extracts the scheduling path, resource occupancy and task completion status of the executed tasks, and analyzes the scheduling visualization data based on AI image recognition technology to extract key scheduling feature parameters; then, by constructing a semantic analysis model, it extracts the task type, execution constraints and key scheduling indicators, parses the text description information of the scheduling task, and finally sets the comprehensive scheduling threshold W and synchronizes it to the scheduling evaluation unit.

[0018] Preferably, the scheduling evaluation unit compares and evaluates the comprehensive scheduling threshold W with the comprehensive scheduling index Zdzh, and dynamically adjusts the scheduling path, resource allocation and task priority; the specific contents are as follows: When the comprehensive scheduling index Zdzh ≥ the comprehensive scheduling threshold W, it means that the overall operation of the current scheduling task is stable, and the task allocation, resource utilization and scheduling strategy are in line with expectations. There is no need to adjust the scheduling path or resource allocation strategy, and the current scheduling plan is maintained; When the comprehensive scheduling index Zdzh is less than the comprehensive scheduling threshold W, it indicates that the overall operation of the current scheduling task is abnormal. At this time, the scheduling path, resource allocation and task priority are adjusted; Preferably, the early warning response module includes an abnormality calculation unit and an early warning evaluation unit; The abnormality calculation unit extracts key abnormality monitoring related data from the scheduling log, task execution status and computing node load information, including task execution delay, resource usage abnormality and data transmission fluctuation, and uses the convolutional neural network to build a timing analysis model; then, based on the timing analysis model, the abnormality detection algorithm is used to perform cluster analysis on the abnormal pattern, obtain the task scheduling fallback ratio TA, task scheduling fallback ratio TB, cross-node data retransmission rate TC and abnormal trigger cooling time TD, and after dimensionless processing, use the following formula to calculate the sudden abnormality prediction coefficient Tycs:

[0019] Secondly, the sudden abnormality prediction coefficient Tycs is synchronized to the early warning evaluation unit.

[0020] Preferably, after receiving the sudden abnormality prediction coefficient Tycs, the early warning evaluation unit reads the set first sudden abnormality prediction threshold Wt1 and second sudden abnormality prediction threshold Wt2 from the early warning database, compares and evaluates them with the sudden abnormality prediction coefficient Tycs, and the first sudden abnormality prediction threshold Wt1 is greater than the second sudden abnormality prediction threshold Wt2; The specific contents are as follows: When the sudden anomaly prediction coefficient Tycs is less than the second sudden anomaly prediction threshold Wt2, it indicates that the current task scheduling is in a stable state, and there are no anomalies in the computing resource allocation, task execution, and data transmission in the system. There is no need to adjust the current scheduling strategy. When the second sudden anomaly prediction threshold Wt2 ≤ the sudden anomaly prediction coefficient Tycs < the first sudden anomaly prediction threshold Wt1, it indicates that the task scheduling trend tends to deviate from the normal range. At this time, there are problems such as computing node load fluctuation, task execution delay, or unstable data transmission. At this time, the abnormality warning mechanism is triggered, including task scheduling adjustment, data transmission adjustment, and scheduling monitoring adjustment. When the sudden anomaly prediction coefficient Tycs ≥ the first sudden anomaly prediction threshold Wt1, it means that an abnormality occurs in task scheduling, resulting in task execution failure, system resource exhaustion, or long-term task incompleteness. At this time, emergency measures are taken immediately and the sudden anomaly emergency mechanism is triggered, including customized task migration strategy, scheduling priority adjustment, computing node load balancing, and data traceability analysis.

[0021] The present invention provides an IoT operation management and dispatching system based on cloud-edge collaboration. It has the following beneficial effects: (1) This is an IoT operation and management scheduling system based on cloud-edge collaboration. It aims to solve the problem of edge computing and cloud-edge collaborative architecture adaptability of IoT operation and management scheduling system. By building an architecture adaptation module, IoT sensing access technology can adapt to and integrate heterogeneous devices, and stably collect various types of IoT data. Combined with the task distribution unit, it dynamically adjusts the task execution position according to the device access status, computing load and data priority. The intelligent scheduling module calculates the edge computing adaptation coefficient Byx, the data fusion effectiveness index Srx and the remote scheduling accuracy coefficient Ldz based on the standardized data set output by the data fusion module, and fits the comprehensive scheduling index Zdzh by combining real-time stream computing and ETL processing mechanism. The optimization control module calculates the scheduling error threshold W, and compares and evaluates it with the comprehensive scheduling index Zdzh, and dynamically adjusts the scheduling path, resource allocation and task priority. This architecture optimizes the task execution mode through the cloud-edge collaborative computing architecture to improve the utilization of computing resources. It reduces the problem of reduced instruction response speed caused by limited computing power on the device side, and optimizes the data transmission pressure, so that IoT devices can run efficiently and realize the precise scheduling of distributed computing needs. (2) This is an IoT operation and management scheduling system based on cloud-edge collaboration. Aiming at the problem of fusion and efficient analysis of massive heterogeneous data, this solution adopts a data lake warehouse architecture. The data fusion module analyzes IoT device data, Internet data and government data when data is accessed, and performs format conversion, data cleaning and data fusion according to the data storage strategy to form a standardized data set; combined with AI semantic analysis and knowledge graph technology, it improves data relevance and data alignment accuracy; the intelligent scheduling module extracts the update drift rate Dup, feature information loss rate Ftl, data alignment error Dal and cross-source data redundancy value Rdx, and calculates the data fusion effectiveness index Srx based on the calculation model; by optimizing the scheduling index and data access strategy, multi-source, multi-format, and multi-dimensional data can still have efficient retrieval, intelligent matching and visualization processing capabilities in a high-concurrency environment, reduce data redundancy, improve data utilization value, and solve the problem that existing data fusion analysis technologies are difficult to effectively clean, extract features and build efficient models; (3) This is an IoT operation and management scheduling system based on cloud-edge collaboration. Aiming at the real-time and accuracy issues of remote linkage and intelligent scheduling, this solution calculates and evaluates the comprehensive scheduling index Zdzh through the optimization control module; according to the evaluation content, the scheduling path, resource allocation and task priority are adjusted to ensure that the task scheduling maintains high efficiency and accuracy during the remote linkage process; the early warning response module calculates the sudden anomaly prediction coefficient Tycs and compares and evaluates it based on the set first sudden anomaly prediction threshold Wt1 and second sudden anomaly prediction threshold Wt2; according to the evaluation content, the sudden anomaly emergency mechanism is triggered, including task migration strategy optimization, scheduling priority adjustment, computing node load balancing and data traceability analysis; secondly, the abnormal early warning mechanism is triggered to optimize task scheduling, data transmission and scheduling monitoring, and improve the accuracy and real-time performance of the scheduling system; by building an efficient and intelligent early warning and scheduling mechanism, the group headquarters can accurately remotely control subordinate units and projects, and realize cross-regional and cross-level intelligent scheduling and optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the framework structure of an Internet of Things operation management and scheduling system based on cloud-edge collaboration in the present invention; DETAILED DESCRIPTION The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0023] Example 1 See also Figure 1, the present invention provides an Internet of Things operation management and scheduling system based on cloud-edge collaboration, including an architecture adaptation module, a data fusion module, an intelligent scheduling module, an optimization control module and an early warning response module; The architecture adaptation module adapts and integrates heterogeneous devices through IoT sensing access technology to stably collect various types of IoT data. After receiving the scheduling task, it uses the task dispatcher to assign the task to the cloud-edge collaborative computing architecture and dynamically adjusts the task execution location based on device access status, computing load, and data priority. The data fusion module, based on the data lake warehouse architecture, converts, cleans, and integrates IoT device data, Internet data, and government data for storage; and finally constructs a standardized data set. The intelligent scheduling module calculates the edge computing adaptability coefficient Byx, the data fusion effectiveness coefficient Srx, and the remote scheduling accuracy coefficient Ldz based on the standardized data set; it also fits the comprehensive scheduling index Zdzh by combining real-time stream computing and ETL processing mechanisms; The optimization control module calculates the scheduling error threshold W based on AI image recognition, semantic analysis, and neural network algorithms, and evaluates it with the comprehensive scheduling index Zdzh; dynamically adjusts scheduling paths, resource allocation, and task priorities; The early warning response module calculates and evaluates the sudden anomaly prediction coefficient Tycs, triggers the anomaly early warning mechanism and sudden anomaly emergency mechanism, adjusts tasks and performs data traceability analysis.

[0024] In this embodiment, the architecture adaptation module adapts and integrates heterogeneous devices through the Internet of Things perception access technology to ensure that different types of IOT devices can be efficiently accessed, and dynamically adjusts the task execution position through the task distributor to improve the flexibility of task scheduling and the utilization of computing resources; the data fusion module performs data format conversion, cleaning and fusion storage based on the data lake warehouse architecture, and builds a standardized data set to ensure the structured management of Internet of Things device data, Internet data and government data, and improve the efficiency of data query and call; the intelligent scheduling module calculates the edge computing adaptation coefficient Byx, data fusion effectiveness index Srx and remote scheduling accuracy coefficient Ldz based on the standardized data set, and combines real-time stream computing and The ETL processing mechanism fits the comprehensive scheduling index Zdzh, so that scheduling tasks can accurately match computing resources and improve computing efficiency; the optimization control module calculates the scheduling error threshold W based on AI image recognition, semantic analysis and neural network algorithm, and evaluates it with the comprehensive scheduling index Zdzh to dynamically adjust the scheduling path, optimize resource allocation and improve task execution priority to ensure the accuracy and real-time performance of task execution; the early warning response module calculates the sudden anomaly prediction coefficient Tycs, and by comparing the anomaly threshold, triggers the anomaly warning mechanism or sudden anomaly emergency mechanism, adjusts the task execution strategy and data traceability analysis, and improves the adaptive adjustment capability and overall stability of the IoT operation management and scheduling system under abnormal conditions.

[0025] Example 2 The architecture adaptation module includes a device access unit and a task distribution unit; When the device access unit is deployed in the IoT system, it first adapts heterogeneous devices through IoT sensing access technology, analyzes the communication protocols of various IoT devices, and establishes device access associations based on standardized data interfaces. Next, it registers the access device with a unique identifier and synchronously stores the device's basic information, communication parameters, and real-time status. During the device access process, it collects the data transmission rate, computing power, and operating status of each device, and dynamically classifies and manages the devices based on the set access rules. Once the device access is complete, the system regularly updates the status of online devices, marks abnormal devices based on real-time monitoring results, and records communication anomaly information. At the same time, it verifies device access permissions in conjunction with the set security authentication mechanism. During the data collection process, the preset data collection strategy is used to extract the original data from the access device according to the set collection cycle and data type, and preliminary data format conversion and integrity verification are performed; the collected data is cached in a distributed storage manner and indexed and managed according to timestamps and device identification.

[0026] After receiving the scheduled task, the task dispatch unit parses the task content by calling the task dispatcher and determines the task execution mode based on the task type, computing requirements and real-time scheduling strategy; the task dispatcher reads the device access status set, extracts the available computing resources, data transmission capacity and network stability parameters of the current online device, and synchronously obtains the computing load information of the cloud and edge computing nodes; According to the task execution requirements, the computing power, storage capacity and task processing delay of the cloud and edge computing nodes are compared and analyzed; after the task adaptability calculation is completed, the system will prioritize the high-priority tasks to the computing nodes with lower computing loads based on the set task scheduling priority, and adjust the task execution path and data call method after the task execution location is determined; the task dispatcher sends the task execution instruction to the target computing node, and records the task allocation results and execution logs, and at the same time triggers the task status monitoring mechanism to collect the computing status, data flow and node load changes during the task execution process in real time, and dynamically adjust the task execution location according to the computing load changes and task urgency during the task execution process.

[0027] The data fusion module is based on the data lake warehouse architecture. When accessing data, it first parses the source data of IoT device data, Internet data, and government data, extracts the data format, field structure, and data association characteristics, and performs format conversion according to the data storage strategy. During the format conversion process, it performs field standardization mapping on structured data, including IoT device data, parses nested fields for semi-structured data, including Internet data, and generates a unified data index. Unstructured data, including government data, is classified and stored, and metadata tags are attached. During the data cleaning phase, missing values, outliers, and redundant data are detected, and missing values ​​are filled, outliers are eliminated, and duplicate data are merged according to data integrity rules. During the data fusion process, a data association model is constructed and based on it, data entities from different data sources are matched, and multi-dimensional associations are performed in combination with timestamps, geographic location information, and device identification. After data fusion is completed, the system divides the data into the original data layer, the cleaned data layer, and the application data layer according to the layered storage strategy of the data lake warehouse architecture, and sets different levels of retrieval indexes based on the data access strategy.

[0028] In this embodiment, the architecture adaptation module ensures efficient access and dynamic task allocation of IoT devices through the device access unit and the task distribution unit. The device access unit analyzes the communication protocols of heterogeneous devices through the IoT perception access technology, and establishes device access associations based on standardized data interfaces to achieve unique identification registration, communication parameter synchronization, and real-time status storage of the devices. The system regularly collects the data transmission rate, computing power, and operating status of the devices, and dynamically classifies and manages the devices according to the set access rules to ensure the efficiency and stability of the access devices. After receiving the scheduling task, the task distribution unit calls the task distributor to parse the task content and determines the task execution mode based on the task type, computing requirements, and real-time scheduling strategy. At the same time, it reads the device access status set, extracts available computing resources, data transmission capacity, and network stability parameters, and synchronously obtains the computing load information of the cloud and edge computing nodes to ensure the accuracy of task scheduling and the rationality of resource allocation. The data fusion module is based on the data lake warehouse architecture, and performs source data analysis on IoT device data, Internet data and government data, extracts data format, field structure and data association features, and performs format conversion according to the data storage strategy to improve the ability of data standardization management; in the data cleaning stage, it detects missing values, outliers and redundant data, and performs missing values ​​filling, outlier elimination and duplicate data merging according to data integrity rules to ensure data integrity and accuracy; in the data fusion process, it constructs and performs multi-dimensional data matching based on the data association model, and combines timestamps, geographic location information and device identification for multi-dimensional association to improve the temporal consistency and spatial association of data; finally, the system divides the data into original data layer, cleaned data layer and application data layer according to the layered storage strategy of the data lake warehouse architecture, and sets different levels of retrieval indexes according to the data access strategy to optimize data storage efficiency, improve the real-time and accuracy of data queries, and ensure that the IoT operation and management scheduling system can still operate efficiently and stably in a massive data environment.

[0029] Example 3 The intelligent scheduling module includes an edge computing adaptation analysis unit and a scheduling index calculation unit; The edge computing adaptation analysis unit analyzes the device access status, computing node load and task execution requirements based on the standardized data set output by the data fusion module, and obtains the computing node execution capability-related data of each computing node based on the task allocation record; including the node load peak ratio Clp, power consumption change rate Pcv, cache hit rate Cmr and task execution interruption rate Tex; according to the set edge computing adaptability evaluation rules, the computing node execution capability is normalized and calculated in combination with the following formula to obtain the edge computing adaptation coefficient Byx;

[0030] Then, based on the multi-task execution data, the task response time, load changes and task scheduling frequency of the computing nodes are counted, and based on the convolutional neural network, a computing capacity trend model is constructed to dynamically update the data of the edge computing adaptation coefficient Byx; finally, the edge computing adaptation coefficient Byx is stored in the scheduling optimization database and synchronized to the scheduling index calculation unit.

[0031] After receiving the edge computing adaptation coefficient Byx, the scheduling index calculation unit extracts fusion validity related data based on the standardized data set, including the update drift rate Dup, the feature information loss rate Ftl, the data alignment error Dal, and the cross-source data redundancy value Rdx, and calculates the data fusion validity coefficient Srx using the following formula:

[0032] Then, based on the standardized data set, we further extract the scheduling record-related data of the remote tasks and perform dimensionless processing, including the task execution dependency dispersion Ted, the number of scheduling path switching times Spt, and the multi-task interference index Mti. Then, we correlate the scheduling record-related data and calculate the remote scheduling accuracy coefficient Ldz. The specific calculation formula is as follows:

[0033] Based on the calculation results of the edge computing adaptation coefficient Byx, the data fusion effectiveness coefficient Srx, and the remote scheduling accuracy coefficient Ldz, linear normalization processing is performed. Combined with real-time stream computing and ETL processing mechanisms, the comprehensive scheduling index Zdzh of task scheduling is calculated using the following formula;

[0034] Where n represents the number of scheduled tasks, i represents the index of the iiith task; Byx represents the edge computing adaptation coefficient of the i-th task, represents the data fusion effectiveness coefficient Srx of the i-th task, represents the remote scheduling accuracy coefficient Ldz of the i-th task; After the calculation is completed, the comprehensive scheduling index Zdzh is stored in the scheduling optimization database and synchronized to the optimization control module.

[0035] The optimization control module includes a threshold setting unit and a scheduling evaluation unit; The threshold preset unit first collects a historical data set of task scheduling, extracts the scheduling path, resource occupancy and task completion status of the executed tasks, and analyzes the scheduling visualization data based on AI image recognition technology to extract key scheduling feature parameters; then, by constructing a semantic analysis model, it extracts the task type, execution constraints and key scheduling indicators, parses the text description information of the scheduling task, and finally sets the comprehensive scheduling threshold W and synchronizes it to the scheduling evaluation unit.

[0036] The scheduling evaluation unit compares and evaluates the comprehensive scheduling threshold W with the comprehensive scheduling index Zdzh, and dynamically adjusts the scheduling path, resource allocation and task priority; the specific contents are as follows: When the comprehensive scheduling index Zdzh ≥ the comprehensive scheduling threshold W, it means that the overall operation of the current scheduling task is stable, and the task allocation, resource utilization and scheduling strategy are in line with expectations. There is no need to adjust the scheduling path or resource allocation strategy, and the current scheduling plan is maintained; When the comprehensive scheduling index Zdzh is less than the comprehensive scheduling threshold W, it indicates that the overall operation of the current scheduling task is abnormal. At this time, the scheduling path, resource allocation and task priority are adjusted; The early warning response module includes an abnormality calculation unit and an early warning evaluation unit; The abnormality calculation unit extracts key abnormality monitoring related data from the scheduling log, task execution status and computing node load information, including task execution delay, resource usage abnormality and data transmission fluctuation, and uses the convolutional neural network to build a timing analysis model; then, based on the timing analysis model, the abnormality detection algorithm is used to perform cluster analysis on the abnormal pattern, obtain the task scheduling fallback ratio TA, task scheduling fallback ratio TB, cross-node data retransmission rate TC and abnormal trigger cooling time TD, and after dimensionless processing, use the following formula to calculate the sudden abnormality prediction coefficient Tycs:

[0037] Secondly, the sudden abnormality prediction coefficient Tycs is synchronized to the early warning evaluation unit.

[0038] After receiving the sudden abnormality prediction coefficient Tycs, the early warning evaluation unit reads the set first sudden abnormality prediction threshold Wt1 and second sudden abnormality prediction threshold Wt2 from the early warning database, compares and evaluates them with the sudden abnormality prediction coefficient Tycs, and the first sudden abnormality prediction threshold Wt1 is greater than the second sudden abnormality prediction threshold Wt2; The specific contents are as follows: When the sudden anomaly prediction coefficient Tycs is less than the second sudden anomaly prediction threshold Wt2, it indicates that the current task scheduling is in a stable state, and there are no anomalies in the computing resource allocation, task execution, and data transmission in the system. There is no need to adjust the current scheduling strategy. When the second sudden anomaly prediction threshold Wt2 ≤ the sudden anomaly prediction coefficient Tycs < the first sudden anomaly prediction threshold Wt1, it indicates that the task scheduling trend tends to deviate from the normal range. At this time, there are problems such as computing node load fluctuation, task execution delay, or unstable data transmission. At this time, the abnormality warning mechanism is triggered, including task scheduling adjustment, data transmission adjustment, and scheduling monitoring adjustment. When the sudden anomaly prediction coefficient Tycs ≥ the first sudden anomaly prediction threshold Wt1, it means that an abnormality occurs in task scheduling, resulting in task execution failure, system resource exhaustion, or long-term task incompleteness. At this time, emergency measures are taken immediately and the sudden anomaly emergency mechanism is triggered, including customized task migration strategy, scheduling priority adjustment, computing node load balancing, and data traceability analysis.

[0039] In this embodiment, the intelligent scheduling module optimizes the task execution strategy and improves the allocation efficiency of computing resources through the edge computing adaptation analysis unit and the scheduling index calculation unit. The edge computing adaptation analysis unit analyzes the device access status, computing node load and task execution requirements based on the standardized data set output by the data fusion module, and obtains data related to the computing node execution capability, including the node load peak ratio Clp, power consumption change rate Pcv, cache hit rate Cmr and task execution interruption rate Tex, to evaluate the resource adaptability and computing stability of the computing node, and calculate the edge computing adaptation coefficient Byx; after receiving Byx, the scheduling index calculation unit extracts data based on the standardized data set Fusion effectiveness related data, including update drift rate Dup, feature information loss rate Ftl, data alignment error Dal and cross-source data redundancy value Rdx, calculates data fusion effectiveness coefficient Srx, evaluates data consistency and fusion stability, and further extracts remote task scheduling record related data, including task execution dependency dispersion Ted, scheduling path switching times Spt and multi-task interference index Mti, to calculate remote scheduling accuracy coefficient Ldz to evaluate the impact of resource competition and scheduling fluctuations on task accuracy during task execution; comprehensive scheduling index Zdzh is calculated through edge computing adaptation coefficient Byx, data fusion effectiveness coefficient Srx and remote scheduling accuracy coefficient Ldz Perform linear normalization processing, and combine real-time stream computing and ETL processing mechanisms to evaluate the overall execution stability of the scheduling task, and synchronize it to the optimization control module; the optimization control module collects historical task data through the threshold preset unit, extracts the scheduling path, resource occupancy and task completion status, and sets the comprehensive scheduling threshold W for comparison and evaluation with the comprehensive scheduling index Zdzh, dynamically adjusts the scheduling path, resource allocation and task priority to optimize the scheduling strategy and improve the accuracy of task scheduling; the early warning response module extracts key abnormal monitoring data from the scheduling log, task execution status and computing node load information based on the abnormal computing unit, including task execution delay, resource usage abnormality and The system calculates the data transmission fluctuation and obtains the task scheduling fallback ratio TA, task scheduling fallback ratio TB, cross-node data retransmission rate TC and abnormal trigger cooling time TD to calculate the sudden abnormality prediction coefficient Tycs, predict the abnormal trend of task scheduling and synchronize it to the early warning evaluation unit, evaluate the comparison results of the sudden abnormality prediction coefficient Tycs with the first sudden abnormality prediction threshold Wt1 and the second sudden abnormality prediction threshold Wt2. When the sudden abnormality prediction coefficient Tycs exceeds the first sudden abnormality prediction threshold Wt1, the sudden abnormality emergency mechanism is triggered, including task migration strategy adjustment, scheduling priority optimization, computing node load balancing and data traceability analysis, to ensure the security and stability of task scheduling.

[0040] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An IoT operation management and dispatching system based on cloud-edge collaboration, characterized by: Including architecture adaptation module, data fusion module, intelligent scheduling module, optimization control module and early warning response module; The architecture adaptation module adapts and integrates heterogeneous devices through IoT sensing access technology to stably collect various types of IoT data. After receiving the scheduling task, it uses the task dispatcher to assign the task to the cloud-edge collaborative computing architecture and dynamically adjusts the task execution location based on device access status, computing load, and data priority. The data fusion module, based on the data lake warehouse architecture, converts, cleans, and integrates IoT device data, Internet data, and government data for storage; and finally constructs a standardized data set. The intelligent scheduling module calculates the edge computing adaptability coefficient Byx, the data fusion effectiveness coefficient Srx, and the remote scheduling accuracy coefficient Ldz based on the standardized data set; it also fits the comprehensive scheduling index Zdzh by combining real-time stream computing and ETL processing mechanisms; The optimization control module calculates the scheduling error threshold W based on AI image recognition, semantic analysis, and neural network algorithms, and evaluates it with the comprehensive scheduling index Zdzh; dynamically adjusts scheduling paths, resource allocation, and task priorities; The early warning response module calculates and evaluates the sudden anomaly prediction coefficient Tycs, triggers the anomaly early warning mechanism and sudden anomaly emergency mechanism, adjusts tasks and performs data traceability analysis.

2. An IoT operation management and dispatching system based on cloud-edge collaboration according to claim 1, characterized in that: The architecture adaptation module includes a device access unit and a task distribution unit; When the device access unit is deployed in the IoT system, it first adapts heterogeneous devices through IoT sensing access technology, analyzes the communication protocols of various IoT devices, and establishes device access associations based on standardized data interfaces. Next, it registers the access device with a unique identifier and synchronously stores the device's basic information, communication parameters, and real-time status. During the device access process, it collects the data transmission rate, computing power, and operating status of each device, and dynamically classifies and manages the devices based on the set access rules. Once the device access is complete, the system regularly updates the status of online devices, marks abnormal devices based on real-time monitoring results, and records communication anomaly information. At the same time, it verifies device access permissions in conjunction with the set security authentication mechanism. During the data collection process, the preset data collection strategy is used to extract the original data from the access device according to the set collection cycle and data type, and preliminary data format conversion and integrity verification are performed; the collected data is cached in a distributed storage manner and indexed and managed according to timestamps and device identification.

3. An IoT operation management and dispatching system based on cloud-edge collaboration according to claim 2, characterized in that: After receiving the scheduled task, the task dispatching unit parses the task content by calling the task dispatcher and determines the task execution mode based on the task type, computing requirements and real-time scheduling strategy; The task dispatcher reads the device access status set, extracts the available computing resources, data transmission capacity and network stability parameters of the current online device, and simultaneously obtains the computing load information of the cloud and edge computing nodes; According to the task execution requirements, the computing power, storage capacity and task processing delay of the cloud and edge computing nodes are compared and analyzed; after the task adaptability calculation is completed, the system will prioritize the high-priority tasks to the computing nodes with lower computing loads based on the set task scheduling priority, and adjust the task execution path and data call method after the task execution location is determined; the task dispatcher sends the task execution instruction to the target computing node, and records the task allocation results and execution logs, and at the same time triggers the task status monitoring mechanism to collect the computing status, data flow and node load changes during the task execution process in real time, and dynamically adjust the task execution location according to the computing load changes and task urgency during the task execution process.

4. An IoT operation management and dispatching system based on cloud-edge collaboration according to claim 1, characterized in that: The data fusion module is based on the data lake warehouse architecture. When accessing data, it first analyzes the source data of IoT device data, Internet data, and government data, extracts the data format, field structure, and data association features, and performs format conversion according to the data storage strategy. During the format conversion process, standardized field mapping is performed on structured data, including IoT device data; nested fields are parsed and a unified data index is generated for semi-structured data, including internet data; and unstructured data, including government data, is classified and stored with metadata tags attached. During the data cleaning phase, missing values, outliers, and redundant data are detected, and missing values ​​are filled, outliers are eliminated, and duplicate data is merged according to data integrity rules. During the data fusion process, a data association model is constructed and used to match data entities from different data sources, and multi-dimensional associations are performed based on timestamps, geographic location information, and device identification. After data fusion is completed, the system divides the data into original data layer, cleaned data layer and application data layer according to the layered storage strategy of the data lake warehouse architecture, and sets different levels of retrieval indexes based on the data access strategy.

5. An IoT operation management and dispatching system based on cloud-edge collaboration according to claim 1, characterized in that: The intelligent scheduling module includes an edge computing adaptation analysis unit and a scheduling index calculation unit; The edge computing adaptation analysis unit analyzes the device access status, computing node load and task execution requirements based on the standardized data set output by the data fusion module, and obtains the computing node execution capability related data of each computing node based on the task allocation record; Including node load peak ratio Clp, power consumption change rate Pcv, cache hit rate Cmr and task execution interruption rate Tex; according to the set edge computing adaptability evaluation rules, the computing node execution capability is normalized and calculated using the following formula to obtain the edge computing adaptability coefficient Byx; Then, based on the multi-task execution data, the task response time, load changes and task scheduling frequency of the computing nodes are counted, and based on the convolutional neural network, a computing capacity trend model is constructed to dynamically update the data of the edge computing adaptation coefficient Byx; finally, the edge computing adaptation coefficient Byx is stored in the scheduling optimization database and synchronized to the scheduling index calculation unit.

6. An IoT operation management and dispatching system based on cloud-edge collaboration according to claim 5, characterized in that: After receiving the edge computing adaptation coefficient Byx, the scheduling index calculation unit extracts fusion validity related data based on the standardized data set, including the update drift rate Dup, the feature information loss rate Ftl, the data alignment error Dal, and the cross-source data redundancy value Rdx, and calculates the data fusion validity coefficient Srx using the following formula: Then, based on the standardized data set, we further extract the scheduling record-related data of the remote tasks and perform dimensionless processing, including the task execution dependency dispersion Ted, the number of scheduling path switching times Spt, and the multi-task interference index Mti. Then, we correlate the scheduling record-related data and calculate the remote scheduling accuracy coefficient Ldz. The specific calculation formula is as follows: Based on the calculation results of the edge computing adaptation coefficient Byx, the data fusion effectiveness coefficient Srx, and the remote scheduling accuracy coefficient Ldz, linear normalization processing is performed. Combined with real-time stream computing and ETL processing mechanisms, the comprehensive scheduling index Zdzh of task scheduling is calculated using the following formula; Where n represents the number of scheduled tasks, i represents the index of the iiith task; Byx represents the edge computing adaptation coefficient of the i-th task, represents the data fusion effectiveness coefficient Srx of the i-th task, represents the remote scheduling accuracy coefficient Ldz of the i-th task; After the calculation is completed, the comprehensive scheduling index Zdzh is stored in the scheduling optimization database and synchronized to the optimization control module.

7. An IoT operation management and dispatching system based on cloud-edge collaboration according to claim 1, characterized in that: The optimization control module includes a threshold setting unit and a scheduling evaluation unit; The threshold preset unit first collects a historical data set of task scheduling, extracts the scheduling path, resource occupancy and task completion status of the executed tasks, and analyzes the scheduling visualization data based on AI image recognition technology to extract key scheduling feature parameters; then, by constructing a semantic analysis model, it extracts the task type, execution constraints and key scheduling indicators, parses the text description information of the scheduling task, and finally sets the comprehensive scheduling threshold W and synchronizes it to the scheduling evaluation unit.

8. An IoT operation management and dispatching system based on cloud-edge collaboration according to claim 7, characterized in that: The scheduling evaluation unit compares and evaluates the comprehensive scheduling threshold W with the comprehensive scheduling index Zdzh, and dynamically adjusts the scheduling path, resource allocation and task priority; the specific contents are as follows: When the comprehensive scheduling index Zdzh ≥ the comprehensive scheduling threshold W, it means that the overall operation of the current scheduling task is stable, and the task allocation, resource utilization and scheduling strategy are in line with expectations. There is no need to adjust the scheduling path or resource allocation strategy, and the current scheduling plan is maintained; When the comprehensive scheduling index Zdzh is less than the comprehensive scheduling threshold W, it indicates that the overall operation of the current scheduling task is abnormal. At this time, the scheduling path, resource allocation and task priority are adjusted.

9. An IoT operation management and dispatching system based on cloud-edge collaboration according to claim 1, characterized in that: The early warning response module includes an abnormality calculation unit and an early warning evaluation unit; The abnormality calculation unit extracts key abnormality monitoring related data from the scheduling log, task execution status and computing node load information, including task execution delay, resource usage abnormality and data transmission fluctuation, and uses the convolutional neural network to build a timing analysis model; then, based on the timing analysis model, the abnormality detection algorithm is used to perform cluster analysis on the abnormal pattern, obtain the task scheduling fallback ratio TA, task scheduling fallback ratio TB, cross-node data retransmission rate TC and abnormal trigger cooling time TD, and after dimensionless processing, use the following formula to calculate the sudden abnormality prediction coefficient Tycs: Secondly, the sudden abnormality prediction coefficient Tycs is synchronized to the early warning evaluation unit.

10. An IoT operation management and dispatching system based on cloud-edge collaboration according to claim 9, characterized in that: After receiving the sudden abnormality prediction coefficient Tycs, the early warning evaluation unit reads the set first sudden abnormality prediction threshold Wt1 and second sudden abnormality prediction threshold Wt2 from the early warning database, compares and evaluates them with the sudden abnormality prediction coefficient Tycs, and the first sudden abnormality prediction threshold Wt1 is greater than the second sudden abnormality prediction threshold Wt2; The specific contents are as follows: When the sudden anomaly prediction coefficient Tycs is less than the second sudden anomaly prediction threshold Wt2, it indicates that the current task scheduling is in a stable state, and there are no anomalies in the computing resource allocation, task execution, and data transmission in the system. There is no need to adjust the current scheduling strategy. When the second sudden anomaly prediction threshold Wt2 ≤ the sudden anomaly prediction coefficient Tycs < the first sudden anomaly prediction threshold Wt1, it indicates that the task scheduling trend tends to deviate from the normal range. At this time, there are problems such as computing node load fluctuation, task execution delay, or unstable data transmission. At this time, the abnormal warning mechanism is triggered, including task scheduling adjustment, data transmission adjustment, and scheduling monitoring adjustment; When the sudden anomaly prediction coefficient Tycs ≥ the first sudden anomaly prediction threshold Wt1, it means that an abnormality occurs in task scheduling, resulting in task execution failure, system resource exhaustion, or long-term task incompleteness. At this time, emergency measures are taken immediately and the sudden anomaly emergency mechanism is triggered, including customized task migration strategy, scheduling priority adjustment, computing node load balancing, and data traceability analysis.

Images