Internet of Things service data management method based on end-side cloud cooperative driving

Abstract

The invention relates to an end-side cloud cooperative driven Internet of Things service data management method, and relates to the technical field of computer processing, and the method comprises the following steps: S01, configuring a data preprocessing rule and an analysis strategy at a cloud management platform, and publishing a data management strategy packet to a cloud strategy library; s02, the cloud management platform dynamically issues the data management strategy packet to a specified edge node according to the type of an end side device and the load state of the edge node; s03, the edge node and the cloud management platform establish a secure channel, and metadata indexes and preprocessing rules in a cloud strategy library are synchronized; and S04, the edge node collects the original data of the end side equipment in real time. According to the invention, based on an end-side cloud three-layer dynamic collaborative architecture, through edge lightweight real-time processing and cloud global intelligent analysis, the transmission efficiency, resource utilization rate and decision intelligence of data management of the Internet of Things are realized.

Description

Technical Field

[0001] The present invention relates to the field of computer processing technology, and in particular to an Internet of Things service data management method driven by end-edge-cloud collaboration. Background Art

[0002] With the explosive growth of Internet of Things (IoT) devices, the amount of data generated by these devices is increasing exponentially. Traditional centralized cloud computing architectures face bottlenecks such as high transmission latency, high bandwidth costs, and insufficient real-time response capabilities. The introduction of edge computing enables local data processing by moving computing resources to the edge of the network. However, a single edge node is limited by resource constraints and cannot support complex, multi-dimensional data analysis needs. Against this backdrop, the device-edge-cloud collaborative architecture has gradually become the core paradigm for IoT service data management. By synergizing device-side data collection, edge preprocessing, and in-depth cloud analysis, it builds a layered, intelligent data processing chain.

[0003] Traditional architectures rely on pre-defined, fixed channels (such as MQTT) for end-to-cloud communication, making it difficult to dynamically adapt to heterogeneous device protocols and diverse analysis needs. Furthermore, cloud databases often use centralized storage architectures, lacking unified support for real-time streaming data, historical batch data, and interactive queries. This leads to data silos in multi-channel analysis scenarios. Summary of the Invention

[0004] To address the above technical issues, the present invention adopts a method for managing IoT service data driven by device-edge-cloud collaboration, which includes the following steps:

[0005] S01. Configure data preprocessing rules and analysis strategies on the cloud management platform, and publish the data management strategy package to the cloud strategy library;

[0006] S02. The cloud management platform dynamically sends the data management policy package to the designated edge node based on the device type and edge node load status;

[0007] S03, the edge node establishes a secure channel with the cloud management platform and synchronizes metadata indexes and pre-processing rules in the cloud policy library;

[0008] S04. The edge node collects raw data from the end-side device in real time, performs lightweight cleaning, compression, and feature extraction according to preprocessing rules, and generates a structured data stream;

[0009] S05. The cloud management platform receives the structured data stream uploaded by the edge node, performs multi-dimensional fusion analysis and persistent storage, and feeds back optimization strategies to the edge node.

[0010] Preferably, executing step S01 further includes training a data classification model based on historical IoT service scenarios, and encapsulating the model and preprocessing rules into a configurable data management policy package.

[0011] Preferably, the training data classification model includes: collecting multimodal raw data from the end-side device, labeling the data value weight labels, and iteratively training the classification model through a federated learning framework.

[0012] Preferably, the step S02 of dynamically sending the data management policy package to the designated edge node includes:

[0013] S21, the cloud management platform monitors the computing resource margin and network status of the edge node;

[0014] S22: Select a target edge node and push a differentiated policy package based on the data priority of the terminal device and the real-time load of the edge node.

[0015] Preferably, the establishing of the secure channel in step S03 includes:

[0016] The edge node is bidirectionally authenticated with the cloud management platform through the device identity certificate. After the authentication is passed, the metadata is synchronized incrementally according to the policy version number.

[0017] Preferably, the step S04 performs lightweight cleaning, compression, and feature extraction according to preprocessing rules to generate a structured data stream, including:

[0018] S44. The edge node divides the original data stream of the end-side device into time windows;

[0019] S42, using a lightweight model to perform denoising, normalization, and key feature extraction on the data in the window to generate a low-dimensional feature vector;

[0020] S43: Encapsulate the low-dimensional feature vector into a time series data packet and upload it to the cloud.

[0021] Preferably, the time window segmentation in step S41 includes:

[0022] Adaptively expand or shrink the window length based on the data generation rate of the end-side device and network latency to ensure edge processing latency.

[0023] Preferably, the step S05 of performing multi-dimensional fusion analysis and persistent storage, and feeding back the optimization strategy to the edge node, includes:

[0024] S51, the cloud management platform aggregates the multi-edge node data streams, constructs a spatiotemporal data cube, then integrates the third-party business system data, analyzes service dependencies through a graph computing engine, and generates a global data topology map;

[0025] S52. Calling a cloud-based analysis model to perform cross-dimensional data mining and generate a service status assessment report;

[0026] S53: triggering a service optimization instruction according to the evaluation report and sending it to the corresponding edge node.

[0027] The present invention has at least the following beneficial effects:

[0028] 1. Edge nodes perform lightweight cleaning, compression, and feature extraction on the original data on the end side, uploading only low-dimensional feature vectors, reducing the amount of uplink data by more than 60%. Combined with adaptive time window segmentation, it reduces the timeliness of edge processing latency, effectively avoiding the high latency and bandwidth bottleneck of direct transmission of original data to the cloud in traditional architectures.

[0029] 2. The cloud dynamically sends differentiated strategy packages to edge nodes and allocates computing tasks based on real-time load status. The edge layer only performs lightweight preprocessing, and complex analysis is transferred to the cloud to achieve the optimal match between resources and tasks, avoiding data processing congestion caused by insufficient computing power at the edge nodes.

[0030] 3. The cloud aggregates structured data streams from multiple edge nodes, builds a data cube with temporal and spatial correlation, and integrates third-party business data to generate a global topology map; calls on high-performance cloud models to perform cross-domain mining, solving complex correlation analysis needs that are difficult to support with traditional edge computing.

[0031] 4. The edge node and the cloud are bidirectionally authenticated through device identity certificates, and metadata is incrementally synchronized and encrypted. In the preprocessing stage, a hierarchical model trained with federated learning is used, so that the original data does not leave the edge domain, improving the security of sensitive IoT data in the three aspects of transmission, storage, and processing.

[0032] 5. The cloud generates a service evaluation report based on the global analysis results and dynamically issues optimization instructions to the edge nodes. Combined with the federated learning framework, the data classification model is iteratively updated to form a "collection-analysis-optimization" closed loop, continuously improving the accuracy and efficiency of IoT service responses. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 This is a flow chart of a method for managing IoT service data driven by end-edge-cloud collaboration, provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Example 1

[0038] This embodiment provides a method for managing IoT service data driven by device-edge-cloud collaboration, which includes the following steps: Figure 1 As shown:

[0039] S01. Configure data preprocessing rules and analysis strategies on the cloud management platform, and publish the data management strategy package to the cloud strategy library;

[0040] Specifically, executing step S01 also includes training a data classification model based on historical IoT service scenarios, and encapsulating the model and preprocessing rules into a configurable data management policy package.

[0041] Furthermore, training the data classification model includes: collecting multimodal raw data from the end-side device, labeling the data value weight labels, and iteratively training the classification model through the federated learning framework.

[0042] S02. The cloud management platform dynamically sends data management policy packages to designated edge nodes based on the device type and edge node load status.

[0043] The above-mentioned dynamic delivery of data management policy packages to designated edge nodes includes:

[0044] S21. The cloud management platform monitors the computing resource margin and network status of the edge node;

[0045] S22. Select the target edge node and push the differentiated policy package based on the data priority of the end-side device and the real-time load of the edge node.

[0046] S03. The edge node establishes a secure channel with the cloud management platform and synchronizes metadata indexes and pre-processing rules in the cloud policy library;

[0047] Specifically, the establishment of the above-mentioned secure channel is based on two-way authentication between the edge node and the cloud management platform through the device identity certificate. After the authentication is passed, the metadata is synchronized incrementally according to the policy version number.

[0048] S04. The edge node collects raw data from the end-side devices in real time, performs lightweight cleaning, compression, and feature extraction according to preprocessing rules, and generates a structured data stream.

[0049] The steps of performing lightweight cleaning, compression, and feature extraction according to the preprocessing rules to generate a structured data stream include:

[0050] S44. The edge node divides the original data stream of the end-side device into time windows. The time window division includes: adaptively expanding or contracting the window length according to the data generation rate of the end-side device and the network delay to ensure the edge processing delay;

[0051] S42, using a lightweight model to perform denoising, normalization, and key feature extraction on the data in the window to generate a low-dimensional feature vector;

[0052] S43. Encapsulate the low-dimensional feature vector into a time series data packet and upload it to the cloud.

[0053] S05. The cloud management platform receives the structured data stream uploaded by the edge node, performs multi-dimensional fusion analysis and persistent storage, and feeds back the optimization strategy to the edge node.

[0054] The steps of performing multi-dimensional fusion analysis and persistent storage, and feeding back optimization strategies to edge nodes, include:

[0055] S51. The cloud management platform aggregates data streams from multiple edge nodes, constructs a spatiotemporal data cube, then integrates data from third-party business systems, analyzes service dependencies through a graph computing engine, and generates a global data topology map.

[0056] S52. Calling a cloud-based analysis model to perform cross-dimensional data mining and generate a service status assessment report;

[0057] S53. Trigger a service optimization instruction based on the evaluation report and send it to the corresponding edge node.

[0058] This first embodiment uses edge nodes to perform lightweight cleaning, compression, and feature extraction on raw data on the device side, uploading only low-dimensional feature vectors and reducing uplink data volume by over 60%. Combined with adaptive time window segmentation, this reduces edge processing latency, effectively avoiding the high latency and bandwidth bottlenecks of direct raw data transmission to the cloud in traditional architectures. Furthermore, the cloud dynamically sends differentiated policy packages to edge nodes, allocating computing tasks based on real-time load status. The edge layer performs only lightweight preprocessing, with complex analysis offloaded to the cloud. This optimizes resource and task matching and avoids data processing bottlenecks caused by insufficient computing power at the edge node. Secondly, the cloud aggregates structured data streams from multiple edge nodes, constructs a spatiotemporally correlated data cube, and integrates third-party business data to generate a global topology map. High-performance cloud-based models are used to perform cross-domain mining, addressing complex correlation analysis requirements that traditional edge computing struggles to support. Furthermore, edge nodes and the cloud utilize bidirectional authentication using device identity certificates and incrementally synchronize encrypted metadata. A hierarchical model trained using federated learning is used in the preprocessing phase, ensuring that raw data never leaves the edge domain. This improves the security of sensitive IoT data across transmission, storage, and processing. Furthermore, the cloud generates a service evaluation report based on the global analysis results and dynamically issues optimization instructions to the edge nodes; combined with the federated learning framework, it iteratively updates the data classification model to form a "collection-analysis-optimization" closed loop, continuously improving the accuracy and efficiency of IoT service responses.

[0059] Example 2

[0060] An embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the steps:

[0061] Configure data preprocessing rules and analysis strategies on the cloud management platform, and publish the data management strategy package to the cloud strategy library;

[0062] The cloud management platform dynamically sends data management policy packages to designated edge nodes based on the device type and edge node load status.

[0063] The edge node establishes a secure channel with the cloud management platform to synchronize metadata indexes and pre-processing rules in the cloud policy library;

[0064] Edge nodes collect raw data from end-side devices in real time, perform lightweight cleaning, compression, and feature extraction according to preprocessing rules, and generate structured data streams.

[0065] The cloud management platform receives structured data streams uploaded by edge nodes, performs multi-dimensional fusion analysis and persistent storage, and feeds back optimization strategies to edge nodes.

[0066] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware using a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0067] Those skilled in the art will clearly understand that for the sake of convenience and brevity in description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0068] Example 3

[0069] An embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the following steps:

[0070] Configure data preprocessing rules and analysis strategies on the cloud management platform, and publish the data management strategy package to the cloud strategy library;

[0071] The cloud management platform dynamically sends data management policy packages to designated edge nodes based on the device type and edge node load status.

[0072] The edge node establishes a secure channel with the cloud management platform to synchronize metadata indexes and pre-processing rules in the cloud policy library;

[0073] Edge nodes collect raw data from end-side devices in real time, perform lightweight cleaning, compression, and feature extraction according to preprocessing rules, and generate structured data streams.

[0074] The cloud management platform receives structured data streams uploaded by edge nodes, performs multi-dimensional fusion analysis and persistent storage, and feeds back optimization strategies to edge nodes.

[0075] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the present profession can make some changes or modifications to equivalent embodiments of the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for managing IoT service data driven by device-edge-cloud collaboration, characterized in that: The method comprises the following steps: S01. Configure data preprocessing rules and analysis strategies on the cloud management platform, and publish the data management strategy package to the cloud strategy library; S02. The cloud management platform dynamically sends the data management policy package to the designated edge node based on the device type and edge node load status; S03, the edge node establishes a secure channel with the cloud management platform and synchronizes metadata indexes and pre-processing rules in the cloud policy library; S04. The edge node collects raw data from the end-side device in real time, performs lightweight cleaning, compression, and feature extraction according to preprocessing rules, and generates a structured data stream; S05. The cloud management platform receives the structured data stream uploaded by the edge node, performs multi-dimensional fusion analysis and persistent storage, and feeds back optimization strategies to the edge node.

2. The method for managing IoT service data driven by device-edge-cloud collaboration according to claim 1, characterized in that: Executing the step S01 also includes training a data classification model based on historical IoT service scenarios, and encapsulating the model and preprocessing rules into a configurable data management strategy package.

3. The method for managing IoT service data driven by device-edge-cloud collaboration according to claim 2, characterized in that: The training data classification model includes: collecting multimodal raw data from end-side devices, labeling data value weight labels, and iteratively training the classification model through a federated learning framework.

4. The method for managing IoT service data driven by device-edge-cloud collaboration according to claim 1, characterized in that: The step S02 of dynamically sending the data management policy package to the designated edge node includes: S21, the cloud management platform monitors the computing resource margin and network status of the edge node; S22: Select a target edge node and push a differentiated policy package based on the data priority of the terminal device and the real-time load of the edge node.

5. The method for managing IoT service data driven by device-edge-cloud collaboration according to claim 1, characterized in that: The step S03 of establishing a secure channel includes: The edge node is bidirectionally authenticated with the cloud management platform through the device identity certificate. After the authentication is passed, the metadata is synchronized incrementally according to the policy version number.

6. The method for managing IoT service data driven by device-edge-cloud collaboration according to claim 1, characterized in that: The step S04 performs lightweight cleaning, compression, and feature extraction according to preprocessing rules to generate a structured data stream, including: S44. The edge node divides the original data stream of the end-side device into time windows; S42, using a lightweight model to perform denoising, normalization, and key feature extraction on the data in the window to generate a low-dimensional feature vector; S43: Encapsulate the low-dimensional feature vector into a time series data packet and upload it to the cloud.

7. The method for managing IoT service data driven by device-edge-cloud collaboration according to claim 6, characterized in that: The time window segmentation in step S41 includes: Adaptively expand or shrink the window length based on the data generation rate of the end-side device and network latency to ensure edge processing latency.

8. The method for managing IoT service data driven by device-edge-cloud collaboration according to claim 1, characterized in that: The multi-dimensional fusion analysis and persistent storage performed in step S05, and the optimization strategy fed back to the edge node, include: S51, the cloud management platform aggregates the multi-edge node data streams, constructs a spatiotemporal data cube, then integrates the third-party business system data, analyzes service dependencies through a graph computing engine, and generates a global data topology map; S52: Calling the cloud analysis model to perform cross-dimensional data mining and generate a service status assessment report; S53: triggering a service optimization instruction according to the evaluation report and sending it to the corresponding edge node.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the end-edge-cloud collaboratively driven IoT service data management method as described in any one of claims 1-8.

10. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the end-edge-cloud collaboratively driven Internet of Things service data management method as described in any one of claims 1-8.

Images