Edge cloud cooperative data processing method and device, and electronic equipment

Abstract

The application provides a kind of edge cloud coordination data processing method, device and electronic equipment, method includes: obtaining to be processed edge data;Based on edge cloud resource state, edge cloud network state, and the task information of to be processed edge data is evaluated, and the task unloading strategy of to be processed edge data is obtained;According to task unloading strategy, to be processed edge data is handled.The method provided by the application obtains the task unloading strategy by introducing the comprehensive evaluation mechanism of edge cloud resource state, edge cloud network state and to be processed task information, to realize the best processing path in data source head dynamic decision, not only significantly reduce the network bandwidth consumption and transmission delay brought by mass original data directly uploaded to cloud, meet the demand of millisecond level response to industrial scene, and by reasonable allocation of tasks, make full use of idle edge computing resources, avoid cloud overload, to improve the processing efficiency and resource utilization of edge cloud coordination system for data processing.

Description

Technical Field

[0001] This invention relates to the field of data processing technology for the Industrial Internet of Things (IIoT), and in particular to a data processing method, apparatus, and electronic device for edge-cloud collaboration. Background Technology

[0002] Current mainstream data processing solutions typically employ a cloud-centric architecture. In this architecture, edge devices deployed in the physical environment primarily handle data acquisition and forwarding. The collected raw data is directly encapsulated and transmitted to a remote cloud platform or data center via mobile communication networks or wired networks. The cloud platform, acting as the core control layer, is centrally responsible for data reception, storage, cleaning, model inference, and analysis and decision-making, and feeds back the results to the terminal after processing.

[0003] However, adopting a fixed cloud-based centralized processing model results in high bandwidth consumption, high response latency, and an inability to achieve effective collaboration and balanced utilization of edge and cloud resources. Summary of the Invention

[0004] This invention provides a data processing method, apparatus, and electronic device for edge-cloud collaboration, which solves the defects of the existing technology that adopts a fixed cloud-centralized processing mode, resulting in high transmission bandwidth consumption, high response latency, and inability to achieve effective collaboration and balanced utilization of edge and cloud resources.

[0005] This invention provides a data processing method for edge-cloud collaboration, comprising: Acquire edge data to be processed; An evaluation is conducted based on the edge cloud resource status, edge cloud network status, and the pending task information of the edge data to be processed, to obtain the task offloading strategy for the edge data to be processed. The edge data to be processed is processed according to the task unloading strategy.

[0006] According to a data processing method for edge-cloud collaboration provided by the present invention, the step of evaluating the edge data to be processed based on the edge-cloud resource status, edge-cloud network status, and the task information to be processed of the edge data to obtain a task offloading strategy for the edge data to be processed includes: Construct a comprehensive cost function; the comprehensive cost function includes a resource overhead function, a network transmission performance function, and a service quality function; The edge cloud resource status, the edge cloud network status, and the task information to be processed are used as inputs to the comprehensive cost function. The comprehensive cost function is solved to obtain the optimal solution of the comprehensive cost. The optimal solution includes the task execution subject, data transmission parameters, and the target task data that the task execution subject needs to execute. Based on the optimal solution, the task unloading strategy is determined.

[0007] According to the edge-cloud collaborative data processing method provided by the present invention, the step of processing the edge data to be processed according to the task offloading strategy includes: When the task execution entity includes the cloud, the target task data to be executed in the cloud is transmitted to the cloud according to the data transmission parameters; Based on the cloud task processing model, data processing is performed on the target task data to be executed in the cloud. The data transmission parameters include the data transmission frequency.

[0008] According to the edge-cloud collaborative data processing method provided by the present invention, the step of processing the edge data to be processed according to the task offloading strategy further includes: When the task execution entity includes an edge node, data processing is performed on the target task data to be executed by the edge node based on the local task processing model; The local task processing model is deployed on the edge node.

[0009] According to the edge-cloud collaborative data processing method provided by the present invention, the step of processing the target task data to be executed in the cloud based on the cloud task processing model includes: The edge processing results obtained from the edge nodes are transmitted to the cloud. Based on the cloud task processing model, data processing is performed on the target task data to be executed in the cloud and the edge processing results.

[0010] According to the data processing method for edge-cloud collaboration provided by the present invention, the method further includes: Obtain historical edge data; Based on the historical edge data, the cloud task processing model is optimized to obtain optimized model parameters; The optimized model parameters are sent to the edge nodes to update the model parameters of the local task processing model.

[0011] According to a data processing method for edge-cloud collaboration provided by the present invention, the information of the task to be processed includes at least one of task priority, real-time requirements, and data volume. The steps for obtaining the tasks to be processed include: Acquire edge data collected by edge nodes; Based on the anomaly detection model, the edge data is detected to obtain the detection results; If the detection result is abnormal, the edge data is treated as edge data to be processed; the anomaly detection model is deployed on the edge node.

[0012] According to the data processing method for edge-cloud collaboration provided by the present invention, the edge-cloud network status includes network quality parameters and bandwidth utilization.

[0013] The present invention also provides a data processing device for edge-cloud collaboration, comprising: The acquisition unit acquires the edge data to be processed; The evaluation unit evaluates the edge cloud resource status, edge cloud network status, and pending task information of the edge data to be processed, and obtains the task offloading strategy for the edge data to be processed. The processing unit processes the edge data to be processed according to the task unloading strategy.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the edge-cloud collaborative data processing method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the edge-cloud collaborative data processing method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the edge-cloud collaborative data processing method as described above.

[0017] The edge-cloud collaborative data processing method, apparatus, and electronic device provided by this invention introduce a comprehensive evaluation mechanism for edge-cloud resource status, edge-cloud network status, and information on tasks to be processed, thereby obtaining a task offloading strategy. This enables dynamic decision-making at the source of data generation to determine the optimal processing path. This not only significantly reduces network bandwidth consumption and transmission latency caused by directly uploading massive amounts of raw data to the cloud, meeting the millisecond-level response requirements of industrial scenarios, but also makes full use of idle edge computing resources by rationally allocating tasks, avoiding cloud overload, thereby improving the processing efficiency and resource utilization of the edge-cloud collaborative system for data processing. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1This is a flowchart illustrating the edge-cloud collaborative data processing method provided by the present invention; Figure 2 This is a flowchart illustrating the comprehensive evaluation method provided by the present invention; Figure 3 This is a schematic diagram of the edge-cloud collaborative data processing device provided by the present invention; Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] To address the aforementioned issues, this invention provides an edge-cloud collaborative data processing method to achieve flexible scheduling and efficient collaboration of data between the edge and the cloud. It should be noted that this method can be applied to collaborative systems comprised of edge and node components and cloud platforms in data-intensive scenarios such as the Industrial Internet of Things and smart cities. Figure 1 This is a flowchart illustrating the edge-cloud collaborative data processing method provided by the present invention, as shown below. Figure 1 As shown, the method specifically includes: Step 110: Obtain the edge data to be processed.

[0022] Here, edge data to be processed refers to raw, multi-source, heterogeneous data generated by terminal devices such as sensors, cameras, and controllers deployed in the physical field that need to be calculated or analyzed. For example, it could be data generated by terminal devices in an industrial IoT environment.

[0023] Understandably, in real-world applications, these data are characterized by high concurrency, strong real-time performance, and diverse formats, including both structured time-series data such as temperature, humidity, pressure, voltage, and current, and unstructured data such as surveillance videos, audio streams, or images.

[0024] Specifically, firstly, connections are established with field terminal devices by adapting to various industrial communication protocols. These industrial communication protocols include Modbus, OPC UA, and PLC-specific protocols. Secondly, raw data streams from different terminals are uniformly acquired at a high frequency.

[0025] Then, in order to ensure the effectiveness of subsequent processing, the raw data collected can be preliminarily cleaned and time-series aligned during the acquisition of edge data to be processed. Obvious erroneous data caused by equipment failure, such as null values ​​and garbled characters, can be removed. Data that has been denoised or formatted can be identified as edge data to be processed, providing a reliable data foundation for subsequent accurate evaluation.

[0026] Step 120: Based on the edge cloud resource status, edge cloud network status, and the pending task information of the edge data to be processed, an evaluation is performed to obtain the task offloading strategy for the edge data to be processed.

[0027] Here, edge-cloud resource status refers to the real-time load of the computing and storage capabilities of the edge node itself and the cloud service platform, such as CPU utilization, memory usage, remaining storage space, and task queue length.

[0028] Here, edge-cloud network status refers to the real-time transmission quality of the communication link connecting edge nodes and the cloud platform, including but not limited to available bandwidth, network latency, packet loss rate, and signal strength.

[0029] In addition, the pending task information here refers to the attribute characteristics of the computing tasks that need to be performed on the specific edge data to be processed, such as the data volume, task priority, real-time response requirements, and estimated computing power required.

[0030] Here, the task offloading strategy refers to the decision result generated after comprehensive evaluation, which clearly indicates whether the data should be processed directly on the local edge node, offloaded to the cloud platform for processing, or processed in a collaborative manner between the edge and the cloud.

[0031] Specifically, after acquiring the edge data to be processed, resource load data between edge nodes and the cloud can be collected in real time through monitoring. Simultaneously, the network environment between edge nodes and the cloud can be perceived through heartbeat detection or probe mechanisms. Furthermore, the processing task information for the edge data can be obtained. For example, if the edge data to be processed is video frames related to fire alarms, it has extremely high real-time requirements and priority; if the edge data to be processed is historical archived temperature and humidity records, the real-time requirements are lower.

[0032] Then, a comprehensive evaluation is conducted based on edge cloud resource status, edge cloud network status, and information on tasks to be processed to derive a task offloading strategy for edge data. For example, during the evaluation process, if a poor edge cloud network status is detected, such as bandwidth congestion or high latency, and the information on tasks to be processed indicates that the data has extremely high real-time requirements, even if the edge cloud resource status shows high local load, a task offloading strategy for local processing may be generated to avoid uncontrollable delays caused by network transmission. Conversely, if the amount of edge data to be processed is huge, such as complex large-scale model inference, and local computing resources are insufficient, but the network status is good, the evaluation result will tend to generate an offloading strategy for uploading to the cloud. Understandably, this multi-dimensional dynamic evaluation breaks the rigid pattern of traditional fixed-rule uploading or processing.

[0033] Step 130: Process the edge data to be processed according to the task unloading strategy.

[0034] Specifically, based on the generated task unloading strategy, corresponding data flow and computation operations are executed. More specifically, if the strategy indicates processing at the edge, the local algorithm model or rule engine deployed on the edge node is invoked to perform real-time analysis of the edge data to be processed, such as anomaly detection and feature extraction, and directly outputs control commands or results without uploading large amounts of raw data, thus achieving rapid response in data processing.

[0035] If the policy instruction is to be processed in the cloud, the communication module of the edge node is used to encapsulate the edge data to be processed through protocols such as MQTT (Message Queuing Telemetry Transport) and HTTP / 2 (Hypertext Transfer Protocol Version 2) and send it to the cloud. The powerful computing power of the cloud is used for deep mining or model inference, and the results are fed back to the edge side or business system.

[0036] The method provided in this invention introduces a comprehensive evaluation mechanism for edge-cloud resource status, edge-cloud network status, and information on tasks to be processed, to obtain a task offloading strategy for edge data to be processed. This enables dynamic decision-making at the source of data generation to determine the optimal processing path. This not only significantly reduces network bandwidth consumption and transmission latency caused by directly uploading massive amounts of raw data to the cloud, meeting the millisecond-level response requirements of industrial scenarios, but also makes full use of idle edge computing resources by rationally allocating tasks, avoiding overload of the cloud center. This improves the processing efficiency and resource utilization of the edge-cloud collaborative system for data processing.

[0037] In the complex environment of edge-cloud collaboration, in order to accurately quantify and evaluate various indicators and make optimal decisions, based on any of the above embodiments, Figure 2 This is a flowchart illustrating the comprehensive evaluation method provided by the present invention, as shown below. Figure 2 As shown, the method is as follows: Step 210: Construct a comprehensive cost function; the comprehensive cost function includes a resource overhead function, a network transmission performance function, and a service quality function.

[0038] Here, the comprehensive cost function is a set of mathematical models or logical rules used to quantify the total cost or expense required to execute the task to be processed. It can consist of three core sub-functions: a resource overhead function, a network transmission performance function, and a service quality function.

[0039] The resource overhead function characterizes the computing resources, storage resources, and corresponding energy costs consumed in executing the task. The network transmission performance function characterizes the time latency and bandwidth usage costs incurred in transmitting data between the edge and the cloud. The service quality function characterizes service quality indicators such as the real-time performance and accuracy of the task processing results, i.e., the penalty cost incurred if excessive processing latency leads to service failure.

[0040] Specifically, a weighted summation approach can be used when constructing the comprehensive cost function. For example, the resource overhead function can be given a higher weight for battery-powered edge devices, while the service quality function can be given a higher weight for scenarios such as autonomous driving or industrial control. In this way, the multi-dimensional physical state is transformed into a unified mathematical expression, enabling the system to measure the merits of different processing methods from a global perspective.

[0041] Step 220: The edge cloud resource status, the edge cloud network status, and the task information to be processed are used as inputs to the comprehensive cost function. The comprehensive cost function is solved to obtain the optimal solution of the comprehensive cost.

[0042] The optimal solution includes the task execution entity, data transmission parameters, and the target task data that the task execution entity needs to execute.

[0043] Here, determining the optimal solution clarifies the task execution entity, data transmission parameters, and the target task data that the task execution entity needs to execute. Specifically, defining the task execution entity involves determining whether it should be executed solely by the edge node, solely by the cloud, or collaboratively by both. Furthermore, determining the data transmission parameters specifies the appropriate compression ratio, transmission frequency, or priority for transmission. Finally, determining the target task data that the task execution entity needs to execute clarifies which specific data segments will be assigned to that entity for processing. For example, the solution might indicate: "The edge node will handle the task of processing the face image target data, while the cloud will handle the task of processing the environmental data target data."

[0044] Specifically, after obtaining the real-time edge cloud resource status, edge cloud network status, and information on tasks to be processed, these parameters are substituted into the aforementioned comprehensive cost function. The comprehensive cost function is then solved using optimization algorithms, such as gradient descent, genetic algorithms, or rule-based fast traversal, to find the configuration scheme that minimizes the total cost, i.e., the optimal solution.

[0045] Step 230: Based on the optimal solution, determine the task unloading strategy.

[0046] Specifically, the mathematically optimal solution is mapped to system-executable control instructions, i.e., task offloading strategies, to guide subsequent data diversion and task scheduling.

[0047] The method provided in this invention transforms the complex edge-cloud collaborative scheduling problem into a quantifiable mathematical optimization problem by constructing a comprehensive cost function and solving for the optimal solution. It not only considers physical constraints such as resources and networks but also takes into account application layer requirements through a service quality function. Furthermore, the strategy obtained through mathematical solution can minimize system energy consumption and bandwidth usage while ensuring service real-time performance and accuracy, achieving global optimization of edge-cloud resource allocation.

[0048] Based on any of the above embodiments, step 130 includes: When the task execution entity includes the cloud, the target task data to be executed in the cloud is transmitted to the cloud according to the data transmission parameters; Based on the cloud task processing model, data processing is performed on the target task data to be executed in the cloud. The data transmission parameters include the data transmission frequency.

[0049] Here, data transmission parameters are transmission control indicators determined based on comprehensive evaluation, including at least one of data compression ratio, data transmission frequency, and transmission priority. The data transmission frequency specifically refers to the time interval or trigger frequency of data reporting. It is understandable that setting data transmission parameters is crucial for balancing real-time performance and network load. Additionally, the target task data to be executed by the cloud refers to the portion of data specified in the policy that must be processed by the cloud.

[0050] Specifically, when the task offloading policy indicates that the task execution entity includes the cloud, the edge device will not blindly upload all the target task data from the cloud. Instead, it will transmit data in a controlled manner according to the data transmission frequency specified in the policy. For example, during periods of limited network bandwidth or non-critical periods, the policy may require reducing the data transmission frequency from real-time streaming to batch transmission once per minute. Alternatively, when abnormal waveforms are detected, the transmission frequency may be dynamically increased to ensure that the cloud can obtain high-precision detailed data. During data transmission, the edge device can send the filtered target task data to the cloud platform via protocols such as MQTT or HTTP / 2, based on the data transmission parameters.

[0051] Then, after receiving the target task data, the cloud platform inputs it into the cloud task processing model. This model can perform complex calculations, such as trend prediction across time periods, correlation analysis of multi-source data, or high-precision image recognition. After processing, the cloud can feed back the analysis results, optimized model parameters, or control commands to the edge. Here, the cloud task processing model refers to a deep learning model or big data analysis algorithm deployed on a cloud server, which has high computational complexity and relies on massive amounts of historical data for training.

[0052] The method provided in this invention avoids the traffic storms and network congestion caused by the coarse data uploads in traditional solutions by introducing fine-grained control of data transmission parameters such as data transmission frequency during the transmission process. Simultaneously, by combining the deep analysis capabilities of cloud-based task processing models, it achieves centralized processing of high-value, highly complex data, leveraging the advantages of the cloud in global data mining and model training, and compensating for the insufficient computing power on the edge.

[0053] Based on any of the above embodiments, step 130 includes: When the task execution entity includes an edge node, data processing is performed on the target task data to be executed by the edge node based on the local task processing model; The local task processing model is deployed on the edge node.

[0054] Here, the local task processing model refers to an algorithm model or rule engine that is lightweight, adapted to edge hardware architecture, and deployed locally on the edge node. It typically focuses on low-latency inference and fast feature extraction, rather than large-scale training. The target task data that the edge node needs to execute refers to the portion of data allocated to local processing by the policy; this is usually data requiring extremely high real-time performance or data involving privacy that cannot be uploaded.

[0055] Specifically, when the task offloading strategy instructs the edge node to handle the task, the edge device directly invokes a local task processing model stored in its local memory. This model can be a pre-built anomaly detection algorithm, a lightweight neural network, or a threshold-based rule base. Understandably, the edge node can utilize this model to perform millisecond-level real-time computations on the target task data. For example, this could be used to directly identify defective products on a conveyor belt in an industrial setting and control a robotic arm to remove them, or to perform privacy masking and keyframe extraction on video data.

[0056] It should be noted that since the local task processing model is deployed on the edge node, the data does not need to be transmitted through the wide area network, thus ensuring extremely low latency and business continuity. Even if the external network is disconnected, the edge node can still independently complete critical control and monitoring tasks.

[0057] The method provided in this invention significantly improves the real-time response speed of the system by deploying a local task processing model and processing data locally at edge nodes under policy guidance, meeting the stringent latency requirements of scenarios such as industrial control. Simultaneously, this approach effectively reduces the computational load on the cloud and enhances the security of user privacy data by reducing the transmission of raw data, truly realizing the edge computing concept of data generation and processing at the source.

[0058] Based on any of the above embodiments, and based on the cloud task processing model, data processing is performed on the target task data to be executed in the cloud, including: The edge processing results obtained from the edge nodes are transmitted to the cloud. Based on the cloud task processing model, data processing is performed on the target task data to be executed in the cloud and the edge processing results.

[0059] Here, edge processing results refer to the information output by edge nodes after preliminary calculations of the edge data to be processed based on their local task processing models. It is understood that edge processing results differ from the original edge data to be processed. Edge processing results are typically highly abstracted and compressed data, including but not limited to extracted key feature vectors, such as facial feature values; they may also include preliminary event recognition labels, such as "abnormal vibration detected," or denoised statistical indicators or structured metadata.

[0060] Specifically, after the edge node completes the inference or computation of the local model, the resulting edge processing results can be serialized and encapsulated. Since the amount of this data is much smaller than the original data, the edge node can quickly synchronize it to the cloud using lightweight protocols such as MQTT or CoAP through a low-bandwidth control channel. It should be noted that this transmission process can be carried out in parallel with the transmission of the original target task data, or it can be sent first as metadata so that the cloud system can perceive the on-site status in advance.

[0061] Then, after receiving the edge processing results uploaded by the edge nodes and the target task data to be executed in the cloud according to the transmission strategy, the cloud platform simultaneously inputs both data into the cloud task processing model. For example, in an industrial vision inspection scenario, the edge nodes first analyze the video stream to generate "edge processing results" containing defect location coordinates and preliminary classification, and upload high-resolution screenshots of suspected defects as "target task data." The cloud model uses the coordinate information provided by the edge nodes to quickly locate the area of ​​interest and performs pixel-level in-depth defect analysis in conjunction with the high-resolution screenshots. Alternatively, in a predictive maintenance scenario, the cloud model combines the real-time alarm signals uploaded by the edge nodes as edge processing results with detailed waveform data for that time period as target task data to perform root cause analysis of faults.

[0062] Therefore, in this way, the cloud no longer processes data from scratch, but rather performs secondary confirmation or in-depth mining based on the initial screening or prediction results from the edge.

[0063] The method provided in this invention constructs an efficient edge-cloud fusion computing model by transmitting the edge processing results obtained from the edge nodes to the cloud and combining them with target task data for joint processing. It utilizes the computing results from the edge side to provide prior knowledge or attention guidance for the cloud model, avoiding blind searching of massive amounts of data by the cloud model and significantly improving the efficiency and accuracy of cloud analysis. Simultaneously, this cascaded processing architecture enables the system to balance agile response on the edge side with global cognition on the cloud side, achieving cross-level intelligent collaboration.

[0064] Based on any of the above embodiments, the method further includes: Obtain historical edge data; Based on the historical edge data, the cloud task processing model is optimized to obtain optimized model parameters; The optimized model parameters are sent to the edge nodes to update the model parameters of the local task processing model.

[0065] Here, historical edge data refers to the data set collected by various edge nodes over a period of time and ultimately aggregated and stored in the cloud database. This data includes raw sensor data and corresponding annotation information or processing results. Understandably, through long-term accumulation, this data covers a variety of operating scenarios.

[0066] Specifically, this data can be periodically extracted from distributed file systems or time-series databases using a cloud platform's big data batch processing engine. To improve the efficiency of subsequent processing, the acquisition process typically involves data deduplication, cleaning, and sample equalization operations tailored to specific business scenarios to ensure the quality and representativeness of the training data.

[0067] Then, leveraging the powerful computing resources of the cloud, the cloud-based task processing model can be iteratively trained using historical edge data, enabling it to identify new failure modes or adapt to new environmental characteristics. Subsequently, to allow the model to run on resource-constrained edge nodes, the cloud will perform lightweight processing on the model, such as using model pruning, parameter quantization, or knowledge distillation techniques, to generate a lightweight version adapted to the edge hardware architecture, thereby obtaining deployable optimized model parameters.

[0068] Here, model optimization refers to the process of retraining, fine-tuning, or structurally compressing a deep learning model using massive amounts of historical data. Furthermore, the optimized model parameters here refer to the neural network weights, bias values, or rule thresholds obtained after the model optimization process, which reflect the latest business characteristics.

[0069] Finally, the optimized model parameters can be sent to designated edge nodes via the control channel between the cloud and the edge, using incremental updates or full replacements. Upon receiving the parameter packets, the edge nodes can replace the parameters of the old local task processing model using a hot-reloading mechanism without interrupting current business operations.

[0070] The method provided in this invention solves the problem that edge nodes are unable to perform large-scale model training due to limited computing power, leading to model aging and failure, by establishing a closed-loop mechanism of cloud-based training optimization and edge inference updates. It utilizes historical edge data in the cloud for continuous learning and distributes optimized model parameters, enabling the local task processing model on the edge side to continuously evolve and adapt, thereby improving processing accuracy and generalization ability over time.

[0071] It should be noted that in massive data scenarios, not all collected data needs to enter the complex edge-cloud collaborative scheduling process. Pre-screening for anomalies can significantly reduce system load. Based on any of the above embodiments, the task information to be processed includes at least one of task priority, real-time requirements, and data volume. The steps for obtaining the tasks to be processed include: Acquire edge data collected by edge nodes; Based on the anomaly detection model, the edge data is detected to obtain the detection results; If the detection result is abnormal, the edge data is treated as edge data to be processed; the anomaly detection model is deployed on the edge node.

[0072] Here, the information for the task to be processed includes at least one of the following: task priority, real-time requirements, and data volume. Task priority can be higher for emergency alarm tasks than for regular log tasks. Real-time requirements can be millisecond-level requirements for vehicle obstacle avoidance and minute-level requirements for environmental monitoring. Data volume can be large for high-definition video streams and small for temperature values.

[0073] Additionally, here, edge data refers to the raw, full data stream that edge nodes read in real time from terminal devices such as sensors.

[0074] Specifically, the edge nodes immediately input the edge data they collect into the anomaly detection model the moment it arrives. It should be noted that the anomaly detection model does not perform complex semantic analysis; it only determines whether the data deviates from the normal waveform, exceeds a safety threshold, or exhibits abrupt changes, thereby quickly outputting detection results labeled as normal or anomalous.

[0075] Understandably, if an on-demand triggering mechanism is adopted, edge data will only be marked as edge data to be processed when the detection result shows that the current data is abnormal, that is, when it may contain high-value fault information or security risks, thus triggering subsequent complex edge-cloud collaborative assessment and processing procedures. For redundant data with normal detection results, it can be directly discarded, archived locally at low frequency, or not subjected to subsequent advanced processing.

[0076] The method provided in this invention effectively filters out massive amounts of invalid or normal state data by deploying an anomaly detection model at the edge and introducing a data acquisition mechanism based on anomaly triggering. This ensures that valuable computing resources and network bandwidth are used only for high-value data with anomalies, i.e. edge data to be processed. This not only significantly reduces the overall energy consumption and communication pressure of the system, but also makes the system more sensitive and focused in its response to sudden failures.

[0077] Based on any of the above embodiments, the edge-cloud network status includes network quality parameters and bandwidth utilization.

[0078] Here, network quality parameters refer to technical indicators that reflect the stability and timeliness of communication links. In practice, these parameters can be measured in real time by sending ICMP heartbeat packets or probe messages from edge nodes to the cloud. These parameters typically include round-trip time (RTD), used to assess the physical delay of data transmission; packet loss rate, used to assess network reliability; and signal strength, used to assess the physical layer quality of the wireless connection.

[0079] Furthermore, bandwidth utilization refers to the ratio between the network bandwidth currently in use and the maximum available bandwidth of the link. In practice, the network driver layer of the edge node will count the throughput per unit time in real time and calculate it based on the theoretical or measured upper limit of the current network standard.

[0080] In the evaluation step of Example 1, if the network quality parameters show high latency or high packet loss, or the bandwidth utilization is close to saturation, such as exceeding 80%, the network transmission cost in the comprehensive cost function will increase sharply, thereby prompting the system to tend to choose the edge local processing strategy to avoid data congestion.

[0081] The method provided in this invention refines the edge-cloud network status into network quality parameters and bandwidth utilization, enabling the system to perceive the communication environment at the protocol and link layers. This refined perception capability ensures that the task offloading strategy can dynamically adapt to network fluctuations, avoiding task timeouts or data loss caused by forcibly uploading data during network congestion, thereby guaranteeing the robustness and reliability of the edge-cloud collaborative system in unstable network environments.

[0082] Based on any of the above embodiments, the present invention also provides a cross-domain data collaborative processing system based on blockchain and multi-party secure computation. The system includes an edge acquisition and preprocessing module, an edge-cloud collaborative communication and scheduling module, a cloud-based intelligent processing and control module, and a security and system management module.

[0083] The edge acquisition and preprocessing module is deployed on-site in edge computing devices such as edge servers, industrial PCs, and smart gateways. It includes: a data acquisition submodule, used to interface with various communication protocols to acquire raw data from terminals such as sensors, cameras, and industrial controllers; a local caching and time synchronization submodule, used for time-series alignment and short-term buffering of data to ensure data integrity; a data preprocessing submodule, used for data cleaning, format conversion, preliminary feature extraction, and anomaly screening; and a local event detection submodule, used to detect key events and trigger responses based on preset rules.

[0084] The edge-cloud collaborative communication and scheduling module is deployed between edge nodes and the cloud platform, including: a two-way communication submodule, used to establish an efficient edge-cloud communication link based on a lightweight protocol; a data transmission control submodule, used to determine whether data is uploaded to the cloud, the upload frequency, and the content type based on factors such as task priority, network status, and bandwidth utilization; and a task distribution submodule, used to dynamically distribute processing rules, model parameters, or decision instructions to edge nodes through the cloud, enabling remote task updates and customized execution.

[0085] The cloud-based intelligent processing and control module is deployed on a cloud platform or data center and includes: a data aggregation and storage submodule, used to receive data results, event reports, or partial raw data from multiple edge nodes and store them in a distributed database; a streaming and batch processing engine, used to combine frameworks such as Flink and Spark Streaming to perform cross-node and cross-time period data fusion analysis; a model training and policy generation submodule, used to build machine learning models based on historical data for prediction, identification, and optimization; and a policy control submodule, used to generate control policies, rule updates, and other content based on the analysis results and remotely distribute them to edge nodes.

[0086] The security and system management module includes: an identity authentication and access control submodule, which uses certificate or token-based authentication mechanisms to ensure the security of edge-cloud communication; a data encryption submodule, which performs symmetric / asymmetric encryption on sensitive data transmitted between the edge and cloud to prevent information leakage; and a unified operation and monitoring submodule, which provides multi-level system operation monitoring, log collection, anomaly alarms, and remote diagnostic capabilities.

[0087] The cross-domain data collaborative processing system based on blockchain and multi-party secure computation proposed in this invention embodiment has been significantly optimized and improved on the basis of existing technologies, and has the following beneficial effects: First, it enhances the security and trustworthiness of data collaboration. By integrating the immutable and traceable technical characteristics of blockchain with the privacy protection capabilities of multi-party secure computation, the system enables cross-domain entities to conduct joint modeling and intelligent analysis without exposing original data, effectively avoiding the security risks and trust deficiencies caused by data silos, and providing strong technical support for the circulation of data elements.

[0088] Secondly, it ensures privacy compliance in data sharing. By employing advanced privacy computing technologies such as homomorphic encryption, federated learning, and secure multi-party computation, it ensures that data is exchanged in a usable but invisible state. This is particularly suitable for fields with extremely high requirements for data security and privacy protection, such as healthcare, government affairs, transportation, and finance, and complies with relevant regulations.

[0089] Third, it enables efficient cross-domain data fusion and governance. The system integrates functional modules such as data collection, standardization transformation, semantic mapping, data anonymization, quality assessment, and knowledge modeling. It can quickly connect to multi-source heterogeneous data to form a unified data semantic view and structured data asset catalog, significantly reducing the data integration threshold between different business systems and improving governance efficiency and data availability.

[0090] Fourth, it supports a verifiable data circulation mechanism. By relying on consortium blockchains and smart contract mechanisms, the system can provide on-chain records and permission verification for every data transfer, enabling full-process auditability and verification of the data lifecycle, preventing data abuse and unauthorized access, and enhancing the trust foundation of the data transaction process.

[0091] Fifth, enhance the system's scalability and adaptability. By adopting a microservice architecture and containerized deployment strategy, the system can flexibly connect to heterogeneous platforms and business interfaces of different organizations, while possessing strong maintainability and scalability, facilitating rapid replication and deployment in more business scenarios in the future.

[0092] Sixth, it supports intelligent analysis services based on large models. By reserving interfaces for calling large model services, the system supports calling large language models, graph neural network models, knowledge graph reasoning models, etc., for complex reasoning and knowledge discovery under privacy protection. This allows data to still exert its value in deep analysis and prediction while being protected, providing strong support for intelligent applications.

[0093] Seventh, promote the market-oriented circulation and asset-based operation of data elements. It is understandable that through metadata management, data pricing models, and data access control strategies, the system can provide technical support for data ownership confirmation, pricing, billing, and revenue distribution, providing underlying capabilities to build a secure, orderly, and controllable data element circulation mechanism, and promoting the assetization and market application of data.

[0094] Based on any of the above embodiments Figure 3 This is a schematic diagram of the edge-cloud collaborative data processing device provided by the present invention, as shown below. Figure 3 As shown, the device includes: Acquisition unit 310 acquires edge data to be processed; Evaluation unit 320 evaluates the edge cloud resource status, edge cloud network status, and pending task information of the pending edge data to obtain the task offloading strategy for the pending edge data. The processing unit 330 processes the edge data to be processed according to the task unloading strategy.

[0095] The device provided in this invention introduces a comprehensive evaluation mechanism for edge-cloud resource status, edge-cloud network status, and information on tasks to be processed, to obtain a task offloading strategy for edge data to be processed. This enables dynamic decision-making at the source of data generation to determine the optimal processing path. This not only significantly reduces network bandwidth consumption and transmission latency caused by directly uploading massive amounts of raw data to the cloud, meeting the millisecond-level response requirements of industrial scenarios, but also makes full use of idle edge computing resources by rationally allocating tasks, avoiding overload of the cloud center. This improves the processing efficiency and resource utilization of the edge-cloud collaborative system for data processing.

[0096] Based on any of the above embodiments, the evaluation unit is specifically used for: Construct a comprehensive cost function; the comprehensive cost function includes a resource overhead function, a network transmission performance function, and a service quality function; The edge cloud resource status, the edge cloud network status, and the task information to be processed are used as inputs to the comprehensive cost function. The comprehensive cost function is solved to obtain the optimal solution of the comprehensive cost. The optimal solution includes the task execution subject, data transmission parameters, and the target task data that the task execution subject needs to execute. Based on the optimal solution, the task unloading strategy is determined.

[0097] Based on any of the above embodiments, the evaluation unit is further specifically used for: When the task execution entity includes the cloud, the target task data to be executed in the cloud is transmitted to the cloud according to the data transmission parameters; Based on the cloud task processing model, data processing is performed on the target task data to be executed in the cloud. The data transmission parameters include the data transmission frequency.

[0098] Based on any of the above embodiments, the processing unit is specifically used for: When the task execution entity includes an edge node, data processing is performed on the target task data to be executed by the edge node based on the local task processing model; The local task processing model is deployed on the edge node.

[0099] Based on any of the above embodiments, the processing unit is specifically used for: The edge processing results obtained from the edge nodes are transmitted to the cloud. Based on the cloud task processing model, data processing is performed on the target task data to be executed in the cloud and the edge processing results.

[0100] Based on any of the above embodiments, the device further includes an updating unit, which is specifically used for: Obtain historical edge data; Based on the historical edge data, the cloud task processing model is optimized to obtain optimized model parameters; The optimized model parameters are sent to the edge nodes to update the model parameters of the local task processing model.

[0101] Based on any of the above embodiments, the task information to be processed includes at least one of task priority, real-time requirements, and data volume; The acquisition unit is specifically used for: Acquire edge data collected by edge nodes; Based on the anomaly detection model, the edge data is detected to obtain the detection results; If the detection result is abnormal, the edge data is treated as edge data to be processed; the anomaly detection model is deployed on the edge node.

[0102] Based on any of the above embodiments, the edge-cloud network status includes network quality parameters and bandwidth utilization.

[0103] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute an edge-cloud collaborative data processing method, which includes: acquiring edge data to be processed; evaluating the edge-cloud resource status, the edge-cloud network status, and the task information to be processed of the edge data to obtain a task offloading strategy for the edge data to be processed; and processing the edge data to be processed according to the task offloading strategy.

[0104] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the edge-cloud collaborative data processing method provided by the above methods. The method includes: acquiring edge data to be processed; evaluating the edge-cloud resource status, edge-cloud network status, and task information to be processed of the edge data to be processed to obtain a task offloading strategy for the edge data to be processed; and processing the edge data to be processed according to the task offloading strategy.

[0106] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the edge-cloud collaborative data processing method provided by the above methods. The method includes: acquiring edge data to be processed; evaluating based on edge-cloud resource status, edge-cloud network status, and task information to be processed of the edge data to be processed to obtain a task offloading strategy for the edge data to be processed; and processing the edge data to be processed according to the task offloading strategy.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method for edge-cloud collaboration, characterized in that, include: Acquire edge data to be processed; An evaluation is conducted based on the edge cloud resource status, edge cloud network status, and the pending task information of the edge data to be processed, to obtain the task offloading strategy for the edge data to be processed. The edge data to be processed is processed according to the task unloading strategy.

2. The edge-cloud collaborative data processing method according to claim 1, characterized in that, The evaluation based on the edge cloud resource status, edge cloud network status, and pending task information of the pending edge data yields a task offloading strategy for the pending edge data, including: Construct a comprehensive cost function; the comprehensive cost function includes a resource overhead function, a network transmission performance function, and a service quality function; The edge cloud resource status, the edge cloud network status, and the task information to be processed are used as inputs to the comprehensive cost function. The comprehensive cost function is solved to obtain the optimal solution of the comprehensive cost. The optimal solution includes the task execution subject, data transmission parameters, and the target task data that the task execution subject needs to execute. Based on the optimal solution, the task unloading strategy is determined.

3. The edge-cloud collaborative data processing method according to claim 2, characterized in that, The step of processing the edge data to be processed according to the task unloading strategy includes: When the task execution entity includes the cloud, the target task data to be executed in the cloud is transmitted to the cloud according to the data transmission parameters; Based on the cloud task processing model, data processing is performed on the target task data to be executed in the cloud. The data transmission parameters include the data transmission frequency.

4. The edge-cloud collaborative data processing method according to claim 3, characterized in that, The step of processing the edge data to be processed according to the task unloading strategy further includes: When the task execution entity includes an edge node, data processing is performed on the target task data to be executed by the edge node based on the local task processing model; The local task processing model is deployed on the edge node.

5. The edge-cloud collaborative data processing method according to claim 4, characterized in that, The cloud-based task processing model performs data processing on the target task data to be executed in the cloud, including: The edge processing results obtained from the edge nodes are transmitted to the cloud. Based on the cloud task processing model, data processing is performed on the target task data to be executed in the cloud and the edge processing results.

6. The edge-cloud collaborative data processing method according to claim 4, characterized in that, The method further includes: Obtain historical edge data; Based on the historical edge data, the cloud task processing model is optimized to obtain optimized model parameters; The optimized model parameters are sent to the edge nodes to update the model parameters of the local task processing model.

7. The edge-cloud collaborative data processing method according to any one of claims 1 to 6, characterized in that, The information of the task to be processed includes at least one of the following: task priority, real-time requirements, and data volume. The steps for obtaining the tasks to be processed include: Acquire edge data collected by edge nodes; Based on the anomaly detection model, the edge data is detected to obtain the detection results; If the detection result is abnormal, the edge data is treated as edge data to be processed; the anomaly detection model is deployed on the edge node.

8. The edge-cloud collaborative data processing method according to any one of claims 1 to 6, characterized in that, The edge cloud network status includes network quality parameters and bandwidth utilization.

9. A data processing device for edge-cloud collaboration, characterized in that, include: The acquisition unit acquires the edge data to be processed. The evaluation unit evaluates the edge cloud resource status, edge cloud network status, and pending task information of the edge data to be processed, and obtains the task offloading strategy for the edge data to be processed. The processing unit processes the edge data to be processed according to the task unloading strategy.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the edge-cloud collaborative data processing method as described in any one of claims 1 to 8.

Images