A data-driven learning model combination optimization method based on edge cloud cooperation

Abstract

The application discloses a data-driven learning model combination optimization method based on edge cloud cooperation, and the method comprises the following steps: constructing a cooperative computing framework based on cloud nodes, edge nodes and device nodes, and distributing distributed tasks and computing resources; constructing a plurality of micro-service interfaces, including a platform service interface, an evaluation service interface and a model training service interface of the edge nodes, a model upgrade service interface and a candidate solution evaluation service interface of the cloud nodes; constructing a multi-round model combination strategy based on integrated learning and a model online enhancement combination training strategy, the plurality of micro-service interfaces are encapsulated into a Docker image and saved to a cloud image warehouse for version management and distribution, and a communication mechanism between the cloud nodes and the edge nodes is constructed. Through the cooperative work of the cloud and the edge computing, the application significantly improves the solution efficiency of complex distributed optimization problems and improves the resource utilization efficiency and data processing intelligence.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data-driven optimization, and particularly relates to a data-driven learning model combination optimization method based on edge-cloud cooperation. BACKGROUND

[0002] In recent years, with the rapid development of the Internet of Things and edge computing, data processing and optimization decision-making face many challenges and development opportunities. In the optimization process of complex industrial systems, how to effectively process massive distributed data, reduce computational delay and improve optimization efficiency has become a key problem to be solved. For example, the complex design system of the automobile industry is usually composed of many subsystems distributed in multiple locations, which perform different simulation experiments, such as noise vibration intensity, computational fluid dynamics and crash durability analysis; such complex problems usually have large computational load, widely distributed data and high evaluation cost, which are difficult to solve effectively by using traditional optimization algorithms.

[0003] Proxy-assisted evolutionary computation is a powerful means to solve complex optimization problems. By introducing a proxy model in the evolutionary process, the computational cost can be significantly reduced and the optimization efficiency can be improved. However, the traditional proxy-assisted evolutionary computation method still has certain limitations when facing the reality challenges of explosive data and limited computing resources. Therefore, how to train a proxy model in the collaborative framework of edge computing and cloud computing, combined with model combination and integration strategies to improve the performance of the proxy model, so as to effectively cope with distributed data-driven complex optimization problems, has become one of the current research hotspots. SUMMARY

[0004] In order to overcome the defects and deficiencies of the prior art, the present application provides a data-driven learning model combination optimization method based on edge-cloud cooperation. In order to cope with the challenges of complex optimization problems in a distributed environment, the present application optimizes the solution process of complex problems in a distributed environment through efficient resource utilization and intelligent data processing. Specifically, by designing the collaborative computing of cloud, edge and device components, the present application realizes the systematization and efficient deployment of distributed proxy-assisted evolutionary algorithms. The present application adopts a micro-service design architecture and a container deployment scheme, thereby realizing the flexibility and scalability of proxy model deployment. In addition, relying on the cluster built by the open source edge computing architecture KubeEdge, the present application provides solid technical support for the interactive operation, real-time monitoring and comprehensive management of the platform.

[0005] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0006] The present application provides a data-driven learning model combination optimization method based on edge-cloud cooperation, comprising the following steps:

[0007] A collaborative computing framework is constructed based on cloud nodes, edge nodes and device nodes, and distributed tasks and computing resources are allocated;

[0008] A plurality of microservice interfaces are constructed, including a platform service interface, an evaluation service interface of an edge node, a model training service interface of an edge node, a model upgrade service interface of a cloud node, and a candidate solution evaluation service interface of a cloud node;

[0009] The platform service interface is provided with a plurality of functional modules, and the process of the distributed agent-assisted evolutionary algorithm is decomposed into a plurality of functional modules. The evaluation service interface evaluates the candidate solution of the evaluation request of the cloud node or the device node, and stores the evaluation result in the edge node. The model training service interface trains the agent model based on the local data of the edge node. The model upgrade service interface obtains the model training result, combines and integrates the model, and distributes the updated agent model version to each edge node. The candidate solution evaluation service interface screens the candidate solution and performs real environment evaluation by calling the evaluation service interface of the edge node.

[0010] A multi-round model combination strategy based on ensemble learning and a model online enhancement combination training strategy are constructed. The plurality of microservice interfaces are encapsulated into Docker images and saved to a cloud image warehouse for version management and distribution. A communication mechanism between the cloud nodes and the edge nodes is constructed.

[0011] As a preferred technical solution, the cloud node executes the agent-assisted evolutionary algorithm, monitors the states of the edge nodes and the device nodes, and dynamically adjusts the distribution of cloud resources. The edge node acquires device node collected data, locally trains and updates the agent model, executes or unloads computing tasks, and manages edge resources. The device node acquires real-time data and manages the computing resources and communication resources of the device.

[0012] As a preferred technical solution, the cloud node adopts Kubernetes technology, interacts with a CloudCore component using an API, and the CloudCore component acts as a scheduling center. Based on a workload prediction algorithm, the cloud node dynamically adjusts the distribution of cloud resources.

[0013] As a preferred technical solution, the edge node realizes the distribution, execution of containerized tasks and unified management of edge resources based on an EdgeCore component. The EdgeCore component and the CloudCore component communicate through a bidirectional encrypted channel to realize real-time synchronization of tasks and resources.

[0014] As a preferred technical solution, the edge node pulls a Docker image through a KubeEdge framework and instantiates an EdgeCore component, the EdgeCore component starts a service container on the edge node, uses monitoring and management tools provided by KubeEdge to monitor the running state of the container instance in real time, and when the container abnormally or crashes, the EdgeCore component automatically restarts the container according to the preset strategy and performs fault recovery.

[0015] As a preferred technical solution, a multi-round model combination strategy based on ensemble learning and a model online enhancement combination training strategy are constructed, specifically including:

[0016] Based on the local data set, a plurality of agent models are trained, a plurality of agent models are aggregated to form a global model, the fitness of each optimization scheme is calculated based on the global model, and the local data set is updated through real environment evaluation;

[0017] Collecting agent models of multiple optimization rounds to obtain cross-round models, and aggregating the cross-round models;

[0018] Based on the weight distribution, the local data set is sampled without replacement to obtain a training set, the agent model is trained based on the training set, each sample in the local data set is evaluated, the relative error of the sample is calculated, and the sample weight is updated.

[0019] As a preferred technical solution, the local data set includes production history, raw material ratio, process parameters, equipment state and production output of each workshop, and the agent model is trained according to the production optimization target and the industrial production formula is evaluated.

[0020] As a preferred technical solution, the fitness of each optimization scheme is calculated based on the global model, specifically including:

[0021] An initial population is randomly generated, each individual represents a potential formula scheme, the fitness of each optimization scheme is calculated based on the global model, the individuals in the population are selected, crossed and mutated to generate a new generation of formula schemes, and the optimal formula scheme is iteratively generated.

[0022] As a preferred technical solution, a multi-round model combination strategy based on ensemble learning and a model online enhancement combination training strategy are constructed, specifically including:

[0023] For each edge node k, the cloud node records the agent models collected in the first m rounds, denoted as Select the current optimal agent model:

[0024]

[0025] where Median(·) operator returns the median of the input set, representing the stable model selected with the median value of the metrics, and the metric of the model is calculated as follows:

[0026]

[0027] where, denotes the metric of the model;

[0028] The weights obtained by the boundary judgment strategy are used to aggregate the cross-round models:

[0029]

[0030] where the weight ω k (x) is calculated according to the data interval of the data set, lb k and ub k denote the upper and lower limits of the statistical data in the edge node k, Z t is the normalization factor;

[0031] The data set is represented as: n k is the total number of samples in the edge node k, and the data set is sampled without replacement to obtain the training set

[0032] Based on the training set the proxy model is trained

[0033] The proxy model is used to evaluate each sample in the data set and compare the result with the true label value to calculate the relative error of the sample

[0034]

[0035] where, is the maximum absolute error, and the model error rate is calculated as follows:

[0036]

[0037] The metric of the model is defined as:

[0038]

[0039] The weight of each sample in the data set is updated using the following formula:

[0040] ​

[0041] wherein, is a standardization factor.

[0042] As a preferred technical solution, a communication mechanism between the cloud nodes and the edge nodes is constructed, specifically including:

[0043] The communication between the cloud nodes and the edge nodes is realized through the KubeEdge framework, a control channel is established between the CloudCore component and the EdgeCore component, and the control channel adopts the TLS encryption protocol for data transmission.

[0044] Based on the periodic heartbeat mechanism, the CloudCore component monitors the state of the EdgeCore component, and if the edge node appears abnormal or goes offline, a preset disaster recovery strategy is triggered to migrate tasks to a standby node.

[0045] The stream data transmission and block concurrent transmission technology are adopted for data transmission, and the WebSocket protocol is adopted to realize the bidirectional real-time communication.

[0046] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0047] (1) The present application significantly improves the solving efficiency of complex distributed optimization problems through the cooperative work of cloud and edge computing, improves the resource utilization efficiency and data processing intelligence, provides strong technical support for the systematization and efficient deployment of distributed agent assisted evolutionary algorithm, and realizes the efficient solution of complex optimization problems in a distributed environment.

[0048] (2) The present application introduces a multi-model combination integration strategy, uses historical data and agent models for assistance, optimizes the model training and prediction process, and solves the performance bottleneck encountered by the prior art when solving high-computing-cost distributed optimization problems.

[0049] (3) The present application combines the powerful computing capability of cloud computing with the low-delay characteristics of edge computing, improves the execution efficiency of the optimization algorithm through dynamic resource scheduling and intelligent task allocation, and solves the problems of data transmission delay and uneven distribution of computing resources in the prior art in a distributed environment. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of the data-driven learning model combination optimization method based on edge-cloud collaboration of the present application;

[0051] Figure 2 is a schematic diagram of the implementation architecture of the data-driven learning model combination optimization method based on edge-cloud collaboration of the present application;

[0052] Figure 3 This is a schematic diagram of the interface arrangement of the service module of the present invention;

[0053] Figure 4 This is a service deployment diagram of the data-driven learning model combinatorial optimization method based on edge-cloud collaboration of the present invention;

[0054] Figure 5 This is a schematic diagram of the cloud-edge communication mechanism of the data-driven learning model combinatorial optimization method based on edge-cloud collaboration of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0056] like Figure 1 As shown, this embodiment provides a data-driven learning model combinatorial optimization method based on edge-cloud collaboration. It includes module microservice interface design and management for model combinatorial optimization, and a distributed agent-assisted evolution algorithm deployment and implementation strategy. This method overcomes the limitations of traditional centralized systems for distributed heterogeneous data-driven optimization, improving the efficiency and effectiveness of solving complex optimization problems. The module microservice interface design and management includes the modular design of the microservice architecture and the definition of service interfaces, ensuring efficient algorithm execution and flexible deployment. The distributed agent-assisted evolution algorithm deployment and implementation strategy covers key steps such as algorithm initialization, evolution, selection, model training, and evaluation, as well as the management of model adjustment based on evaluation results and algorithm iteration termination conditions. Specifically, it includes the following steps:

[0057] S1: Construct a three-layer collaborative computing framework based on "cloud-edge-device". Through the collaboration of cloud nodes, edge nodes and device nodes, it realizes the efficient execution of distributed optimization tasks and the rational allocation of computing resources. This architecture aims to make full use of the powerful computing capabilities of cloud computing, the low latency characteristics of edge computing and the distributed processing capabilities of device resources.

[0058] Cloud nodes are used to achieve the following:

[0059] 1) Core process of Surrogate-Assisted Evolutionary Computation. The cloud utilizes high-performance computing resources to execute the Surrogate-Assisted Evolutionary Algorithm (SAEA), which includes global search of the optimization problem, selection and combination of surrogate models, and global scheduling of optimization schemes;

[0060] 2) System Monitoring and Management. Real-time monitoring of edge nodes and device nodes' status, including resource usage, task execution progress, and data transmission stability, through an integrated control panel.

[0061] 3) Resource Allocation and Scheduling. As shown in Figure 2 , using Kubernetes (K8s) technology, interacting with CloudCore core components through its API, achieving dynamic allocation of cloud resources and intelligent scheduling of tasks. CloudCore components act as a scheduling center, dynamically adjusting the resource distribution between cloud and edge based on workload prediction algorithms.

[0062] Edge nodes are located at the network edge close to devices, and their core responsibilities include:

[0063] 1) Agent Model Training. After collecting data from device nodes, edge nodes use the lightweight training framework TensorFlow Lite to locally train and update agent models to adapt to real-time data changes.

[0064] 2) Task Offloading and Execution. By analyzing the load of devices and data transmission delay, dynamically offloading optimized computing tasks, the task offloading mechanism is based on multi-index decision-making (including device processing capacity, bandwidth conditions and latency constraints).

[0065] 3) Edge Resource Management. EdgeCore components run in edge nodes, responsible for the distribution, execution of containerized tasks and unified management of edge resources. EdgeCore and CloudCore communicate through a secure two-way encrypted channel (TLS protocol) to achieve real-time synchronization of tasks and resources.

[0066] Device nodes are composed of Internet of Things devices (sensors, actuors and embedded terminals), mainly responsible for:

[0067] 1) Data Collection and Transmission. Device nodes send real-time data collected through lightweight protocol MQTT to edge nodes.

[0068] 2) Task Auxiliary Execution. Device nodes assist edge nodes in executing local optimization tasks or evaluation tasks, such as calculating the fitness value of optimization candidate solutions or performing simple inference calculations.

[0069] 3) Resource Integration and Scheduling. Device controllers are responsible for coordinating the computing resources and communication resources of devices to minimize latency and optimize bandwidth utilization.

[0070] S2: Microservice Interface Design. Implementing each functional component of the distributed agent-assisted evolutionary algorithm in a lightweight manner, deploying services to the cloud and edge in a stateless image manner, and ensuring real-time and reliable data transmission through communication mechanisms;

[0071] The embodiment designs multiple micro-service interfaces to realize seamless collaboration between various components of the distributed agent-assisted evolutionary algorithm. These interfaces cover the evaluation and model training functions of the edge node, as well as the model upgrade, combined integration, and other functions of the cloud node. As shown in Figure 3 The embodiment realizes the flexibility and scalability of the system through modular design and micro-service orchestration.

[0072] S21: Platform service interface orchestration:

[0073] 1) Modular design algorithm process decomposition: The entire distributed agent-assisted evolutionary algorithm process is subdivided into multiple independent modules I = {I1, I2,..., In}, each module is responsible for a single function, facilitating independent updating and deployment between modules, and the tasks of each module are as follows: n}, each module is responsible for a single function, facilitating independent updating and deployment between modules, and the tasks of each module are as follows:

[0074] 1.1) Initialization module: used to generate the initial population, provide configurable parameters such as population size, gene representation, etc.

[0075] 1.2) Evolution module: execute evolution operators in the algorithm, such as mutation, crossover operations, this module supports multiple evolution operator strategies, and has algorithm parameter adjustment function, supports dynamic strategy switching.

[0076] 1.3) Selection module: use roulette, tournament selection, etc. to select excellent individuals from the current population, support sorting by fitness, random selection, etc.

[0077] 1.4) Candidate evaluation module: reduce computational overhead and improve evaluation efficiency through proxy model.

[0078] 1.5) Real evaluation module: this module evaluates the candidate solution in the real environment through cloud-edge collaboration, ensuring that the evaluation result has practical application significance.

[0079] 1.6) Model training module: train and optimize the proxy model using historical data, including data preprocessing, model training, parameter tuning, etc.

[0080] 2) Platform service interface definition: each module exposes clear RESTful API interfaces, and exchanges data through JSON protocol. Interface design follows the principles of high cohesion and low coupling to ensure the independence and scalability of system modules, and realizes interface version management to update at any time as needed.

[0081] S22: Define the evaluation service interface on the edge node;

[0082] Interface name: evaluation;

[0083] Function description: Perform high-cost model combination evaluation tasks on Internet of Things devices.

[0084] Workflow:

[0085] 1) Receive evaluation request R from cloud nodes or device nodes e ;

[0086] 2) Perform real evaluation E(x) on candidate solution "x" in the request;

[0087] 3) Store the evaluation result r = E(x) in the local database of the edge node;

[0088] 4) Return the evaluation result r to the requester.

[0089] S23: Define model training service interface on edge node;

[0090] Interface name: model_training;

[0091] Function description: Train proxy model using edge node local data and return updated model parameters.

[0092] Workflow:

[0093] 1) Sample the required data set from the historical database, support data preprocessing (such as normalization, feature selection).

[0094] 2) Train the proxy model using the sampled data, support Adam, SGD algorithm, and use cross-validation method to optimize hyperparameters during training.

[0095] 3) After training, return the updated model parameters and store them in the local model library.

[0096] S24: Design model upgrade service interface on cloud node;

[0097] Interface name: model_upgrade;

[0098] Function description: Manage and distribute new versions of proxy models to edge nodes;

[0099] Workflow:

[0100] 1) Collect model training results from edge nodes;

[0101] 2) Aggregate and optimize model parameters, combine and integrate heterogeneous models;

[0102] 3) Distribute the updated proxy model version to each edge node;

[0103] S25: design a candidate solution evaluation service interface on the cloud node;

[0104] Interface name: candidate_evaluation;

[0105] Function description: screen candidate solutions and perform real environment evaluation by calling the evaluation interface of the edge node;

[0106] Workflow:

[0107] 1) Select potential candidate solutions C = {c1, c2,..., cN} from the population; p};

[0108] 2) Send candidate solutions c i ∈ C to the evaluation interface of the edge node for evaluation, supporting parallel processing of batch evaluation tasks;

[0109] 3) Receive evaluation results E(c i ) and adjust individuals in the population according to the results, and the evaluation results are fed back to other modules of the cloud node for subsequent model upgrade and other operations.

[0110] S3: Plan the specific implementation process of the distributed agent-assisted evolutionary algorithm, and construct a multi-round model combination strategy based on ensemble learning and an online enhanced combination training strategy for models;

[0111] Taking the industrial production recipe optimization problem as an example, industrial production recipe optimization is a typical application of distributed data-driven optimization, aiming to improve production efficiency, reduce costs, and optimize product production quality. In this process, each production workshop is an independent and autonomous entity with its own production data, including raw material ratio, process parameters, equipment status, and other information, and can build agent models based on these local data to evaluate the pros and cons of different recipes. The cloud server is responsible for integrating edge agent models from various workshops, continuously improving production recipes through model-driven evolutionary optimization algorithms; at the same time, each workshop will provide real-time feedback and evaluation on the optimization strategies given by the cloud, ensuring that the optimization scheme matches the actual production environment of the workshop, thereby achieving a more precise and efficient production process.

[0112] S31: Edge model training. Each edge client (edge k, k = 1, 2,..., K) trains the agent model using the local data set The model is then uploaded to the cloud.

[0113] In the recipe optimization problem, one workshop corresponds to one edge client, and the local data set The production history, raw material ratio, process parameters, equipment status, and production output of each workshop are included. The agent model The impact of different formulations on production efficiency, cost, and product quality can be predicted. Each workshop trains the agent model according to the same optimization goal (such as maximizing product quality or minimizing raw material waste) to generate a model that can evaluate the pros and cons of different formulations. After training, the workshop uploads its model to the cloud for global optimization.

[0114] S32: Cloud model aggregation. The cloud aggregates agent models from various edge clients Form a global model This aggregation process can be achieved through different model combination operators, such as average, weighted average, etc.

[0115] Global evaluation model In the aggregation, the models of each workshop may have strong adaptability under certain specific production conditions, therefore, the cloud can assign different weights to the models of each workshop according to the quality of the data or the importance of the workshop.

[0116] S33: Evolution algorithm execution. The initial population is randomly generated, each individual represents a potential solution. The evolution algorithm evolves the population through iterations, using the global model to predict the quality of individuals and guide the evolution of the population. Ultimately, the individual closest to the optimal solution will be broadcast to each edge.

[0117] The initial population consists of multiple potential formulation schemes generated randomly, each formulation scheme (individual) includes different raw material ratios, process parameters, etc. The evolution algorithm calculates the fitness of each individual with the assistance of the global model (predicts the impact of each formulation scheme on production efficiency, cost, and quality). Through evolution strategies such as genetic algorithms, the best individuals in the population are selected, crossed, and mutated to generate a new generation of formulation schemes. As the iterations proceed, the population will continuously approach the optimal solution, and ultimately a formulation scheme close to the optimal solution will be obtained, which will be broadcast to each workshop for execution.

[0118] S34: Data set and model update. In the online scenario, the edge client updates the local data by performing real evaluation, represented as Local agent model is updated accordingly and re-transmitted to the cloud for subsequent evolution.

[0119] The edge workshops perform real production behaviors on the optimized recipes to validate their effectiveness and update the local datasets. Specifically, each workshop will produce according to the optimized recipe provided by the cloud, collect relevant production data such as production time, yield, quality indicators, etc., and add these new data points to the dataset. The local agent model of the workshop will be updated based on the new data, improving its ability to evaluate future recipe optimization. The updated agent model is returned to the cloud for subsequent optimization iterations. Through this process, the edge workshops and the cloud form a continuous feedback and optimization cycle, allowing the production recipe to be continuously adjusted to adapt to new production requirements and changing process conditions.

[0120] S35: Multi-round model combination strategy based on ensemble learning is a model combination method running on the cloud, which is used to aggregate the agent models trained by different edges. This method not only aggregates the models trained in a single round, but also uses the information of cross-round models to improve the effectiveness and robustness of the overall model. The specific steps are as follows:

[0121] 1) Edge cross-round model combination. In the current optimization round (the mth round), for each edge k (k = 1, 2,..., K), the cloud will record the agent models collected in the first m rounds, denoted as The algorithm first obtains the current optimal agent model according to the designed index:

[0122]

[0123] The Median(·) operator returns the median of the input set, representing the selection of a stable model measured by the median of the index. The metric of the model is calculated as follows:

[0124]

[0125] The index is calculated by reference to step S36.

[0126] 2) Cloud-edge model combination. The second step of the algorithm uses the weights obtained by the boundary judgment strategy to aggregate the cross-round models obtained in 1):

[0127]

[0128] The weight ω k (x) is calculated according to the data interval of the sample, lb k and ub k represent the upper and lower limits of the statistical data in edge k, and Z t is the normalization factor.

[0129] S36: Combined with the powerful computing power of cloud computing and the low delay characteristics of edge computing, the embodiment dynamically adjusts the model parameters in the distributed evolutionary process through edge-cloud collaborative mode, continuously improves the performance of the combined model to optimize the algorithm efficiency. The embodiment constructs an online enhanced combined training strategy, and the specific steps are as follows:

[0130] 1) Input sample and sample weight. Dataset Where n k is the total number of samples in edge k. The sample weight in the last round of optimization process At the beginning (first round of training) Set to 1 / n k , all samples have equal weight, and the total is 1.

[0131] 2) Model training set sampling. Based on the weight distribution The dataset is sampled without replacement to obtain the training set The training set is a subset of , and the higher the weight of the sample, the greater the chance of being selected.

[0132] 3) Agent model training. Use to train the agent model This algorithm does not limit the selection of the agent model, and machine learning models such as RBF and GP can be selected;

[0133] 4) Calculate the model error rate and model metric index, use the agent model to evaluate each sample in and compare the result with the true label value, calculate the sample relative error

[0134]

[0135] Where, is the maximum absolute error, and the model error rate (sample relative error weighted sum) is calculated as follows:

[0136]

[0137] The model metric index is defined as:

[0138]

[0139] 5) Sample weight iteration. The weight of each sample in is updated using the following formula:

[0140]

[0141] is a standardization factor, ensuring that the sum of the weights is 1.

[0142] S37: Service distribution containerization deployment scheme;

[0143] As shown in Figure 4 After the algorithm flow is decomposed into multiple independent service interfaces, the interfaces are encapsulated into Docker images and saved to the cloud image repository (Docker Hub) for version management and distribution. The cloud uses K8s to manage the life cycle of all containerized services. CloudCore, as the core control component of the cloud, defines and manages the deployment of each service image through Pod YAML files. Pod YAML files include deployment, network configuration, resource limits, and other information to ensure efficient scheduling and automated management of service containers. CloudCore is responsible for: 1) managing the start, stop, expansion, and recovery of service containers; 2) continuously monitoring the status of service containers to ensure their health and stability; 3) dynamically performing elastic expansion and load balancing based on traffic and load conditions to ensure system high availability and performance. Edge nodes pull Docker images and instantiate EdgeCore through the KubeEdge framework, which is responsible for: 1) starting service containers on edge nodes; 2) using monitoring and management tools provided by KubeEdge to monitor the running status of container instances in real time, including CPU usage, memory consumption, and network traffic; 3) when a container appears abnormal or crashes, EdgeCore will automatically restart the container according to the preset policy and perform fault recovery.

[0144] S38: Build cloud-edge communication mechanism;

[0145] In this embodiment, the core of the design of the communication and data transmission mechanism is to ensure stable and low-latency communication between the cloud and edge computing nodes in the system and efficient data processing, while ensuring security and scalability. Communication between the cloud and edge nodes is achieved through the KubeEdge framework, as shown in Figure 5 The specific design is as follows:

[0146] 1) Control channel. A stable control channel is established between CloudCore and EdgeCore for task scheduling, resource management, state monitoring, and other key operations. The control channel uses the TLS encryption protocol to ensure the security of data transmission and prevent data from being tampered with or leaked during transmission.

[0147] 2) Heartbeat mechanism. By sending periodic heartbeat signals, CloudCore can monitor the status of EdgeCore in real time, including its health, load, etc. If the edge node appears abnormal or goes offline, it will trigger the preset disaster recovery strategy to migrate tasks to the standby healthy node, ensuring the stable operation of the system.

[0148] 3) Data pipeline. Using streaming data transmission and block concurrent transmission technology, it supports fast transmission of large-scale data. Through the WebSocket protocol, bidirectional real-time communication is realized to ensure low latency and reliability of data transmission, avoiding the delay problem of traditional HTTP requests. All interfaces support full-duplex communication, allowing cloud and edge nodes to perform bidirectional data transmission when processing tasks. The communication between edge nodes and underlying devices uses the lightweight MQTT protocol based on the publish-subscribe mode, supporting real-time data collection and task instruction issuance.

[0149] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application should be equivalent replacement methods and are included in the protection scope of the present application.

Claims

1. A method for data-driven learning model combination optimization based on edge cloud collaboration, characterized in that, Comprising the following steps: Based on cloud nodes, edge nodes and device nodes, a collaborative computing framework is constructed, distributed tasks and computing resources are allocated; A plurality of micro-service interfaces are constructed, including platform service interfaces, evaluation service interfaces of edge nodes, model training service interfaces of edge nodes, model upgrade service interfaces of cloud nodes, candidate solution evaluation service interfaces of cloud nodes; The process of the distributed agent-assisted evolutionary algorithm is decomposed into a plurality of independent modules, each module is responsible for a single function, and each module exposes a clear RESTful API interface as a platform service interface for data exchange through a JSON protocol, the evaluation service interface evaluates the candidate solution of the evaluation request of the cloud node or the device node, and stores the evaluation result in the edge node, the model training service interface trains the agent model based on the local data of the edge node, the model upgrade service interface obtains the model training result, combines and integrates the model, and distributes the updated agent model version to each edge node, and the candidate solution evaluation service interface screens the candidate solution and performs real environment evaluation by calling the evaluation service interface of the edge node; A multi-round model combination strategy and a model online enhancement combination training strategy based on ensemble learning are constructed, a plurality of micro-service interfaces are encapsulated into Docker images and saved to a cloud image warehouse for version management and distribution, and a communication mechanism between cloud nodes and edge nodes is constructed; A multi-round model combination strategy and a model online enhancement combination training strategy based on ensemble learning are constructed, specifically including: A plurality of agent models are trained based on a local data set, a global model is formed by aggregating the plurality of agent models, the fitness of each optimization scheme is calculated based on the global model, and the local data set is updated through real environment evaluation; A cross-round model is obtained by collecting a plurality of optimization rounds of agent models, and the cross-round model is aggregated; The local data set is sampled without replacement based on a weight distribution to obtain a training set, an agent model is trained based on the training set, each sample in the local data set is evaluated, the relative error of the sample is calculated, and the sample weight is updated; For each edge node , the cloud node records the agent model before the round collection, denoted as , and selects the current optimal agent model: ； ； wherein, The operator returns the median of the input set, representing the selection of a stable model measured by the median of the indicators, the metric indicator of the model is calculated as follows: ； wherein, denotes a model metric indicator; The cross-round model is aggregated by using the weight obtained by the boundary judgment strategy: ； ； ， ； where the weight According to the data set falling data interval calculation, and respectively represent the edge node The upper and lower limits of statistical data, is the normalization factor; The data set is represented as: , is the total number of samples in the edge node , and the data set is sampled without replacement to obtain the training set ; Based on the training set Training the agent model ; Using a proxy model Evaluation dataset For each sample in the dataset, the result is compared to the true label value, and the sample relative error is calculated : ； ； wherein, is the maximum absolute error, the model error rate The calculation formula is represented as: ； Model metrics is defined as: ； Dataset Weight of each sample in the dataset The update is done using the following formula: ； ； wherein is a standardization factor.

2. The edge cloud cooperation based data driven learning model combination optimization method according to claim 1, characterized in that, The cloud node executes the agent-assisted evolutionary algorithm, monitors the states of the edge nodes and the device nodes, and dynamically adjusts the distribution of cloud resources, the edge node acquires device node collected data to locally train and update the agent model, executes or offloads a computing task, and manages edge resources, and the device node acquires real-time data and manages the computing resources and communication resources of the device.

3. The edge cloud cooperation based data driven learning model combination optimization method according to claim 2, characterized in that, The cloud node adopts Kubernetes technology, interacts with a CloudCore component by using an API, the CloudCore component acts as a scheduling center, dynamically adjusts the distribution of cloud resources based on a workload prediction algorithm.

4. The edge cloud cooperation based data driven learning model combination optimization method according to claim 2, characterized in that, The edge node implements distribution, execution and unified management of containerized tasks based on an EdgeCore component, and the EdgeCore component and the CloudCore component communicate through a bidirectional encryption channel to realize real-time synchronization of tasks and resources.

5. The edge cloud cooperation based data driven learning model combination optimization method according to claim 4, characterized in that, The edge node pulls a Docker image and instantiates an EdgeCore component through the KubeEdge framework, the EdgeCore component starts a service container on the edge node, uses the monitoring and management tools provided by KubeEdge to monitor the running state of the container instance in real time, and when the container is abnormal or crashes, the EdgeCore component automatically restarts the container according to the preset strategy and performs fault recovery.

6. The edge cloud cooperation based data driven learning model combination optimization method according to claim 1, characterized in that, The local data set includes the production history, raw material ratio, process parameters, equipment state and production output of each workshop, and the agent model is trained according to the production optimization target, and the industrial production formula is evaluated.

7. The edge cloud cooperation based data driven learning model combination optimization method according to claim 1, characterized in that, The fitness of each optimization scheme is calculated based on the global model, specifically including: An initial population is randomly generated, each individual represents a potential formula scheme, the fitness of each optimization scheme is calculated based on the global model, the individuals in the population are selected, crossed and mutated to generate a new generation of formula schemes, and the optimal formula scheme is iteratively generated. 8.The edge cloud cooperation based data driven learning model combination optimization method of claim 1, wherein, A communication mechanism between the cloud node and the edge node is constructed, specifically including: The communication between the cloud node and the edge node is realized through the KubeEdge framework, a control channel is established between the CloudCore component and the EdgeCore component, and the control channel adopts the TLS encryption protocol for data transmission; Based on the periodic heartbeat mechanism, the CloudCore component monitors the state of the EdgeCore component, if the edge node is abnormal or offline, a preset disaster recovery strategy is triggered, and the task is migrated to a standby node; The data transmission is carried out by using the stream data transmission and block concurrent transmission technology, and the bidirectional real-time communication is realized through the WebSocket protocol.

Images