Generation method and device of task unloading strategy based on privacy perception

Abstract

The invention discloses a method and device for generating a task unloading strategy based on privacy awareness, and relates to the field of information security or other related technical fields, and the method comprises the steps: collecting the state information and task information of a target system; preprocessing the state information and the task information to obtain a state feature vector and a task feature vector; the state feature vector and the task feature are input into a task unloading strategy generation model, a task unloading strategy of the target system is output, the task unloading strategy generation model is a model constructed based on a privacy perception mechanism, and the task unloading strategy generation model is a model constructed based on a privacy perception mechanism. The task unloading strategy is used for indicating that the computing task of the cloud platform is transferred to the edge node to be executed. According to the method and the device, the technical problem of relatively low data security when collaborative computing of a cloud platform and an edge node is involved in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the field of information security or other related technical fields. Specifically, it relates to a method and device for generating a task offloading strategy based on privacy awareness. Background Art

[0002] In recent years, with the popularization of Internet of Things devices, smartphones, social media, etc., the scale and speed of data generation have reached unprecedented growth. In the era of information explosion, the amount of data is huge and the types are diverse. These data provide a rich foundation for intelligent and digital transformation, but also bring unprecedented computing and storage challenges. The generation of massive data, as well as the requirements of various industries for the real-time, accuracy, and stability of computing tasks, make complex computing tasks more and more common.

[0003] To cope with these complex computing tasks, more powerful computing resources must be relied on. Therefore, these computing tasks are usually processed through distributed computing architectures such as cloud computing and edge computing. Cloud computing, with its advantages of centralization and rich resources, can provide high-performance support for a large number of computing tasks. As a supplement to cloud computing, edge computing deploys computing resources on edge nodes close to the data source to reduce the latency of data transmission and meet the requirements of low latency and high real-time. In actual application scenarios, in order to provide a faster response speed and a better user experience, tasks often need to be offloaded. For example, the computing tasks of the cloud platform are offloaded to edge nodes, and the edge nodes assist in processing some computing tasks, which can reduce the latency and bandwidth requirements of data transmission, thereby accelerating the task processing efficiency.

[0004] When performing task offloading between the cloud platform and edge nodes, how to formulate the best task offloading strategy becomes a key issue. The offloading strategy needs to reasonably allocate resources between edge nodes and the cloud platform, not only maximizing the use of their respective resource advantages, but also balancing the overall performance of the system. The best offloading strategy can help reduce computing latency, improve task processing efficiency, reduce resource consumption and costs, and enhance the user experience.

[0005] On the basis of task offloading, the frequent transmission of data between edge nodes and the cloud platform also brings challenges to privacy protection. Especially for data involving sensitive information, there may be a risk of privacy leakage during transmission and processing. In related technologies, when it comes to the collaborative computing of the cloud platform and edge nodes, there is a lack of privacy protection mechanisms, resulting in the technical problem of low data security in task offloading operations.

[0006] In response to the above problems, no effective solution has been proposed yet. Summary of the Invention

[0007] An embodiment of the present invention provides a method and device for generating a task offloading strategy based on privacy awareness, so as to at least solve the technical problem of low data security in the related art when it comes to collaborative computing between a cloud platform and an edge node.

[0008] According to one aspect of the embodiments of the present invention, a method for generating a task offloading strategy based on privacy awareness is provided, including: collecting the status information and task information of a target system, where the target system includes multiple cloud platforms and multiple edge nodes; preprocessing the status information and task information to obtain a status feature vector and a task feature vector; inputting the status feature vector and the task feature into a task offloading strategy generation model, and outputting the task offloading strategy of the target system, where the task offloading strategy generation model is a model constructed based on a privacy awareness mechanism, and the task offloading strategy is used to indicate transferring the computing tasks of the cloud platform to the edge node for execution. The task offloading strategy includes an offloading strategy and a resource allocation strategy, where the offloading strategy is used to indicate the offloading method and offloading time of the computing tasks, and the resource allocation strategy is used to indicate the allocation method and resource allocation amount for allocating resources to the offloaded computing tasks.

[0009] Further, the steps of constructing the task offloading strategy generation model include: modeling the task offloading process between each cloud platform and each edge node in the target system, and constructing a delay model, an energy consumption model, and a privacy model; combining the delay model, the energy consumption model, and the privacy model to construct an optimization objective; establishing a Markov decision model for the optimization objective, and constructing the task offloading strategy generation model based on the Markov decision model and the deep deterministic policy gradient algorithm.

[0010] Further, the steps of constructing the delay model include: for each offloaded computing task, defining the data reception delay for the target cloud platform to offload the task to the target edge node; for each offloaded computing task, defining the computing delay for the edge node to execute the computing task; obtaining the completion delay of each offloaded computing task based on the data reception delay and the computing delay; obtaining the maximum value of all completion delays during the task offloading process, and using the maximum value of the completion delay as the task offloading delay of the target cloud platform; determining the delay for the target cloud platform to execute the local computing task to obtain the computing task execution delay; based on the task offloading delay of the target cloud platform and the computing task execution delay, determining the total delay for the target cloud platform and each edge node to execute the task offloading process; constructing the delay model of the target system based on the total delay for all cloud platforms to execute the task offloading process.

[0011] Further, the steps of constructing the energy consumption model include: defining the communication resource consumption of the target system during the task offloading process, where the communication resource consumption includes the amount of transmitted data and the bandwidth requirement; defining the computing resource consumption of the target system during the task offloading process, where the computing resource consumption includes the computing resource requirement and the computing frequency; constructing the energy consumption model based on the communication resource consumption and the computing resource consumption.

[0012] Further, the steps of constructing the privacy model include: obtaining the task type of each offloaded computing task during the task offloading process; determining the proportion of each offloaded computing task in all offloaded computing tasks based on the task type of each offloaded computing task to obtain the task proportion; calculating the privacy entropy of the target cloud platform for the target edge node based on the task proportion of the offloaded computing tasks, and constructing the privacy model of the target system based on the privacy entropy, where the privacy entropy is used to measure the possibility of the target edge node obtaining the privacy information of the target cloud platform from the computing tasks offloaded from the target cloud platform.

[0013] Further, the steps of constructing the optimization objective by combining the delay model, the energy consumption model, and the privacy model include: respectively configuring weight factors for the delay model, the energy consumption model, and the privacy model; establishing the loss function of the target system based on the delay model and the corresponding weight factor, the energy consumption model and the corresponding weight factor, and the privacy model and the corresponding weight factor; constructing the optimization objective based on the loss function, where the optimization objective is expressed as: (P1): C1: C2: (P1) indicates that the optimization objective is to minimize the loss function, G(t) is the loss function of the target system, C1 is the first constraint function, C2 is the second constraint function, and tgt(t) represents the task offloading strategy. represents the computing resource of the computing task x offloaded by the edge node i to the cloud platform k, f i m represents the maximum computing resource quantity owned by the edge node i. represents the sum of the computing resources of the computing tasks offloaded by the edge node i to all cloud platforms.

[0014] Further, the steps of establishing a Markov decision model for the optimization objective and constructing the task offloading policy generation model based on the Markov decision model and the deep deterministic policy gradient algorithm include: defining the state information and task information of the target system as the state space of the Markov decision model, and defining the optimization objective of the target system and the task offloading policy as the action space of the Markov decision model, defining the negative value of the loss function as the system reward of the Markov decision model; using the state space of the Markov decision model as the input data of the deep deterministic policy gradient algorithm, and using the action space of the Markov decision model as the output data of the deep deterministic policy gradient algorithm, and using the system reward of the Markov decision model as the basis for decision evaluation and model parameter update during the iterative calculation process of the deep deterministic policy gradient algorithm, and obtaining the task offloading policy generation model through iterative training.

[0015] Further, the steps of obtaining the task offloading policy generation model through iterative training include: Step 1, normalizing the state space and inputting the normalized state space into the decision network of the deep deterministic policy gradient algorithm framework, and outputting the action space through the decision network; Step 2, adding noise to the output action space, and inputting the normalized state space and the action space with added noise into the evaluation network of the deep deterministic policy gradient algorithm framework to output the system reward; Step 3, constructing a state transition matrix based on the normalized state space, the action space with added noise, and the system reward, and storing the state transition matrix in the experience replay pool; Step 4, repeating the above Steps 1 to 3, and when the data volume of the state transition matrix in the experience replay pool is greater than a preset data volume threshold, randomly selecting a batch of state transition matrices from the experience replay pool, and using the selected state transition matrices as training samples to input into the target decision network and target evaluation network of the deep deterministic policy gradient algorithm framework to output the target action space and the target system reward, where a batch of state transition matrices contains N state transition matrices, and N is a positive integer; repeating the above Step 4 for iterative training, and when the number of iterations reaches the iteration number threshold, stopping the iteration to obtain the trained task offloading policy generation model.

[0016] According to another aspect of the embodiments of the present invention, there is also provided a generating device for a task offloading strategy based on privacy awareness, including: an acquisition unit, configured to acquire the status information and task information of a target system, where the target system includes a plurality of cloud platforms and a plurality of edge nodes; a preprocessing unit, configured to preprocess the status information and task information to obtain a status feature vector and a task feature vector; an output unit, configured to input the status feature vector and the task into a task offloading strategy generation model, and output the task offloading strategy of the target system, where the task offloading strategy generation model is a model constructed based on a privacy awareness mechanism, and the task offloading strategy is used to indicate transferring the computing tasks of the cloud platform to the edge nodes for execution. The task offloading strategy includes an offloading strategy and a resource allocation strategy, where the offloading strategy is used to indicate the offloading method and offloading time of the computing tasks, and the resource allocation strategy is used to indicate the allocation method and resource allocation amount for allocating resources to the offloaded computing tasks.

[0017] Further, the generating device for the task offloading strategy based on privacy awareness further includes a construction unit, configured to construct a task offloading strategy generation model. The construction unit includes: a first construction subunit, configured to model the task offloading process between each cloud platform and each edge node in the target system, and construct a delay model, an energy consumption model, and a privacy model; a second construction subunit, configured to construct an optimization objective by combining the delay model, the energy consumption model, and the privacy model; a third construction subunit, configured to establish a Markov decision model for the optimization objective, and construct the task offloading strategy generation model based on the Markov decision model and the deep deterministic policy gradient algorithm.

[0018] Further, the first construction subunit includes: a first definition module, configured to define, for each offloaded computing task, the data reception delay for the target cloud platform to offload the task to the target edge node; a second definition module, configured to define, for each offloaded computing task, the computing delay for the edge node to execute the computing task; a first acquisition module, configured to obtain the completion delay of each offloaded computing task based on the data reception delay and the computing delay; a first acting module, configured to obtain the maximum value of all completion delays during the task offloading process, and use the maximum value of the completion delay as the task offloading delay of the target cloud platform; a first determination module, configured to determine the delay for the target cloud platform to execute the local computing task to obtain the computing task execution delay; a second determination module, configured to determine the total delay of the task offloading process of the target cloud platform and each edge node based on the task offloading delay of the target cloud platform and the computing task execution delay; a first construction module, configured to construct the delay model of the target system based on the total delay of all cloud platforms during the task offloading process.

[0019] Furthermore, the first construction subunit further includes: a third definition module, configured to define the communication resource consumption of the target system during the task offloading process, where the communication resource consumption includes the amount of transmitted data and the bandwidth requirement; a fourth definition module, configured to define the computing resource consumption of the target system during the task offloading process, where the computing resource consumption includes the computing resource requirement and the computing frequency; a second construction module, configured to construct the energy consumption model based on the communication resource consumption and the computing resource consumption.

[0020] Furthermore, the first construction subunit includes: a fifth definition module, configured to define the task type of each offloaded computing task during the task offloading process; a second determination module, configured to determine the proportion of each offloaded computing task in all offloaded computing tasks based on the task type of each offloaded computing task, to obtain the task proportion; a third construction module, configured to calculate the privacy entropy of the target cloud platform for the target edge node based on the privacy proportion of the offloaded computing tasks, and construct the privacy model of the target system based on the privacy entropy, where the privacy entropy is used to measure the possibility that the target edge node obtains the privacy information of the target cloud platform from the computing tasks offloaded from the target cloud platform.

[0021] Furthermore, the second construction subunit includes: a first configuration module, configured to configure weight factors for the delay model, the energy consumption model, and the privacy model respectively; a first establishment module, configured to establish the loss function of the target system based on the delay model and the corresponding weight factor, the energy consumption model and the corresponding weight factor, and the privacy model and the corresponding weight factor; a fourth construction module, configured to construct an optimization objective based on the loss function, where the optimization objective is expressed as: (P1): C1: C2: (P1) indicates that the optimization objective is to minimize the loss function, G(t) is the loss function of the target system, C1 is the first constraint function, C2 is the second constraint function, and tgt(t) represents the task offloading strategy. represents the computing resource of the computing task x offloaded by the edge node i to the cloud platform k, f i m represents the maximum computing resource quantity owned by the edge node i. represents the sum of the computing resources of the computing tasks offloaded by the edge node i to all cloud platforms.

[0022] Further, the fourth construction module includes: a first definition sub-module, configured to define the state information and task information of the target system as the state space of the Markov decision model, and define the optimization objective of the target system and the task offloading policy as the action space of the Markov decision model, and define the negative value of the loss function as the system reward of the Markov decision model; a first training sub-module, configured to use the state space of the Markov decision model as the input data of the deep deterministic policy gradient algorithm, and use the action space of the Markov decision model as the output data of the deep deterministic policy gradient algorithm, and use the system reward of the Markov decision model as the basis for decision evaluation and model parameter update during the iterative calculation process of the deep deterministic policy gradient algorithm, and obtain a task offloading policy generation model through iterative training.

[0023] Further, the first training sub-module is further configured to perform Step 1: normalize the state space, and input the normalized state space into the decision network of the deep deterministic policy gradient algorithm framework, and output the action space through the decision network; Step 2: add noise to the output action space, and input the normalized state space and the action space with added noise into the evaluation network of the deep deterministic policy gradient algorithm framework, and output the system reward; Step 3: construct a state transition matrix based on the normalized state space, the action space with added noise, and the system reward, and store the state transition matrix in the experience replay pool; Step 4: repeat the above Steps 1 to 3, and when the data volume of the state transition matrix in the experience replay pool is greater than a preset data volume threshold, randomly select a batch of state transition matrices from the experience replay pool, and use the selected state transition matrices as training samples to input into the target decision network and target evaluation network of the deep deterministic policy gradient algorithm framework, and output the target action space and the target system reward, where a batch of state transition matrices includes N state transition matrices, and N is a positive integer; repeat the above Step 4 for iterative training, and when the number of iterations reaches the iteration number threshold, stop the iteration, and obtain the trained task offloading policy generation model.

[0024] According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including one or more processors and a memory, where the memory is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the above-mentioned method for generating a task offloading policy based on privacy awareness.

[0025] In this application, through the following steps: first, collect the status information and task information of the target system, where the target system includes multiple cloud platforms and multiple edge nodes, then preprocess the status information and task information to obtain a status feature vector and a task feature vector, and finally input the status feature vector and the task features into a task offloading policy generation model to output the task offloading policy of the target system. The task offloading policy generation model is a model constructed based on a privacy awareness mechanism. The task offloading policy is used to indicate transferring the computing tasks of the cloud platform to the edge nodes for execution. The task offloading policy includes an offloading policy and a resource allocation policy. The offloading policy is used to indicate the offloading method and offloading time of the computing tasks, and the resource allocation policy is used to indicate the allocation method and resource allocation amount for allocating resources to the offloaded computing tasks.

[0026] In this application, considering the problem of privacy leakage in the task offloading process, a task offloading policy generation model based on privacy awareness is constructed. By analyzing the status information and task information of the target system through the pre-constructed task offloading policy generation model to generate the best task offloading policy for task offloading, it is possible to seek the optimal solution for task offloading on the basis of improving data security, achieving the purpose of ensuring data security in the task offloading process, obtaining the technical effect of enhancing data security, and thus solving the technical problem of relatively low data security in the related art when it comes to collaborative computing of cloud platforms and edge nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0028] Figure 1 is a flowchart of an optional method for generating a task offloading policy based on privacy awareness according to an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of an optional generation process of a task offloading policy based on privacy awareness according to an embodiment of the present invention;

[0030] Figure 3 is a schematic diagram of an optional device for generating a task offloading policy based on privacy awareness according to an embodiment of the present invention;

[0031] Figure 4 is a hardware structure block diagram of an electronic device (or mobile device) for executing the method for generating a task offloading policy based on privacy awareness according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts shall fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] It should be noted that the method and device for generating a task offloading strategy based on privacy awareness in this application can be used in the field of information security. In the case of generating a task offloading strategy based on privacy awareness, it can also be used in any field other than the field of information security. The application field of the method and device for generating a task offloading strategy based on privacy awareness in this application is not limited.

[0035] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties. And the processing of the relevant data, such as collection, storage, use, processing, transmission, provision, disclosure and application, all comply with the relevant laws, regulations and standards of the relevant region, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set between this system and relevant users or institutions. Before obtaining relevant information, a request for obtaining information needs to be sent to the aforementioned users or institutions through the interface, and after receiving the consent information feedback from the aforementioned users or institutions, the relevant information is obtained.

[0036] It should be noted that when collecting and analyzing customer information in this application, a corresponding operation entrance is provided for users to choose to agree or refuse the automated decision result; if the user chooses to refuse, the expert decision-making process will be entered.

[0037] The following embodiments of the present invention can be applied to various systems / applications / devices for generating privacy-aware task offloading strategies. First, mathematical models are established for various behaviors and overheads during the interaction between the edge node and the cloud platform, including a latency model, a resource consumption model, and a privacy model. Based on these models, an optimization objective is constructed to train a task offloading strategy generation model, reducing the latency and resource consumption of task completion, and on this basis, improving the privacy protection level. The task offloading strategy generated by the trained task offloading strategy generation model can find the optimal task offloading strategy while ensuring system privacy and reducing system losses.

[0038] The present invention will be described in detail below with reference to each embodiment.

[0039] Embodiment 1

[0040] According to an embodiment of the present invention, an embodiment of a method for generating a privacy-aware task offloading strategy is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0041] Figure 1 is a flowchart of an optional method for generating a privacy-aware task offloading strategy according to an embodiment of the present invention, as Figure 1 shown, the method includes the following steps:

[0042] Step S101, collect the state information and task information of the target system.

[0043] In some embodiments, when the cloud platform needs to execute a large number of complex computing tasks, these tasks may consume a large amount of resources, or data transmission may cause a long latency. At this time, if the computing power of the cloud platform itself is not sufficient to quickly process these tasks, or the latency for processing these tasks is long, then these tasks can be "offloaded" to the edge node for processing.

[0044] Task offloading refers to the process of transferring computational tasks from one computing device (such as a cloud platform) to another computing device (such as an edge node) for processing. In the scenarios of cloud computing and edge computing, such offloading can improve computational efficiency, reduce latency, and in some cases save energy. Edge nodes can be routers, switches, base stations, small servers, or even some high-performance Internet of Things devices. They are usually deployed at the edge of the network, i.e., near the place where data is generated. The main functions of edge nodes include data processing, storage, preprocessing, filtering, and analysis, etc., which can reduce the need to transmit a large amount of raw data to the cloud, and only transmit the processed data or the data that must be processed by the cloud. Compared with the cloud platform, edge nodes can not only supplement the problem of insufficient computing power of the cloud platform, but also bring less latency. However, due to the additional latency and energy consumption in the interaction process between the cloud platform and edge nodes, it is necessary to intelligently determine which tasks should be offloaded and how to allocate these tasks to ensure the optimal efficiency and resource utilization of the entire system, while effectively protecting the data security on the cloud platform.

[0045] Figure 2 It is a schematic diagram of the generation process of an optional privacy-aware task offloading strategy according to an embodiment of the present invention, as Figure 2 shown. The generation process of the privacy-aware task offloading strategy specifically includes:

[0046] Mathematical modeling, constructing three mathematical models, namely a latency model, a resource consumption model, and a privacy model, based on the interaction process between the cloud platform and edge nodes;

[0047] Model training, constructing a privacy-aware task offloading strategy generation model through daddy training by means of a Markov decision model and a deep deterministic policy gradient algorithm;

[0048] Real-time decision-making, collecting environmental information, including status information and task information, and then inputting it into the task offloading strategy generation model, and making decisions through the model to generate a task offloading strategy.

[0049] In the above step S101, the target system includes multiple cloud platforms and multiple edge nodes. The status information includes, but is not limited to, the current resource usage of the edge nodes and cloud platforms (such as CPU usage rate, remaining storage, available bandwidth, etc.), network status (such as transmission rate, latency, etc.), and geographical location information of the edge nodes. This information is crucial for evaluating the real-time performance of the system and for making effective resource allocation. For example, by monitoring the CPU usage rate and remaining storage of an edge node, it can be determined whether the edge node has the ability to handle additional tasks; and understanding the network status helps predict the latency of task transmission and reception, thus more accurately estimating the task completion time. Task information refers to the specific attributes of the computing tasks to be processed in the target system, including, but not limited to, the data volume of the task, computing requirements (such as the number of CPU cycles), task type, and priority. This information helps the algorithm evaluate the importance of the task and the required resources, so as to formulate the most suitable offloading strategy for each task. For example, tasks with a large data volume may be more suitable for local processing on the cloud platform to utilize its high-speed network transmission capabilities and powerful computing resources; while compute-intensive tasks may be more suitable for offloading to edge nodes for processing to reduce data transmission latency.

[0050] Through the collection of the above status information and task information, the embodiments of the present invention can comprehensively consider the current state of the system and the characteristics of the tasks, so as to make the best task offloading decision in each time slot, improve the resource utilization efficiency and task processing efficiency, and on this basis ensure the data security of the cloud platform.

[0051] Step S102: Preprocess the status information and task information to obtain a status feature vector and a task feature vector.

[0052] In the above step S102, when the status information and task information of the target system in time slot t are obtained, it is first necessary to preprocess the information. The preprocessing operations specifically include data cleaning, data normalization, feature extraction, and feature encoding, etc., to convert the original status information and task information with inconsistent possible formats into a unified representation form that the model can recognize and process, namely the status feature vector and the task feature vector.

[0053] Specifically, data cleaning is to fill in or remove missing values from the status information and task information, remove data noise, etc., and then perform data normalization on the cleaned information to ensure that the formats of all input information are consistent, which is convenient for algorithm processing and analysis. Whether the information comes from multiple cloud platforms or multiple edge nodes, the preprocessed feature vectors will be presented in a unified form, eliminating the difference in data formats.

[0054] After data standardization, key features need to be extracted from the status information and task information. These features will directly affect the task offloading decision. For example, CPU usage, memory utilization, network bandwidth, task data volume, computing requirements, etc. are all key factors determining whether a task should be offloaded and to which computing entity. Finally, the features are encoded to obtain the status feature vector and task feature vector that the model can process.

[0055] Step S103: Input the status feature vector and task features into the task offloading policy generation model, and output the task offloading policy of the target system.

[0056] In the above step S103, the status feature vector and task features are used as input data and input into the pre-constructed task offloading policy generation model. The task offloading policy generation model is a model constructed based on the privacy awareness mechanism. By executing dynamic environment perception and intelligent decision-making through the task offloading policy generation model, it ensures that the task offloading policy can not only adapt to system resource changes but also meet privacy protection requirements, thus improving the overall system efficiency.

[0057] In addition, the above task offloading policy is used to indicate the transfer of computing tasks on the cloud platform to the edge node for execution. The task offloading policy includes an offloading policy and a resource allocation policy. Among them, the offloading policy is used to indicate the offloading method and offloading time of the computing task. For example, which computing task on which cloud platform is offloaded to which edge node. The resource allocation policy is used to indicate the allocation method and resource allocation amount for resource allocation for the offloaded computing task. For example, how much bandwidth or CPU cycle number is allocated for the offloaded computing task.

[0058] Furthermore, the steps for constructing the task offloading policy generation model include: modeling the task offloading process between each cloud platform and each edge node in the target system, constructing a delay model, an energy consumption model, and a privacy model; combining the delay model, the energy consumption model, and the privacy model to construct an optimization objective; establishing a Markov decision model for the optimization objective, and constructing a task offloading policy generation model based on the Markov decision model and the deep deterministic policy gradient algorithm.

[0059] In the embodiments of the present invention, when constructing a task offloading policy generation model, the dynamic interaction between each cloud platform and each edge node in the target system is deeply considered. By constructing a latency model, an energy consumption model, and a privacy model, the optimization objectives are clarified. Finally, based on the Markov decision model and the deep deterministic policy gradient algorithm, a model capable of intelligently making decisions on task offloading policies is constructed. The latency model is used to quantify the time required for a computing task to be completed under different offloading paths, helping to evaluate the system response speed and efficiency. The energy consumption model is used to evaluate the impact of different task offloading decisions on the system energy consumption to achieve reasonable utilization of resources. The privacy model is used to quantify the degree of protection of user privacy during the task offloading process to ensure the security of user data on the cloud platform while providing efficient services. Combining latency, energy consumption, and privacy protection, the comprehensive objective to be optimized during algorithm decision-making is clarified.

[0060] Further, the steps for constructing the latency model include: for each offloaded computing task, defining the data reception latency for the target cloud platform to offload the task to the target edge node; for each offloaded computing task, defining the computing latency for the edge node to execute the computing task; obtaining the completion latency for each offloaded computing task based on the data reception latency and the computing latency; obtaining the maximum value of all completion latencies during the task offloading process, and taking the maximum value of the completion latency as the task offloading latency of the target cloud platform; determining the latency for the target cloud platform to execute the local computing task to obtain the computing task execution latency; based on the task offloading latency of the target cloud platform and the computing task execution latency, determining the total latency for the target cloud platform and each edge node to execute the task offloading process; constructing the latency model of the target system based on the total latency for all cloud platforms to execute the task offloading process.

[0061] In the embodiments of the present invention, it is assumed that tasks are randomly offloaded to the cloud or edge nodes. On this basis, for cloud platform k, when it chooses to offload some computing tasks to the edge node for execution, these offloaded computing tasks will be executed in parallel with the local tasks. At this time, the latency for cloud platform k to complete all computing tasks will be divided into the task offloading latency and the computing task execution latency. Finally, the overall latency for cloud platform k to complete the task will be determined by the maximum value of the two.

[0062] Specifically, for each offloaded computing task, its task offloading latency includes the data reception latency during the task transmission process and the computing latency for executing the computing task after the task is offloaded to the edge node. First, define the data reception latency for the target cloud platform to offload the task to the edge node. At time slot t, the data transmission latency for cloud platform k to transmit computing task x to target edge node i is related to the data size of the computing task and the data transmission rate of the transmission channel. Define the data reception latency, and the calculation formula for the data reception latency is obtained as: Where Denote the data volume of task x. Denote the rate at which cloud platform k transfers the data of task x to edge node i.

[0063] When edge node i executes the offloaded computing task x, the computing delay for executing this computing task will be related to the computing resources allocated by edge node i to task x. Define the computing delay for an edge node to execute a computing task, and the formula for the computing delay is expressed as: Among them, Denote the number of CPU cycles required to complete computing task x. Denote the amount of computing resources allocated by edge node i to computing task x.

[0064] The completion delay of computing task x is jointly determined by the data reception delay and the computing delay. Combine the two parts of the delay to obtain the completion delay of computing task x, and the formula for the completion delay is expressed as: Among them, Is the completion delay of computing task x.

[0065] Since the tasks offloaded by cloud platform k to the edge nodes are executed in parallel, the value of the offloading task completion delay of cloud platform k will be determined by the maximum completion delay among all the computing tasks in the offloading task set Thereby obtaining the task offloading delay of the target cloud platform, and its formula is expressed as: Among them, T_off k (t) is the task offloading delay of target cloud platform k.

[0066] Furthermore, the computing task execution delay is the time taken for target cloud platform k to complete all local computing tasks. When cloud platform k chooses to execute computing task x locally, the computing delay will be related to the computing power of cloud platform k itself. Define the expression for the computing task execution delay as: Among them, T_loc k (t) represents the computing task execution delay of cloud platform k, and f k Represents the computing resources of cloud platform k.

[0067] Since the computing tasks are executed in parallel locally and at the edge nodes, the overall delay of cloud platform k will take the maximum value between the task offloading delay and the computing task execution delay, that is, Uplatform_T k (t) = max{T_loc k (t), T_off k (t)}, where Uplatform_T k (t) represents the total delay of target cloud platform k during the task offloading process.

[0068] On this basis, a delay model of the target system is constructed based on the total delay of the task offloading process executed on all cloud platforms, and where T(t) is the total delay of the target system for task offloading within time slot t.

[0069] The construction of the delay model provides a means to quantify the system delay for the algorithm, helps to evaluate the impact of different offloading strategies on the system response time, and thus optimizes resource allocation while ensuring system performance.

[0070] Furthermore, the steps for constructing the energy consumption model include: defining the communication resource consumption of the target system during the task offloading process, where the communication resource consumption includes the amount of data transmitted and the bandwidth requirement; defining the computing resource consumption of the target system during the task offloading process, where the computing resource consumption includes the computing resource requirement and the computing frequency; and constructing the energy consumption model based on the communication resource consumption and the computing resource consumption.

[0071] In this embodiment, a mathematical model of energy consumption is constructed through the communication resource consumption during the task offloading process and the computing resource consumption when executing the offloaded computing tasks. Specifically, the communication resource consumption refers to the amount of data transmitted and the bandwidth requirement between the edge node and the cloud platform during the task offloading process. Among them:

[0072] Communication data volume D i,k (t) means that assuming at time slot t, edge node i needs to transmit task data D i,k (t) to the cloud platform. The task data volume depends on the execution methods of task x at edge node i and cloud platform k, and is defined as: where, and are optimization decision variables that determine whether the task is offloaded to the edge node or executed on the cloud platform. D raw,i,k (t) is the original data volume of the task, representing the data volume before the computing task. D mid,i,k (t) is the intermediate computing data volume, representing the data volume when part of the task is executed at the edge node and part at the cloud platform. D res,i,k (t) is the final computing result data volume of the task, representing the data volume of the final computing result of the computing task that needs to be stored or transmitted to other computing nodes.

[0073] Bandwidth requirement B i,k (t) refers to the bandwidth required during the task transmission to ensure that the data can be transmitted within a certain time. The bandwidth requirement is defined as: where T transIt is the time to complete data transmission within time slot t set according to the amount of transmitted data. The bandwidth requirement between the edge node and the cloud platform should meet its maximum available bandwidth to ensure that there is no problem of bandwidth overload.

[0074] Furthermore, the computing resource consumption refers to the computing power required to process tasks on the edge node or the cloud platform, including the computing resource demand (i.e., the number of CPU cycles) and the computing frequency. Among them:

[0075] Computing resource demand represents the number of CPU cycles required for edge node i or cloud platform k to execute task x at time t. Different tasks have different requirements for computing resources, and the required number of cycles can be determined according to the complexity of the tasks.

[0076] Computing frequency F i,k (t) refers to the computing frequency allocated by the edge node or the cloud platform to computing task x.

[0077] Given the number of CPU cycles The time required for task execution can be expressed as: and where, represents the computing time of the computing task on edge node i, represents the computing time of the computing task on cloud k, and represent the computing frequencies of the edge node and the cloud platform respectively. represents the number of CPU cycles required for computing task x. In addition, the total computing resource demand of the edge node and the cloud platform cannot exceed their maximum computing resources.

[0078] Finally, by combining the communication resource consumption in the offloading process and the computing resource consumption when executing the offloaded computing tasks, a mathematical model of energy consumption is constructed. The mathematical model of energy consumption can be expressed as: R total (t) = R comm (t) + R comp (t), where R total (t) represents the energy consumption of the target system during task offloading. The communication resource consumption R comm (t) is measured by the amount of transmitted data and the bandwidth requirement: where, represents the offloading task set of cloud platform k and edge node i at the current time t. The computing resource consumption R comp (t) is measured by the computing demand and the computing frequency: where, represents the computing frequency of edge node i or cloud platform k at time t, For calculating the computing resource requirements.

[0079] The energy consumption model can accurately quantify the resource consumption of the target system during the task offloading process. It not only focuses on the computing resource requirements during the execution of a single task but also considers the impact of task data transmission on the network communication capacity. The construction of the energy consumption model provides a solid data foundation for formulating efficient and economical offloading strategies, contributing to the optimal allocation and utilization of resources in the cloud-edge collaborative computing scenario.

[0080] Furthermore, the steps for constructing the privacy model include: obtaining the task type of each offloaded computing task during the task offloading process; determining the proportion of each offloaded computing task among all offloaded computing tasks based on the task type of each offloaded computing task to obtain the task proportion; calculating the privacy entropy of the target cloud platform for the target edge node based on the task proportion of the offloaded computing task, and constructing the privacy model of the target system based on the privacy entropy, where the privacy entropy is used to measure the possibility of the target edge node obtaining the privacy information of the target cloud platform from the computing tasks offloaded from the target cloud platform.

[0081] The embodiment of the present invention proposes a privacy entropy model that can quantify privacy. The privacy entropy is used to quantify the privacy protection level and describes the degree of uncertainty of the occurrence of a random event, that is, the possibility of the edge node obtaining the privacy of the cloud platform from the tasks offloaded from the cloud platform. When the privacy entropy is smaller, the privacy of the cloud platform is more likely to be leaked, and vice versa.

[0082] When constructing the privacy model, define the tasks offloaded from cloud platform k to edge node i at time slot t and the proportion of each type of task as: Where represents the proportion of computing task x among all computing tasks offloaded to edge node i, which can be defined by the task type, that is, determine the proportion of each offloaded computing task among all offloaded computing tasks based on the task type of each offloaded computing task to obtain the task proportion, and the task type is the privacy level type of this task. Satisfy

[0083]

[0084] In this case, for edge node i, the privacy entropy of cloud platform k can be expressed as: P i,k (t) has a smaller value, which indicates that for cloud platform k, the possibility of the exposure of its location privacy and usage pattern privacy is greater, and vice versa, indicating a better privacy protection effect.

[0085] The privacy entropy of the target system can thus be obtained as the sum of the privacy entropies of all cloud platforms, and the privacy model of the target system can be defined as follows: where PL(t) is the privacy entropy of the target system.

[0086] Furthermore, the steps of constructing the optimization objective by combining the delay model, energy consumption model, and privacy model include: configuring weight factors for the delay model, energy consumption model, and privacy model respectively; establishing the loss function of the target system based on the delay model and the corresponding weight factor, energy consumption model and the corresponding weight factor, and privacy model and the corresponding weight factor; constructing the optimization objective based on the loss function, where the optimization objective is expressed as: (P1): C1: C2: (P1) indicates that the optimization objective is to minimize the loss function, G(t) is the loss function of the target system, C1 is the first constraint function, C2 is the second constraint function, and tgt(t) represents the task offloading strategy. represents the computing resource of the computing task x assigned by edge node i to cloud platform k for offloading, f i m represents the maximum computing resource quantity owned by edge node i. represents the sum of the computing resources of the computing tasks assigned by edge node i to all cloud platforms for offloading.

[0087] In the embodiments of the present invention, after constructing the delay model, energy consumption model, and privacy model, a loss function can be constructed according to multiple mathematical models, so as to transform the goal of reducing the delay and resource consumption of completing the offloading task and improving the privacy protection level on this basis into a problem of minimizing the solution.

[0088] Specifically, weight factors are configured for the delay model, energy consumption model, and privacy model according to their importance in offloading strategy generation, and then the loss function of the target system is established based on the delay model and the corresponding weight factor, energy consumption model and the corresponding weight factor, and privacy model and the corresponding weight factor respectively: G(t) = aR total (t) + bT(t) + cPL(t), where a, b, and c respectively represent the weight factors of delay, resource consumption, and privacy protection level in the loss function, and a + b + c = 1. The greater the weight factor, the greater the influence of the corresponding term on the system loss function, and the smaller the weight factor, the smaller the influence on the system loss function.

[0089] Then, based on the loss function, the optimization objective is transformed into a problem of minimizing the loss function, and thus the optimization objective is obtained:

[0090] (P1):

[0091] C1:

[0092] C2:

[0093] In the above optimization objectives, (P1) indicates that the optimization objective is to minimize the loss function, G(t) is the loss function of the target system, C1 is the first constraint function, C2 is the second constraint function, C1 and C2 constrain the allocation of computing resources of edge nodes, and tgt(t) represents the task offloading strategy. represents the computing resources of the computing task x offloaded from edge node i to cloud platform k, f i m represents the maximum amount of computing resources owned by edge node i. represents the sum of the computing resources of the computing tasks offloaded from edge node i to all cloud platforms.

[0094] Furthermore, the steps of establishing a Markov decision model for this optimization objective and constructing a task offloading strategy generation model based on the Markov decision model and the deep deterministic policy gradient algorithm include: defining the state information and task information of the target system as the state space of the Markov decision model, and defining the optimization objective and task offloading strategy of the target system as the action space of the Markov decision model, defining the negative value of the loss function as the system reward of the Markov decision model; using the state space of the Markov decision model as the input data of the deep deterministic policy gradient algorithm, and using the action space of the Markov decision model as the output data of the deep deterministic policy gradient algorithm, and using the system reward of the Markov decision model as the basis for decision evaluation and model parameter update during the iterative calculation process of the deep deterministic policy gradient algorithm, and obtaining the task offloading strategy generation model through iterative training.

[0095] In the embodiments of the present invention, after determining the optimization objective in the process of generating the offloading task strategy, a Markov decision model and a deep deterministic policy gradient algorithm are selected for coordinated calculation, and a privacy-aware task offloading strategy generation model is obtained through multiple iterative trainings. Specifically, first, it is necessary to define the state space, action space, and system reward of the Markov decision model. Specifically, the state space describes the variables of the environment where the agent is currently located, and is composed of the state information and task information of each computing node in the target system, and this state space will determine the action of the agent at the next moment. The state space at time slot t can be expressed as: S(t) = {T k (t), R k (t), C k (t), H k (t)}, where, T k(t) is the task information, representing the set of computing tasks at time t, R k (t) represents the data requirements of the task, such as data volume and bandwidth requirements, C k (t) represents the computing requirements, such as the required CPU cycles, H k (t) represents the state information of the target system, including the current computing resource distribution and usage of the target system.

[0096] The action space is the action made by the agent based on the state space of this time slot, that is, the task offloading strategy at time slot t. The action space is represented as: A(t) = {tgt(t)}, where tgt(t) represents the task offloading strategy. That is, it decides which computing tasks should be offloaded to which edge node or cloud platform.

[0097] The system reward is used to describe the feedback reward obtained by the agent after taking a certain action. From the description of the optimization problem, it can be seen that when the loss of the target system is smaller, it means that the overall energy consumption and delay of the system are smaller, and the privacy protection level is higher. Therefore, the reward function is composed of the negative value of the loss function, indicating that when the offloading strategy is better, the total revenue of the system is higher. Thus, the system reward is obtained as: R(t) = -G(t).

[0098] Furthermore, after determining the state space, action space, and system reward of the Markov decision model, the state space of the Markov decision model is used as the input data of the deep deterministic policy gradient algorithm, and the action space of the Markov decision model is used as the output data of the deep deterministic policy gradient algorithm. The system reward of the Markov decision model is used as the basis for decision evaluation and model parameter update during the iterative calculation process of the deep deterministic policy gradient algorithm. Through iterative training, a task offloading strategy generation model based on privacy awareness is constructed.

[0099] Further, the steps of obtaining the task offloading policy generation model through iterative training include: Step 1, normalizing the state space and inputting the normalized state space into the decision network of the deep deterministic policy gradient algorithm framework, and outputting the action space through the decision network; Step 2, adding noise to the output action space, and inputting the normalized state space and the action space with added noise into the evaluation network of the deep deterministic policy gradient algorithm framework to output the system reward; Step 3, constructing a state transition matrix based on the normalized state space, the action space with added noise, and the system reward, and storing the state transition matrix in the experience replay pool; Step 4, repeating the above Steps 1 to 3. When the amount of state transition matrix data in the experience replay pool is greater than the preset data volume threshold, randomly select a batch of state transition matrices from the experience replay pool, and use the selected state transition matrices as training samples to input into the target decision network and target evaluation network of the deep deterministic policy gradient algorithm framework, and output the target action space and target system reward. Among them, a batch of state transition matrices contains N state transition matrices, and N is a positive integer; repeat the above Step 4 for iterative training. When the number of iterations reaches the iteration number threshold, stop the iteration to obtain the trained task offloading policy generation model.

[0100] In the embodiment of the present invention, the deep deterministic policy gradient algorithm framework includes four networks, namely the Actor network for real-time interaction (corresponding to the above decision network) μ(s|θ μ ) and the Critic network (corresponding to the above evaluation network) Q(s,a|θ Q ), the target network target Actor network (corresponding to the above target decision network) μ′(s′|θ μ′ ), and the target Critic network (corresponding to the above target evaluation network) Q′(s′,a′|θ Q′ ). In addition, the framework also includes an experience replay pool and a noise mechanism.

[0101] Through the deep deterministic policy gradient algorithm, first normalize the input state space s t , and then input it into the Actor network to output the action space a t = μ(s t ). At the same time, in order to ensure sufficient exploration of the state space, appropriate noise n i will be added to the action space. This noise conforms to the normal distribution so that the final output becomes a t = μ(s t ) + n i . Input the generated action space and state space into the Critic network together to obtain the system reward rt .

[0102] Subsequently, the state transition matrix (s t , a t , r t , s t+1 ) at this moment is stored in the experience replay pool. When the amount of state transition data in the experience replay pool reaches the threshold, a batch (including multiple state transition matrices) of state transitions will be randomly sampled from the replay pool for replay, and it is input into the target Actor network and the target Critic network to sequentially obtain the corresponding target action space (i.e., the optimized offloading strategy) and the target system reward.

[0103] Repeat the above process for iterative training, continuously optimize the task offloading strategy through the deep deterministic policy gradient algorithm until the number of iterations reaches the iteration number threshold, and stop the iteration. To minimize the loss function, during the iteration process, the parameters of the four networks will be updated after a certain time.

[0104] Through the above steps, first collect the state information and task information of the target system, where the target system includes multiple cloud platforms and multiple edge nodes, then preprocess the state information and task information to obtain the state feature vector and the task feature vector, and finally input the state feature vector and the task into the task offloading strategy generation model to output the task offloading strategy of the target system. Among them, the task offloading strategy generation model is a model constructed based on the privacy awareness mechanism, and the task offloading strategy is used to indicate that the computing tasks of the cloud platform are transferred to the edge nodes for execution. The task offloading strategy includes an offloading strategy and a resource allocation strategy. Among them, the offloading strategy is used to indicate the offloading method and offloading time of the computing task, and the resource allocation strategy is used to indicate the allocation method and resource allocation amount for allocating resources to the offloaded computing task.

[0105] In this embodiment, considering the problem of privacy leakage in the task offloading process, a task offloading strategy generation model based on privacy awareness is constructed. By analyzing the state information and task information of the target system through the pre-constructed task offloading strategy generation model to generate the best task offloading strategy for task offloading, it can seek the optimal solution for task offloading on the basis of improving data security, achieving the purpose of ensuring data security in the task offloading process, obtaining the technical effect of improving data security, and thus solving the technical problem of low data security in the related technology when it comes to collaborative computing of cloud platforms and edge nodes.

[0106] The following is a detailed description in combination with another embodiment.

[0107] Embodiment 2

[0108] The generation device of a privacy-aware task offloading strategy provided in this embodiment includes multiple implementation units, and each implementation unit corresponds to each implementation step in the first embodiment above. The specific implementation manner and beneficial effects can be referred to the foregoing method embodiment and will not be elaborated here.

[0109] Figure 3 is a schematic diagram of an optional generation device of a privacy-aware task offloading strategy according to an embodiment of the present invention. As Figure 3 shown, the generation device of the privacy-aware task offloading strategy may include: an acquisition unit 31, a preprocessing unit 32, and an output unit 33. Among them,

[0110] The acquisition unit 31 is configured to acquire the status information and task information of the target system, where the target system includes multiple cloud platforms and multiple edge nodes;

[0111] The preprocessing unit 32 is configured to preprocess the status information and task information to obtain a status feature vector and a task feature vector;

[0112] The output unit 33 is configured to input the status feature vector and the task feature into a task offloading strategy generation model, and output the task offloading strategy of the target system. The task offloading strategy generation model is a model constructed based on a privacy awareness mechanism. The task offloading strategy is used to indicate that the computing tasks of the cloud platform are transferred to the edge nodes for execution. The task offloading strategy includes an offloading strategy and a resource allocation strategy. The offloading strategy is used to indicate the offloading method and offloading time of the computing tasks, and the resource allocation strategy is used to indicate the allocation method and resource allocation amount for allocating resources to the offloaded computing tasks.

[0113] For the above-mentioned generation device of the privacy-aware task offloading strategy, the acquisition unit 31 acquires the status information and task information of the target system, where the target system includes multiple cloud platforms and multiple edge nodes; the preprocessing unit 32 preprocesses the status information and task information to obtain a status feature vector and a task feature vector; the output unit 33 inputs the status feature vector and the task feature into a task offloading strategy generation model, and outputs the task offloading strategy of the target system. The task offloading strategy generation model is a model constructed based on a privacy awareness mechanism. The task offloading strategy is used to indicate that the computing tasks of the cloud platform are transferred to the edge nodes for execution. The task offloading strategy includes an offloading strategy and a resource allocation strategy. The offloading strategy is used to indicate the offloading method and offloading time of the computing tasks, and the resource allocation strategy is used to indicate the allocation method and resource allocation amount for allocating resources to the offloaded computing tasks.

[0114] In this embodiment, considering the problem of privacy leakage existing in the task offloading process, a task offloading policy generation model based on privacy awareness is constructed. By analyzing the state information and task information of the target system through the pre-constructed task offloading policy generation model, the best task offloading policy is generated for task offloading, which can seek the optimal solution of task offloading on the basis of improving data security, achieving the purpose of ensuring data security in the task offloading process and obtaining the technical effect of improving data security, thus solving the technical problem of low data security in the related technology when it comes to collaborative computing between the cloud platform and edge nodes.

[0115] Further, the generating device of the task offloading policy based on privacy awareness further includes a construction unit for constructing a task offloading policy generation model. The construction unit includes: a first construction subunit for modeling the task offloading process between each cloud platform and each edge node in the target system to construct a delay model, an energy consumption model, and a privacy model; a second construction subunit for constructing an optimization objective by combining the delay model, the energy consumption model, and the privacy model; and a third construction subunit for establishing a Markov decision model for the optimization objective and constructing a task offloading policy generation model based on the Markov decision model and the deep deterministic policy gradient algorithm.

[0116] Further, the first construction subunit includes: a first definition module for defining, for each offloaded computing task, the data reception delay for the target cloud platform to offload the task to the target edge node; a second definition module for defining, for each offloaded computing task, the computing delay for the edge node to execute the computing task; a first acquisition module for obtaining the completion delay of each offloaded computing task based on the data reception delay and the computing delay; a first acting module for obtaining the maximum value of all completion delays in the task offloading process and taking the maximum value of the completion delays as the task offloading delay of the target cloud platform; a first determination module for determining the delay for the target cloud platform to execute the local computing task to obtain the computing task execution delay; a second determination module for determining the total delay of the target cloud platform and each edge node to execute the task offloading process based on the task offloading delay of the target cloud platform and the computing task execution delay; and a first construction module for constructing a delay model of the target system based on the total delay of all cloud platforms to execute the task offloading process.

[0117] Further, the first construction subunit further includes: a third definition module for defining the communication resource consumption when the target system executes the task offloading process, where the communication resource consumption includes the amount of transmitted data and the bandwidth demand; a fourth definition module for defining the computing resource consumption when the target system executes the task offloading process, where the computing resource consumption includes the computing resource demand and the computing frequency; and a second construction module for constructing an energy consumption model based on the communication resource consumption and the computing resource consumption.

[0118] Further, the first construction subunit includes: a fifth definition module for defining the task type of each offloaded computing task during the task offloading process; a second determination module for determining the proportion of the computing task in all offloaded computing tasks based on the task type of each offloaded computing task, to obtain the task proportion; and a third construction module for calculating the privacy entropy of the target cloud platform for the target edge node based on the privacy proportion of the offloaded computing tasks, and constructing a privacy model of the target system based on the privacy entropy, where the privacy entropy is used to measure the possibility of the target edge node obtaining the privacy information of the target cloud platform from the computing tasks offloaded from the target cloud platform.

[0119] Further, the second construction subunit includes: a first configuration module for configuring weight factors for the delay model, the energy consumption model, and the privacy model respectively; a first establishment module for establishing a loss function of the target system based on the delay model and the corresponding weight factor, the energy consumption model and the corresponding weight factor, and the privacy model and the corresponding weight factor; and a fourth construction module for constructing an optimization objective based on the loss function, where the optimization objective is expressed as: (P1): C1: C2: (P1) indicates that the optimization objective is to minimize the loss function, G(t) is the loss function of the target system, C1 is the first constraint function, C2 is the second constraint function, and tgt(t) represents the task offloading strategy. represents the computing resource of the computing task x offloaded by the edge node i to the cloud platform k, f i m represents the maximum computing resource quantity owned by the edge node i. represents the sum of the computing resources of the computing tasks offloaded by the edge node i to all cloud platforms.

[0120] Further, the fourth construction module includes: a first definition sub-module for defining the state information and task information of the target system as the state space of the Markov decision model, and defining the optimization objective and task offloading strategy of the target system as the action space of the Markov decision model, and defining the negative value of the loss function as the system reward of the Markov decision model; a first training sub-module for using the state space of the Markov decision model as the input data of the deep deterministic policy gradient algorithm, and using the action space of the Markov decision model as the output data of the deep deterministic policy gradient algorithm, and using the system reward of the Markov decision model as the basis for decision evaluation and model parameter update during the iterative calculation process of the deep deterministic policy gradient algorithm, and obtaining a task offloading strategy generation model through iterative training.

[0121] Further, the first training sub-module is further used in Step 1 to normalize the state space and input the normalized state space into the decision network of the deep deterministic policy gradient algorithm framework, and output the action space through the decision network; Step 2, add noise to the output action space, and input the normalized state space and the action space with added noise into the evaluation network of the deep deterministic policy gradient algorithm framework to output the system reward; Step 3, construct a state transition matrix based on the normalized state space, the action space with added noise, and the system reward, and store the state transition matrix in the experience replay pool; Step 4, repeat the above Steps 1 to 3. When the data volume of the state transition matrix in the experience replay pool is greater than the preset data volume threshold, randomly select a batch of state transition matrices from the experience replay pool, and use the selected state transition matrices as training samples to input into the target decision network and the target evaluation network of the deep deterministic policy gradient algorithm framework to output the target action space and the target system reward, where a batch of state transition matrices contains N state transition matrices, and N is a positive integer; repeat the above Step 4 for iterative training. When the number of iterations reaches the iteration number threshold, stop the iteration to obtain the trained task offloading policy generation model.

[0122] The above-mentioned generation device of the task offloading policy based on privacy awareness may further include a processor and a memory. The above-mentioned acquisition unit 31, preprocessing unit 32, output unit 33, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions.

[0123] The above-mentioned processor contains a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and the task offloading policy is generated by adjusting the kernel parameters.

[0124] The above-mentioned memory may include non-permanent memory in a computer-readable medium, forms such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0125] According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium. The computer-readable storage medium includes a stored computer program, wherein when the computer program runs, it controls the device where the computer-readable storage medium is located to execute any one of the above-mentioned methods for generating a task offloading policy based on privacy awareness.

[0126] According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including one or more processors and a memory, where the memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method for generating any of the above privacy-aware task offloading strategies.

[0127] According to another aspect of the embodiments of the present invention, there is also provided a computer program product. The computer program product includes a computer program, where when the computer program is executed by a processor, it implements the method for generating any of the above privacy-aware task offloading strategies.

[0128] The present application also provides a computer program product. When executed on a data processing device, it is adapted to execute a program initialized with the following method steps: collecting status information and task information of a target system, where the target system includes multiple cloud platforms and multiple edge nodes; preprocessing the status information and task information to obtain a status feature vector and a task feature vector; inputting the status feature vector and the task feature into a task offloading strategy generation model to output a task offloading strategy of the target system, where the task offloading strategy generation model is a model constructed based on a privacy awareness mechanism, and the task offloading strategy is used to indicate transferring the computing tasks of the cloud platform to the edge nodes for execution. The task offloading strategy includes an offloading strategy and a resource allocation strategy, where the offloading strategy is used to indicate the offloading method and offloading time of the computing tasks, and the resource allocation strategy is used to indicate the allocation method and resource allocation amount for allocating resources to the offloaded computing tasks.

[0129] Figure 4 is a hardware structure block diagram of an electronic device (or mobile device) for implementing the method for generating a privacy-aware task offloading strategy according to an embodiment of the present invention. As Figure 4 shown, the electronic device may include one or more processors ( Figure 4 denoted as 402a, 402b,..., 402n in [the figure], and the processor may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 404 for storing data. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 4 the structure shown is only schematic and does not limit the structure of the above electronic device. For example, the electronic device may further include more or fewer components than Figure 4 shown in [the figure], or have a different configuration from Figure 4 shown in [the figure].

[0130] The serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0131] In the above embodiments of the present invention, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0132] In the several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the units or modules can be in electrical or other forms.

[0133] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0135] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks or optical disks, and other media that can store program codes.

[0136] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for generating a privacy-aware task offloading strategy, characterized in that: include: Collecting status information and task information of a target system, wherein the target system includes multiple cloud platforms and multiple edge nodes; Preprocessing the state information and task information to obtain a state feature vector and a task feature vector; The state feature vector and the task feature are input into a task offloading strategy generation model, and the task offloading strategy of the target system is output, wherein the task offloading strategy generation model is a model constructed based on a privacy-aware mechanism, and the task offloading strategy is used to indicate that the computing task of the cloud platform is transferred to the edge node for execution, and the task offloading strategy includes an offloading strategy and a resource allocation strategy, wherein the offloading strategy is used to indicate the offloading method and offloading time of the computing task, and the resource allocation strategy is used to indicate the allocation method and resource allocation amount for the offloaded computing task.

2. The method according to claim 1, characterized in that The steps to build a task offloading strategy generation model include: Modeling the task offloading process between each cloud platform and each edge node in the target system, and constructing a delay model, an energy consumption model, and a privacy model; Constructing an optimization target by combining the delay model, the energy consumption model and the privacy model; A Markov decision model is established for the optimization goal, and the task offloading strategy generation model is constructed based on the Markov decision model and a deep deterministic policy gradient algorithm.

3. The method according to claim 2, characterized in that The steps of constructing the delay model include: For each offloaded computing task, define the data reception delay of the target cloud platform to offload the task to the target edge node; For each offloaded computing task, defining a computing delay for the edge node to execute the computing task; Obtaining a completion delay of each offloaded computing task based on the data receiving delay and the computing delay; Obtaining the maximum value of all completion delays during the task offloading process, and using the maximum value of the completion delay as the task offloading delay of the target cloud platform; Determine the latency of the target cloud platform to execute the local computing task, and obtain the computing task execution latency; Based on the task offloading delay of the target cloud platform and the computing task execution delay, determining the total delay of the task offloading process between the target cloud platform and each edge node; A delay model of the target system is constructed based on the total delay of all cloud platforms performing task offloading processes.

4. The method according to claim 2, characterized in that: The steps of constructing the energy consumption model include: Defining the communication resource consumption of the target system when executing the task offloading process, wherein the communication resource consumption includes the amount of transmitted data and the amount of bandwidth required; Defining computing resource consumption of the target system when executing the task offloading process, wherein the computing resource consumption includes computing resource demand and computing frequency; The energy consumption model is constructed based on the communication resource consumption and the computing resource consumption.

5. The method according to claim 2, characterized in that: The steps of constructing the privacy model include: Get the task type of each offloaded computing task during the task offloading process; Determine the proportion of each offloaded computing task in all offloaded computing tasks based on the task type of each offloaded computing task, and obtain the task proportion; The privacy entropy of the target cloud platform for the target edge node is calculated based on the task ratio of the offloaded computing tasks, and a privacy model of the target system is constructed based on the privacy entropy, wherein the privacy entropy is used to measure the possibility of the target edge node obtaining the privacy information of the target cloud platform from the computing tasks offloaded from the target cloud platform.

6. The method according to claim 2, characterized in that The steps of constructing the optimization target by combining the delay model, energy consumption model, and privacy model include: respectively configuring weight factors for the delay model, the energy consumption model, and the privacy model; Establishing a loss function of the target system based on the delay model and the corresponding weight factor, the energy consumption model and the corresponding weight factor, and the privacy model and the corresponding weight factor; An optimization objective is constructed based on the loss function, wherein the optimization objective is expressed as: (P1): C1: C2: (P1) indicates that the optimization objective is to minimize the loss function, G(t) is the loss function of the target system, C1 is the first constraint function, C2 is the second constraint function, tgt(t) indicates the task offloading strategy, represents the computing resources allocated by edge node i to computing task x offloaded by cloud platform k, f i m represents the maximum number of computing resources owned by edge node i, It represents the sum of computing resources allocated by edge node i to all computing tasks offloaded from cloud platforms.

7. The method according to claim 6, characterized in that The steps of establishing a Markov decision model for the optimization goal and constructing the task offloading strategy generation model based on the Markov decision model and the deep deterministic policy gradient algorithm include: The state information and task information of the target system are defined as the state space of the Markov decision model, the optimization target of the target system and the task offloading strategy are defined as the action space of the Markov decision model, and the negative value of the loss function is defined as the system reward of the Markov decision model; The state space of the Markov decision model is used as the input data of the deep deterministic policy gradient algorithm, and the action space of the Markov decision model is used as the output data of the deep deterministic policy gradient algorithm. The system reward of the Markov decision model is used as the basis for decision evaluation and model parameter update during the iterative calculation process of the deep deterministic policy gradient algorithm, and the task offloading policy generation model is obtained through iterative training.

8. The method according to claim 7, characterized in that The steps of obtaining the task offloading strategy generation model through iterative training include: Step 1: normalize the state space, and input the normalized state space into the decision network of the deep deterministic policy gradient algorithm framework, and output the action space through the decision network; Step 2: adding noise to the output action space, and inputting the normalized state space and the action space with added noise into the evaluation network of the deep deterministic policy gradient algorithm framework, and outputting the system reward; Step 3: construct a state transfer matrix based on the normalized state space, the action space with added noise, and the system reward, and store the state transfer matrix in the experience replay pool; Step 4, repeating the above steps 1 to 3, when the amount of state transfer matrix data in the experience replay pool is greater than the preset data amount threshold, randomly selecting a batch of state transfer matrices from the experience replay pool, and inputting the selected state transfer matrices as training samples into the target decision network and target evaluation network of the deep deterministic policy gradient algorithm framework, outputting the target action space and the target system reward, wherein a batch of state transfer matrices contains N state transfer matrices, where N is a positive integer; Repeat the above step 4 to perform iterative training. When the number of iterations reaches the iteration number threshold, stop the iteration to obtain the task offloading strategy generation model that has completed the training.

9. A device for generating a privacy-aware task offloading strategy, characterized in that: include: A collection unit, used to collect status information and task information of a target system, wherein the target system includes multiple cloud platforms and multiple edge nodes; A preprocessing unit, used for preprocessing the state information and task information to obtain a state feature vector and a task feature vector; An output unit is used to input the state feature vector and the task feature into a task offloading strategy generation model, and output the task offloading strategy of the target system, wherein the task offloading strategy generation model is a model built based on a privacy-aware mechanism, and the task offloading strategy is used to indicate that the computing task of the cloud platform is transferred to the edge node for execution. The task offloading strategy includes an offloading strategy and a resource allocation strategy, wherein the offloading strategy is used to indicate the offloading method and offloading time of the computing task, and the resource allocation strategy is used to indicate the allocation method and resource allocation amount for the offloaded computing task.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for generating a privacy-aware task offloading strategy as described in any one of claims 1 to 8.

Images