High-user-satisfaction energy-saving task unloading method based on edge computing

Abstract

The invention belongs to the technical field of intelligent digital computing, and particularly relates to a high-user-satisfaction energy-saving task unloading method based on edge computing. According to the method, intelligent digital twinning of edge calculation enabling is considered to be applied to a user dense area, so that user privacy and low-delay satisfaction are improved, and energy-saving task processing is realized. According to the invention, an intelligent digital twinning low-delay task unloading energy-saving architecture is constructed, and a twinning model is constructed in an edge server by using user information in reality. An intelligent reflecting surface is adopted to enhance communication conditions, and the dynamic voltage frequency adjustment technology of end equipment is considered, so that the energy-saving requirement is better met. Due to the fact that digital twinning needs to obtain a large amount of user data, in order to guarantee the privacy of users using the framework, the user data privacy is achieved through the differential privacy technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent digital computing, and in particular relates to an edge computing-based energy-saving task unloading method with high user satisfaction. Background Art

[0002] In the future, the next generation of communication technology will bring ultra-low latency communication and promote the development of perception and communication integration. While enhancing communication capabilities, synaesthesia integration can better perceive environmental information and obtain environmental data. 6G technology can use its own characteristics as a bridge to connect actual scenes and virtual environments, and realize the combination of virtual and real through intelligent technology. The advancement of synaesthesia integration technology has enhanced more real-time details in the actual application of digital twin technology, but with the development of technology, a large amount of data will inevitably be generated. User devices such as sensors and portable IoT devices can perform simple data processing by themselves, but it is difficult to process with low latency or even impossible to process when the data volume and big data relationship are complex. If the data can process tasks independently with strong computing power, it will lead to data island problems, making it difficult to integrate data from multiple parties to achieve overall data analysis. At the same time, local computing is not conducive to the development of digital twins. Offloading to cloud server computing is currently a better solution to solve limited computing power and various data fusion analysis, and it can also realize the training of intelligent models. However, due to the decentralized distribution characteristics of edge user dense areas, edge computing technology with cloud computing capabilities is a better choice. Its distribution is closer to users in different regions and has lower latency than cloud computing.

[0003] Intelligent digital twins enabled by edge computing can not only realize the virtual mapping of the real world by traditional digital twins, but also use the computing power of edge servers to perform intelligent analysis on twin data. By building a twin control center based on intelligent models in edge servers, predictive analysis can be performed based on historical data, real-time resource scheduling, and task offloading decisions can be issued. Based on the above information and the sensor data provided by 6G interawareness, channel status, user coordinates and other information are analyzed to make real-time decisions. However, cloud servers and edge servers are considered untrustworthy, and the privacy of user data must be guaranteed when building digital twin models in them.

[0004] The intelligent digital twins enabled by edge computing have the ability to mine and analyze user data. In this scenario, differential privacy technology is more widely used than privacy protection methods such as data encryption. Differential privacy has a strong mathematical theoretical basis. It can protect the user's privacy information by adding corresponding noise to the original data to prevent the leakage of sensitive information data during data processing. Although differential privacy has strong privacy protection capabilities, the addition of noise will also reduce the effectiveness of data processing, so it is necessary to weigh data privacy and availability. At the same time, in order to reduce the secondary loss caused by data transmission, multiple intelligent reflective surfaces are used in the research scenario for auxiliary communication. Intelligent reflective surface is one of the key technologies of 6G. It uses a large number of passive reflective elements to adjust the phase to achieve beamforming, which can effectively enhance signal transmission and reduce signal fading. And IRS has lower cost and energy-saving characteristics compared to traditional beamforming technology phased array.

[0005] Data processing not only needs to consider the characteristics of low latency, but also energy sustainability. Transmission and computing power consumption are the two major sources of energy consumption after users generate data. Using IRS in different directions during signal propagation increases the probability of LoS links, reduces signal fading during transmission, and reduces transmission power consumption. Mobile user devices are energy-scarce compared to active edge servers. Currently, existing devices use dynamic voltage and frequency adjustment technology to achieve a balance between energy and computing power. Different devices have different application scenarios and require different computing power. Chips with dynamic voltage and frequency adjustment can dynamically adjust chip frequency and voltage during operation to achieve energy saving goals. Dynamic voltage and frequency scaling (DVFS) is a very important power management method that can adjust the voltage clock frequency of the device's processor to reduce energy consumption according to different device conditions.

[0006] At present, intelligence is gradually penetrating into people's lives, and users generate a large amount of data in real time. It is difficult for existing technologies to meet the requirements of low-latency reliable data processing while achieving energy sustainability. To solve the problem of large user computing in smart cities, the current better solution is to offload computing. However, offloading tasks to edge servers will increase computing time, consume a lot of energy, and make it difficult to protect user data privacy. Summary of the invention

[0007] The present invention proposes a high user satisfaction energy-saving task offloading method based on edge computing, which solves the problem that the existing task offloading methods are difficult to simultaneously meet the requirements of low-latency energy sustainability and reliable data processing. The method in the present invention can be applied to urban user-dense areas, where the user devices that generate tasks are mobile phones, computers, vehicles, smart assistants, etc., or the architecture can be applied to factory manufacturing, where the user devices can be robotic arms, smart cars, smart sensors, etc. During peak hours in these fields, the tasks generated by the devices surge, and it is difficult to ensure efficient and low-energy computing by relying solely on the computing power of the devices themselves. Therefore, it is necessary to design a solution to solve the problem that large tasks are difficult to be calculated in a timely manner, and the method in the present invention can offload tasks to edge servers for computing, while ensuring low-latency power consumption and offloaded data privacy.

[0008] The technical solution of the present invention is achieved in this way:

[0009] A high user satisfaction energy-saving task offloading method based on edge computing, including the following contents: S1, establishing a system model: the task offloading process includes three levels, user layer, communication layer and server layer, the user layer model includes a user delay energy consumption model and a user privacy model, the communication layer includes an IRS model and a delay energy consumption model, and the service layer includes an offloading calculation delay and energy consumption model;

[0010] S2, formulate the problems of reducing latency, energy consumption and protecting user privacy to form an objective function;

[0011] S3. Under the premise of protecting user privacy, with the goal of minimizing latency and energy consumption, an algorithm is proposed to solve the problem, forming the ISES algorithm and obtaining an offloading solution.

[0012] Optionally, in step S1, the content of the user delay energy consumption model is as follows:

[0013] (1) The power consumption of CMOS circuits is mainly composed of dynamic power consumption P sa , static power consumption P sl And short-circuit power consumption P sc Composition, represented by the following formula:

[0014] P cmos =P sa +P sl +P sc ;

[0015] (2) Energy consumption calculation takes into account the dynamic power consumption of the CPU during use;

[0016]

[0017] Among them, f qis the CPU clock frequency of the user device corresponding to the qth task, ρ is the activity factor, and C is the load capacitance; The square of the power supply voltage;

[0018] (3)

[0019] Among them, κ q Determined by chip performance and dynamic effective capacitance;

[0020] (4) The computational size of the task is expressed as the number of CPU cycles c required for the task. q , the number of CPU cycles required for the qth task processing can be calculated by the following formula:

[0021] c q =s q b q ;

[0022] Among them, s q Indicates the task size, b q Indicates the number of CPU cycles required to process a 1-byte task;

[0023] (5) The time required for local calculation of this task is expressed as follows:

[0024]

[0025] Among them, T q is the CPU clock cycle size, which is related to the CPU clock frequency f q The two are reciprocal;

[0026] (6) The energy consumption of this task can be calculated by the following formula:

[0027] E local =P sa t q ;

[0028]

[0029] (7) When the user equipment is unloaded, the reflection elements of the IRS are all used to reflect the signal of the transmitted data. The transmission signal of the qth user equipment in the task queue is expressed as:

[0030]

[0031] Among them, P q ≤P max is the transmission power of the qth user equipment in the task queue, and assumes that S q ~CN(0,1) is a circularly symmetric complex Gaussian random variable with unit variance and mean zero. The signal received by ES from the qth user device in the task queue is expressed as:

[0032] y q =h q X q +n q ;

[0033] Among them, n q It means the mean is zero and the variance is Additive Gaussian white noise, X q Represents the transmission signal of the qth user device in the task queue;

[0034] The signal-to-noise ratio can be expressed as:

[0035]

[0036] (8) According to Shannon's formula, the unloading rate of the qth user device in the task queue can be calculated as:

[0037] R q =BW q log2(1+SINR q );

[0038] Among them, BW q is the channel bandwidth.

[0039] Optionally, in step S1, the content of the user privacy model is as follows:

[0040] (1) Randomization algorithm Can satisfy the following formula:

[0041]

[0042] This means that the randomization algorithm Satisfying ε-differential privacy, the maximum divergence is introduced, where ε represents the privacy budget;

[0043] (2) Given an arbitrary query function Its sensitivity can be expressed as:

[0044]

[0045] Among them, ‖·‖1 represents the l1 norm of the vector, and ΔY represents the maximum impact of any operation on an element in the data set on the data set query;

[0046] (3) For a given query function if:

[0047]

[0048] in, The randomization algorithm The ε-differential privacy of the Laplace mechanism is provided. The function Z obeys the Laplace distribution, and its probability density function is:

[0049]

[0050] in,

[0051] (4) The perturbed data is expressed as follows:

[0052]

[0053] (5) Define the query function as:

[0054]

[0055] Among them, ‖·‖2 represents the l2 norm of the vector, Indicates the location coordinates of devices that can establish channels with users, such as edge servers and IRSs. Indicates the coordinate information of the user device;

[0056] (6) Assuming that the coverage distance of the edge server is K, the user location in the area must be no more than K away from the edge server, that is, This can be used to The global sensitivity of is defined as:

[0057]

[0058] (7) The randomization algorithm is defined as:

[0059]

[0060] in, It follows the Laplace distribution;

[0061] (8) Define the neighboring set of user devices within the edge server coverage K and have:

[0062]

[0063] Optionally, in step S1, after the user layer model is established, data is transmitted from the user layer to the server, passing through the communication layer, thereby solving the energy consumption and delay problems of the communication layer. The communication layer is based on IRS-assisted communication to enhance the channel state. The constructed IRS model is as follows:

[0064] (1) The IRS uses its own controller to control R = l × w passive reflection elements to achieve beamforming. l and w are the number of reflection elements in each row and column of the IRS, respectively. The IRS is the relay of the communication link. The reflection element can adjust the phase and amplitude of the received signal. The passive reflection performs beamforming. Its phase shift reflection coefficient matrix is ​​expressed as follows:

[0065]

[0066] (2) The channel between the qth user in the task queue and the ES is modeled as follows:

[0067]

[0068] in, represents the baseband equivalent channel h from the qth user equipment in the task queue to the IRS q,r =[h 1,1 (q),…,h 1,w (q),…,h l,w (q)], represents the channel gain from the edge server to the IRS, and the baseband equivalent channel is expressed as G = [g 1,1 ,…,g 1,l ,…,g l,w ], Among them, i∈[1,l],j∈[1,w], and i,j are both integers;

[0069] (3)h q,d is the communication channel gain between the user and ES, and the baseband equivalent channel is represented by h q,d =[h d,1 (q),h d,2 (q),…,h d,A (q)], then the channel states of multiple users can be formulated as:

[0070]

[0071] Where H = [h1,h2,…,h Q ], H r =[h 1,r ,h 2,r ,…,h Q，r ], H d =[h 1,d ,h 2,d ,…,h Q,d ];

[0072]

[0073] Where ρ0 represents the path loss per unit distance, represents the distance between the qth user device in the task queue and the edge server antenna a, represents the distance between the qth user device in the task queue and the IRS element, K d , K r is the Rasen coefficient, and represents the LoS link component, and represents the NLoS component that follows the Rayleigh distribution.

[0074] Optionally, in step S1, user data is unloaded to the server through the communication layer, and the delay and energy consumption at the communication layer during the unloading process are calculated as follows: wherein the specific content of the delay energy consumption model of the communication layer is as follows:

[0075] The user task offloading process will generate transmission power consumption, which is expressed as follows:

[0076] E u =P q t u ;

[0077] Among them, t u It is composed of unloading delay and network waiting delay:

[0078]

[0079] in, Indicates network delay.

[0080] Optionally, in step S1, user data is offloaded to the server to solve the problem of computing energy consumption and latency at the server layer. The content of the offload computing latency and energy consumption model at the service layer is as follows:

[0081] Computational energy consumption: The computation time of the qth task on the edge server can be defined as:

[0082]

[0083] Among them, f s Describes the CPU clock frequency of the edge server, c q The computational size of a task is expressed as the number of CPU cycles required for the task. The energy generated by the qth task on the edge server can be defined as:

[0084]

[0085] Among them, κ s It is related to the CPU chip parameters of the edge server.

[0086] Optionally, in step S2, an objective function for minimizing delay energy consumption is defined as follows:

[0087] (1) Optimization objective function:

[0088] (P1):

[0089] Among them, λ=1 means that the task is calculated locally, and λ=0 means that the task is calculated on the edge server. After being processed by the intelligent digital twin, a sliding task queue length binary offloading strategy will be generated, λ q represents the unloading strategy of the task of the qth user in the task queue, Θ q represents the task of the qth user in the task queue, and the reflection angles of all the reflection elements of the IRS during the offloading process;

[0090] (2) The energy and delay generated by local computing and edge server computing are calculated as follows:

[0091]

[0092] Among them, μ and ν represent the weight ratio of energy consumption and delay in the optimization objective.

[0093] Optionally, in step S3, in order to achieve the user task offloading goal of minimizing delay and energy consumption, an ISES algorithm is proposed, and the specific contents are as follows:

[0094] (1) Status of low-latency task offloading energy-saving architecture of intelligent digital twins: To build a low-latency task offloading energy-saving architecture of intelligent digital twins, the status involved in DRL model construction mainly includes the following four aspects: a) Privacy information SP q : b) Task queue information SQ q :c q ,f q ,P q ; c) Channel information SC q : d) Edge server information SE q : The overall environmental state can be expressed as follows:

[0095] s q ={SP q ,SQ q ,SC q ,SE q};

[0096] (2) Actions taken by the low-latency task offloading energy-saving architecture of the intelligent digital twin: The low-latency task offloading energy-saving architecture of the intelligent digital twin needs to make the decision to offload and regulate the R = l × w reflective element of the IRS. When the task queue reaches the qth task, the action can be expressed as:

[0097]

[0098] (3) Reward function: The ultimate goal of DRL is to maximize the reward. The DRL algorithm reward is defined as follows:

[0099] r q =―G(λ q ,Θ q );

[0100] (4) The target action is obtained according to the target strategy smoothing idea, which is expressed as follows:

[0101]

[0102] Among them, clip(·) means clipping the data within an interval, and the above formula means between -c and c;

[0103] (5) Calculate the smoothed target value y n :

[0104]

[0105] γ is the discount factor;

[0106] (6) Calculate the losses of the two critic networks respectively and perform gradient updates:

[0107]

[0108] (7) The actor network is updated after T, using the same update strategy as DDPG:

[0109]

[0110] (8) Update the parameters of all target networks:

[0111] φ i ′←ξφ i +(1―ξ)φ i ′;

[0112] ψ′←ξψ+(1―ξ)ψ′;

[0113] Where ξ is the soft update coefficient of the target network.

[0114] After adopting the above technical solution, the beneficial effects of the present invention are:

[0115] This paper focuses on the needs of users in edge user-dense areas for computing power, data privacy, and energy sustainability. This paper proposes a privacy-enhanced deep reinforcement learning (DRL) algorithm for building a digital twin framework in edge user-dense areas. The proposed algorithms are the perturbed sliding task queue (PSTQ) algorithm and the DRL-based intelligent scheduling for energy sustainability (ISES) algorithm. These algorithms are designed to address the multi-objective requirements of the framework for low-latency data processing, low energy consumption, and privacy assurance.

[0116] Based on edge computing-enabled digital twin technology, this paper builds a low-latency task offloading energy-saving architecture for intelligent digital twins, and uses the characteristics of digital twins to provide a large amount of data for edge computing intelligent models. The architecture introduces the idea of ​​sliding task queues, which enables the architecture to make reasonable decisions for different numbers of tasks in real time.

[0117] Since users need to provide a large amount of data in the proposed framework, the present invention adopts differential privacy technology for the data before joining the processing queue, and adds Laplace noise to the user's sensitive data to ensure data invisibility.

[0118] The present invention proposes an enhanced communication computing model in the studied scenario, adopts multiple intelligent reflective surfaces to enhance communication conditions, and considers that the user-end equipment has the ability to dynamically adjust voltage and frequency to better achieve energy sustainability.

[0119] In order to improve user satisfaction with low latency and privacy, and achieve the energy-saving goal of the overall framework, this paper proposes a privacy-oriented deep reinforcement learning algorithm to achieve the above multi-objective requirements. The experimental results fully verify the good performance of the proposed architecture and algorithm in relevant scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0121] Figure 1 This is the low-latency task offloading and energy-saving architecture diagram of the intelligent digital twin;

[0122] Figure 2 It is the ISES algorithm flow chart. DETAILED DESCRIPTION

[0123] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0124] The embodiment of the present application discloses a high user satisfaction energy-saving task offloading method based on edge computing.

[0125] Example

[0126] according to Figure 1 to Figure 2 As shown, a high user satisfaction and energy-saving task offloading method based on edge computing is proposed.

[0127] 1. System model

[0128] The present invention constructs a low-latency task offloading energy-saving architecture of intelligent digital twins, which is divided into two parts: physical entity and virtual twin, such as Figure 1 As shown. The physical entity consists of multiple users, one IRS and an edge server. The present invention considers a user-dense area covered by an edge server, where users are randomly distributed in the area and continuously generate computing tasks, and each intelligent reflective surface is responsible for enhancing users in different location ranges. The virtual twin is a virtual mapping of the physical environment. The data generated by the service layer, communication layer and user layer are divided according to the characteristics of the physical entity. The intelligent digital twin uses its computing power in the edge server to build an intelligent model for the joint analysis of the three-layer data, and realizes the real-time resource scheduling and decision-making capabilities of the architecture. The ultimate goal of this architecture is to improve the privacy satisfaction of users in the area and ensure the idea of ​​energy sustainability.

[0129] Depend on Figure 1 It can be seen that the overall process of the architecture is that the user layer generates tasks, uploads task information through the communication layer, and the service layer acts as the control center. The intelligent digital twin deployed in the service layer collects data information of the three-layer architecture, and trains and issues decisions on the ISES algorithm model. The service layer schedules the resources for the task calculation based on the results of the ISES algorithm, and decides whether each task in the task queue is processed locally or offloaded to the server.

[0130] 1.1 User-level DVFS energy saving

[0131] The user layer uses a CPU with DVFS technology to process user tasks. The CPU contains complex complementary metal oxide semiconductor (CMOS) circuit technology. The power consumption generated by the CMOS circuit is mainly composed of dynamic power consumption P sa , static power consumption P sl And short-circuit power consumption P scThe composition is represented by the following formula.

[0132] P cmos =P sa +P sl +P sc ; (1)

[0133] Dynamic power is the change of the node's logic state, that is, the switch activity, which is called the activity factor, represented by ρ, and is reflected in the process of charging and discharging the load capacitor C. Static power is generated by leakage current, that is, the extra power consumption when the circuit is not active. Short-circuit power consumption is because the charging and discharging process of the capacitor has a certain time duration, and the feedback signal is difficult to change the logic state immediately, resulting in the CPU chip's power supply voltage V DD Power consumption generated by direct grounding. Since dynamic energy consumption is the main component of energy consumption, and the user layer of this paper considers the local computing process, the energy consumption calculation considers the dynamic power consumption during CPU use.

[0134]

[0135] Among them, f q is the CPU clock frequency of the user device corresponding to the qth task, ρ is the activity factor, C is the load capacitance, The square of the power supply voltage. Most user devices are mobile terminals, which rely on batteries for power supply. To ensure that the user's computing is carried out normally, it is necessary to be energy-efficient while meeting the user's computing needs. Studies have shown that when running under low voltage conditions, the clock frequency f of the user's processor chip q The voltage V DD In order to influence the linear relationship, this study rewrites the above formula (2) into the following formula, which is consistent with the research conclusion.

[0136]

[0137] Among them, κ q It is determined by the chip performance and dynamic effective capacitance. When the task is chosen to be calculated locally, the energy consumption depends not only on the computing power of the CPU but also on the size of the task to be calculated. The present invention expresses the size of the task to be calculated as the number of CPU cycles c required for the task q The number of CPU cycles required to process the qth task can be calculated by the following formula.

[0138] c q =s q b q ; (4)

[0139] Among them, s q Indicates the task size, b q Indicates the number of CPU cycles required to process a 1-byte task. The time required for local calculation of this task is expressed as follows:

[0140]

[0141] Among them, T q is the CPU clock cycle size, which is related to the CPU clock frequency f q The energy consumption of this task can be calculated by the following formula:

[0142] E local =P sa t q ; (6)

[0143]

[0144] In general, if the hardware conditions of the device remain unchanged, trying to reduce the computing energy consumption of a task is mainly related to the CPU clock frequency f q and the CPU chip voltage V DD The present invention uses DVFS technology to adjust the CPU frequency according to the power status of the user device. From formula (5), we can know that the task calculation time t q and CPU clock frequency f q Inversely proportional, reducing the frequency will lead to an increase in computing latency, so the actual scenario needs to consider the urgency of the task and the balance between latency sensitivity and energy saving.

[0145] 1.2 Communication layer multi-IRS enhancement

[0146] 1.2.1 Enhanced Communication Model

[0147] If the user needs to offload the task to the edge server for computing, a low-latency and low-power communication process is required. In the architecture proposed in this invention, IRS auxiliary communication is used to enhance communication conditions. The edge server ES has The present invention considers making full use of the IRS reflection capability and deploying it at the edge of the cell of the architecture. A smart reflective surface with an edge server with a smart digital twin deployed in the center of the community, such as Figure 1 shown.

[0148] The present invention considers the existence of a communication link for direct signal transmission and a communication link using multiple IRSs for auxiliary communication. In the communication link using multiple IRSs for auxiliary communication, the IRS uses its own controller to regulate R = l×w passive reflective elements to achieve beamforming. l and w are the number of reflective elements in each row and column of the IRS, respectively. The IRS is the relay of the communication link. The reflective element can adjust the phase and amplitude of the received signal, and passive reflection is used for beamforming. Its phase shift reflection coefficient matrix is ​​expressed as follows:

[0149]

[0150] Therefore, the present invention models the channel between the qth user in the task queue and the ES as follows:

[0151]

[0152] in, represents the baseband equivalent channel h from the qth user equipment in the task queue to the IRS q,r =[h 1,1 (q),…,h 1,w (q),…,h l,w (q)], represents the channel gain from the edge server to the IRS, and the baseband equivalent channel is expressed as G = [g 1,1 ,…,g 1,l ,…,g l,w ], Among them, i∈[1,l],j∈[1,w], and i,j are both integers. q,d is the communication channel gain between the user and ES, and the baseband equivalent channel is represented by h q,d =[h d,1 (q),h d,2 (q),…,h d,A (q)], then the channel states of multiple users can be formulated as:

[0153]

[0154] Where H = [h1,h2,…,h Q ], H r =[h 1,r ,h 2,r ,…,h Q,r ], H d =[h 1,d ,h 2,d ,…,h Q,d ].

[0155] The environment for constructing digital twins in the present invention is a user-dense area, and there are a large number of obstacles in the communication environment. LoS links and NLoS links exist in the process of signal transmission. The complexity of the environment causes path loss and fading of the signal, and there are large-scale fading and small-scale fading at the same time. During the process of unloading a certain task by the user, it is assumed that the channel is quasi-static during this period. The user device communicates directly with the edge server without the help of IRS. Since there are both LoS links and NLoS links in the communication channel, it is considered to be modeled as Ruisen fading.

[0156]

[0157]

[0158] Where ρ0 represents the path loss per unit distance, represents the distance between the qth user device in the task queue and the edge server antenna a, represents the distance between the qth user device in the task queue and the IRS element, K d , K r is the Rasen coefficient, and represents the LoS link component, and represents the NLoS component that follows the Rayleigh distribution.

[0159] When the user equipment is unloaded, the reflection elements of the IRS are used to reflect the signal of the transmitted data. The transmission signal of the qth user equipment in the task queue is expressed as:

[0160]

[0161] Among them, P q ≤P max is the transmission power of the qth user equipment in the task queue, and assumes that S q ~CN(0,1) is a circularly symmetric complex Gaussian random variable with unit variance and mean zero. The signal received by ES from the qth user device in the task queue is expressed as:

[0162] y q =h q X q +n q ; (14)

[0163] Among them, n q It means the mean is zero and the variance is Additive white Gaussian noise (AWGN), X q Represents the transmission signal of the qth user device in the task queue. When the user device of the present invention transmits a task, the unloading instruction is allocated according to the task queue. If there are multiple tasks to be unloaded at the same time, the edge server receives the data signal according to the data queue sequence to prevent the unloading data from causing data loss due to interference. Then the signal-to-noise ratio (SINR) can be expressed as:

[0164]

[0165] According to Shannon's formula, the unloading rate of the qth user device in the task queue can be calculated as:

[0166] R q =BW q log2(1+SINR q ); (16)

[0167] It is related to the current channel status, where BW q is the channel bandwidth.

[0168] 1.2.2 Communication Energy Consumption

[0169] The user task offloading process will generate transmission power consumption, which is expressed as follows:

[0170] E u =P q t u ; (17)

[0171] Among them, t u It is composed of unloading delay and network waiting delay:

[0172]

[0173] in, Indicates network delay.

[0174] 1.3 Intelligent Decision-making at the Service Layer

[0175] 1.3.1 Sliding Task Queue

[0176] This architecture is not restricted by the number of users in the environment and the number of tasks generated by users. It introduces the STQ idea, with each task as the minimum processing unit. The task contains information such as user location and size. The study divides the large number of tasks generated in real time into multiple segments according to the queue length, and makes resource scheduling and unloading decisions based on the queue length. Among them, STQ is represented by Q, Q = {1,2,…,Q}. The queue length is Q, which represents the number of tasks processed by the service layer using the algorithm each time. STQ can slide forward 1 to Q units each time according to the number of real-time tasks.

[0177] 1.3.2 Calculating Energy Consumption

[0178] The computation time of the qth task on the edge server can be defined as:

[0179]

[0180] Among them, f s Describes the CPU clock frequency of the edge server. As can be seen from the above, c q The computational size of a task is expressed as the number of CPU cycles required for the task. The energy generated by the qth task on the edge server can be defined as:

[0181]

[0182] Among them, κ sIt is related to the CPU chip parameters of the edge server. The edge server is assumed to be connected to a power source, have sufficient computing power, and have a fixed computing frequency.

[0183] 2. Digital twins with privacy protection

[0184] After computing power is marginalized, data centers are scattered and do not have mature strategies to resist attacks. When implementing digital twins in edge servers, user privacy needs to be considered as a key consideration. Edge servers collect user location information, device status information, etc. If this information is obtained through attacks, even if it has been anonymized or desensitized, it may leak related information due to large data or mining. The present invention proposes an algorithm for perturbing the sliding task queue, and considers using differential privacy technology on the user side. The algorithm can add noise disturbances to communication perception and user task information before being received by the edge server, preventing the intelligent digital twin from analyzing the user's original privacy-containing data.

[0185] 2.1 Basic Principles of Differential Privacy

[0186] Definition 1: ε-DifferentialPrivacy: For a given randomization algorithm If any pair of adjacent data sets and There is a difference in at most one data point (Hamming distance is 1), and any subset Randomization Algorithm Can satisfy the following formula:

[0187]

[0188] This means that the randomization algorithm To satisfy ε-Differential Privacy, the maximum divergence is introduced, where ε represents the privacy budget.

[0189] Definition 2: Global Sensitivity: Given any query function Its sensitivity can be written as:

[0190]

[0191] Among them, ‖·‖1 represents the l1 norm of the vector, and ΔY represents the maximum impact of any operation on an element in the data set on the data set query. For numerical data, adding noise can provide simple privacy protection. The Laplace mechanism of differential privacy adds Laplace noise to the query results to ensure the privacy of the data.

[0192] Definition 3: Laplace Mechanism: For a given query function if:

[0193]

[0194] in, The randomization algorithm It provides ε-Differential Privacy of Laplace mechanism. Function Z obeys Laplace distribution, and its probability density function is:

[0195]

[0196] in,

[0197] 2.2 Sliding Task Queue Based on Differential Privacy

[0198] The sliding task queue collects information about user information and the current channel, and adds perturbations based on the Laplace mechanism according to formula (28) to meet differential privacy. The perturbed data is expressed as follows:

[0199] Before the edge server makes an offloading decision, the intelligent digital twin collects the channel information of the device with the processing task. According to equations (11) and (12), if the edge server is untrustworthy or at risk of being attacked, the user location information may be leaked. The query function is defined as:

[0200]

[0201] Among them, ‖·‖2 represents the l2 norm of the vector, Indicates the location coordinates of devices that can establish channels with users, such as edge servers and IRSs. Represents the coordinate information of the user device. Assuming that the coverage distance of the edge server is K, the user location in this area must be no more than K away from the edge server. That is, This can be used to The global sensitivity of is defined as:

[0202]

[0203] According to Definition 1 and Definition 2, the above formula and The phase difference between adjacent data sets is one element. The differential privacy of the Laplace mechanism is used to protect the user's location information. According to Definition 3, the randomization algorithm is defined as:

[0204]

[0205] in, It follows the Laplace distribution.

[0206] Theorem 1:

[0207] The proposed privacy-preserving intelligent digital twin algorithm satisfies ε-differential privacy.

[0208] Proof: The neighboring set of user devices is defined within the coverage area K of the edge server and have:

[0209] 3. Prioritize solutions that meet user satisfaction and high energy efficiency

[0210] 3.1 Problem Definition

[0211] Based on the intelligent DT task offloading energy-saving architecture, this invention processes the privacy-protected data and makes offloading decisions. With the premise of improving user satisfaction with low latency and privacy and the goal of achieving energy saving of the overall architecture, the following optimization objective function is defined:

[0212]

[0213] Among them, λ=1 means that the task is calculated locally, and λ=0 means that the task is calculated on the edge server. After being processed by the intelligent digital twin, a sliding task queue length binary offloading strategy will be generated, λ q represents the unloading strategy of the task of the qth user in the task queue, Θ q It represents the task of the qth user in the task queue, and the reflection angle of all reflective elements of the IRS during the offloading process. The energy and delay generated by local computing and edge server computing are calculated as follows:

[0214]

[0215] Among them, μ and ν represent the weight ratio of energy consumption and delay in the optimization target. If the calculation is performed locally, the energy saving of battery-based equipment is considered. According to the DVFS technology, the CPU clock frequency is related to the chip voltage. The present invention focuses on the impact of the device power status on energy saving. If the calculation is performed on the edge server, the energy consumption and delay also include those generated by unloading.

[0216] 3.2 Intelligent Solutions

[0217] In order to maximize the multi-objective optimization of the present invention, it is proposed to allocate system resources and make unloading decisions based on privacy-sensitive reinforcement learning. After differential privacy information protection, the state space and action space of the low-latency task unloading energy-saving architecture of the resource scheduling part of the ISES algorithm of the present invention are continuous. In this context, some studies have considered using the DDPG algorithm as a DRL algorithm for generating unloading decisions. However, the DDPG algorithm has a common problem of overestimation of the Q value, and the TD3 algorithm DDPG has made the following improvements. In order to alleviate the overestimation problem, TD3 introduces two critic networks to estimate the action value function and selects a smaller Q value during training. The target strategy is smoothed, and the influence of extreme value functions is reduced by adding noise and clipping, making the critic network more stable. The idea of ​​delayed update is introduced to make the critic network update more frequently than the actor network, which will make the critic network estimate more accurately and the actor network take more effective actions.

[0218] The study deployed the algorithm in the edge server and used the intelligent digital twin of the area for model training. Deep reinforcement learning is widely used in edge computing environments. Each element in the sliding task queue is the user data corresponding to each task. Some private user information is added with differential privacy technology. Based on this information and the changing environmental information, the decision process of queue advancement and unloading is modeled as a Markov decision process. State action and reward are the three key elements of Markov. The state action space of the low-latency task offloading energy-saving architecture of the intelligent digital twin and the reward function are defined as follows:

[0219] (1) Status of low-latency task offloading energy-saving architecture of intelligent digital twins: To build a low-latency task offloading energy-saving architecture of intelligent digital twins, the status involved in DRL model construction mainly includes the following four aspects: a) Privacy information SP q : b) Task queue information SQ q : c) Channel information SC q : d) Edge server information SE q : The overall environmental state can be expressed as follows:

[0220] s q ={SP q ,SQ q ,SC q ,SE q}; (32)

[0221] (2) Actions taken by the low-latency task offloading energy-saving architecture of the intelligent digital twin: Process the environmental information collected by DT, train the DRL model in the edge server and deploy the model. In order to achieve low latency and energy saving for local or offloaded processing of user tasks, the low-latency task offloading energy-saving architecture of the intelligent digital twin needs to make the decision to offload and regulate the R = l × w reflective element of the IRS. Therefore, the action when the task queue reaches the qth task can be expressed as:

[0222]

[0223] (3) Reward function: The ultimate goal of DRL is to maximize the reward. According to formula (29), the optimization problem is to minimize the delay and energy consumption. Therefore, the reward of the proposed DRL algorithm is defined as follows:

[0224] r q =―G(λ q ,Θ q ); (34)

[0225] Next, the algorithm flow of the invention based on TD3 will be described, as follows: Figure 2 First, use randomized parameters φ1, φ2, ψ to initialize two critic networks. and the actor network π ψ The parameters of the target network of the three networks are not obtained by training, but by copying the parameters of the above three networks, that is, φ1′←φ1, φ2′←φ2, ψ′←ψ. In addition, the buffer pool used to store the experience Initialize. Loop each training round e, the present invention trains E times in total, and there are Step steps in each round. The size of Step depends on the length Q of the sliding task queue. Take the qth step as an example, according to the state s q The strategy determines action a q , represented by a q =π ψ (s q )+∈. And according to this step action a q Calculate the reward r q and next state s q+1 The obtained value in this step is (s q ,a q ,r q ,s q+1 ) is stored in the experience cache pool Then from the experience pool A small batch of N state transition records are randomly sampled in . The target action is obtained according to the target strategy smoothing idea, which is expressed as follows:

[0226]

[0227] Among them, clip(·) means clipping the data within a range, and the above formula means between -c and c. At the same time, the smoothed target value y is calculated n :

[0228]

[0229] γ is the discount factor. Calculate the losses of the two critic networks respectively and perform gradient updates:

[0230]

[0231] Due to the introduction of the delayed update idea, the actor network is updated after T, using the same update strategy as DDPG:

[0232]

[0233] Finally update the parameters of all target networks:

[0234] φ i ′←ξφ i +(1―ξ)φ i ′; (39)ψ′←ξψ+(1―ξ)ψ′; (40)

[0235] Among them, ξ is the soft update coefficient of the target network.

[0236] The present invention constructs a low-latency task offloading energy-saving architecture for intelligent digital twins, and realizes adaptive processing of user tasks arriving in real time through sliding task queues. Specifically, in order to ensure the privacy of users, the present invention introduces differential privacy technology based on the Laplace mechanism before the user data is collected by the edge server for building the digital twin model. In terms of low latency and energy saving, the present invention uses multiple IRSs to assist the user's data transmission process, so that the communication link conditions are enhanced, the transmission rate is increased, the latency is reduced, and the communication energy consumption is reduced. At the same time, the user device adopts DVFS technology to adjust the CPU frequency according to the energy state of the user device to optimize the power consumption of the local computing process. In order to achieve the above multiple goals, the present invention uses a deep reinforcement learning algorithm to optimize the architecture. Through a large number of experiments on scenarios and algorithms, the ISES algorithm parameters that are optimal for the overall architecture are obtained, and compared with other algorithms, the effectiveness of the algorithm proposed in the present invention is verified.

[0237] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A high user satisfaction energy-saving task offloading method based on edge computing, characterized in that: Includes the following: S1. Establishing system model: The task offloading process includes three levels: user layer, communication layer and server layer. The user layer model includes user delay energy consumption model and user privacy model. The communication layer includes IRS model and delay energy consumption model. The service layer includes offloading calculation delay and energy consumption model. S2, formulate the problems of reducing latency, energy consumption and protecting user privacy to form an objective function; S3. Under the premise of protecting user privacy, with the goal of minimizing latency and energy consumption, an algorithm is proposed to solve the problem, forming the ISES algorithm and obtaining an offloading solution.

2. According to claim 1, a high user satisfaction energy-saving task offloading method based on edge computing is characterized in that: In step S1, the content of the user delay energy consumption model is as follows: (1) The power consumption of CMOS circuits is mainly composed of dynamic power consumption P sa , static power consumption P sl And short-circuit power consumption P sc Composition, represented by the following formula: P cmos =P sa +P sl +P sc ; (2) Energy consumption calculation takes into account the dynamic power consumption of the CPU during use; Among them, f q is the CPU clock frequency of the user device corresponding to the qth task, ρ is the activity factor, and C is the load capacitance; The square of the power supply voltage; (3) Among them, κ q Determined by chip performance and dynamic effective capacitance; (4) The computational size of the task is expressed as the number of CPU cycles c required for the task. q , the number of CPU cycles required for the qth task processing can be calculated by the following formula: c q =s q b q ; Among them, s q Indicates the task size, b q Indicates the number of CPU cycles required to process a 1-byte task; (5) The time required for local calculation of this task is expressed as follows: Among them, T q is the CPU clock cycle size, which is related to the CPU clock frequency f q The two are reciprocal; (6) The energy consumption of this task can be calculated by the following formula: E local =P sa t q ; (7) When the user equipment is unloaded, the reflection elements of the IRS are all used to reflect the signal of the transmitted data. The transmission signal of the qth user equipment in the task queue is expressed as: Among them, P q ≤P max is the transmission power of the qth user equipment in the task queue, and assumes that S q ~CN(0,1) is a circularly symmetric complex Gaussian random variable with unit variance and mean zero. The signal received by ES from the qth user device in the task queue is expressed as: y q =h q X q +n q ; Among them, n q It means the mean is zero and the variance is Additive Gaussian white noise, X q Represents the transmission signal of the qth user device in the task queue; The signal-to-noise ratio can be expressed as: (8) According to Shannon's formula, the unloading rate of the qth user device in the task queue can be calculated as: R q =BW q log2(1+SINR q ); Among them, BW q is the channel bandwidth.

3. According to claim 2, a high user satisfaction energy-saving task offloading method based on edge computing is characterized in that: In step S1, the content of the user privacy model is as follows: (1) Randomization algorithm Can satisfy the following formula: This means that the randomization algorithm Satisfying ε-differential privacy, the maximum divergence is introduced, where ε represents the privacy budget; (2) Given any query function Its sensitivity can be expressed as: Among them, ‖·‖1 represents the l1 norm of the vector, and ΔY represents the maximum impact of any operation on an element in the data set on the data set query; (3) For a given query function if: in, The randomization algorithm The ε-differential privacy of the Laplace mechanism is provided. The function Z obeys the Laplace distribution, and its probability density function is: in, (4) The perturbed data is expressed as follows: (5) Define the query function as: Among them, ‖·‖2 represents the l2 norm of the vector, Indicates the location coordinates of devices that can establish channels with users, such as edge servers and IRSs. Represents the coordinate information of the user device; (6) Assuming that the coverage distance of the edge server is K, the user location in the area must be no more than K away from the edge server, that is, This can be used to The global sensitivity of is defined as: (7) The randomization algorithm is defined as: in, It follows the Laplace distribution; (8) Define the neighboring set of user devices within the edge server coverage K and have:

4. According to claim 3, a high user satisfaction energy-saving task offloading method based on edge computing is characterized in that: In step S1, after the user layer model is established, the data is transmitted from the user layer to the server and passes through the communication layer, thereby solving the energy consumption and delay problems of the communication layer. The communication layer is based on IRS-assisted communication to enhance the channel state. The constructed IRS model is as follows: (1) The IRS uses its own controller to control R = l × w passive reflection elements to achieve beamforming. l and w are the number of reflection elements in each row and column of the IRS, respectively. The IRS is the relay of the communication link. The reflection element can adjust the phase and amplitude of the received signal. The passive reflection performs beamforming. Its phase shift reflection coefficient matrix is ​​expressed as follows: (2) The channel between the qth user in the task queue and the ES is modeled as follows: in, represents the baseband equivalent channel h from the qth user equipment in the task queue to the IRS q,r =[h 1,1 (q),…,h 1,w (q),…,h l,w (q)], represents the channel gain from the edge server to the IRS, and the baseband equivalent channel is expressed as G = [g 1,1 ,…,g 1,l ,…,g l,w ], Among them, i∈[1,l],j∈[1,w], and i,j are both integers; (3)h q,d is the communication channel gain between the user and ES, and the baseband equivalent channel is represented by h q,d =[h d,1 (q),h d,2 (q),…,h d,A (q)], then the channel states of multiple users can be formulated as: in, Where ρ0 represents the path loss per unit distance, represents the distance between the qth user device in the task queue and the edge server antenna a, represents the distance between the qth user device in the task queue and the IRS element, K d , K r is the Rasen coefficient, and represents the LoS link component, and represents the NLoS component that follows the Rayleigh distribution.

5. According to claim 4, a high user satisfaction energy-saving task offloading method based on edge computing is characterized in that: In step S1, user data is unloaded to the server through the communication layer. The delay and energy consumption in the communication layer during the unloading process are calculated as follows: Among them, the specific content of the delay energy consumption model of the communication layer is as follows: The user task offloading process will generate transmission power consumption, which is expressed as follows: E u =P q t u ; Among them, t u It is composed of unloading delay and network waiting delay: in, Indicates network delay.

6. According to claim 5, a high user satisfaction energy-saving task offloading method based on edge computing is characterized in that: In step S1, user data is unloaded to the server to solve the problem of computing energy consumption and latency at the server layer. The content of the unloading computing latency and energy consumption model at the service layer is as follows: Computational energy consumption: The computation time of the qth task on the edge server can be defined as: Among them, f s Describes the CPU clock frequency of the edge server, c q The computational size of a task is expressed as the number of CPU cycles required for the task. The energy generated by the qth task on the edge server can be defined as: Among them, κ s It is related to the CPU chip parameters of the edge server.

7. The method for energy-saving task offloading with high user satisfaction based on edge computing according to claim 6 is characterized in that: In step S2, an objective function for minimizing delay energy consumption is defined as follows: (1) Optimization objective function: Among them, λ=1 means that the task is calculated locally, and λ=0 means that the task is calculated on the edge server. After being processed by the intelligent digital twin, a sliding task queue length binary offloading strategy will be generated, λ q represents the unloading strategy of the task of the qth user in the task queue, Θ q represents the reflection angle of all reflective elements of IRS during the offloading process of the task of the qth user in the task queue; (2) The energy and delay generated by local computing and edge server computing are calculated as follows: Among them, μ and ν represent the weight ratio of energy consumption and delay in the optimization objective.

8. The method for energy-saving task offloading with high user satisfaction based on edge computing according to claim 7 is characterized in that: In step S3, in order to achieve the user task offloading goal of minimizing delay and energy consumption, the ISES algorithm is proposed. The specific contents are as follows: (1) Status of low-latency task offloading energy-saving architecture of intelligent digital twins: To build a low-latency task offloading energy-saving architecture of intelligent digital twins, the status involved in DRL model construction mainly includes the following four aspects: a) Privacy information SP q : b) Task queue information SQ q :c q ,f q ,P q ; c) Channel information SC q : d) Edge server information SE q : The overall environmental state can be expressed as follows: s q ={SP q ,SQ q ,SC q ,SE q }; (2) Actions taken by the low-latency task offloading energy-saving architecture of the intelligent digital twin: The low-latency task offloading energy-saving architecture of the intelligent digital twin needs to make the decision to offload and regulate the R = l × w reflective element of the IRS. When the task queue reaches the qth task, the action can be expressed as: (3) Reward function: The ultimate goal of DRL is to maximize the reward. The DRL algorithm reward is defined as follows: r q =―G(λ q ,I q ); (4) The target action is obtained according to the target strategy smoothing idea, which is expressed as follows: Among them, clip(·) means clipping the data within an interval, and the above formula means between -c and c; (5) Calculate the smoothed target value y n : γ is the discount factor; (6) Calculate the losses of the two critic networks respectively and perform gradient updates: (7) The actor network is updated after T, using the same update strategy as DDPG: (8) Update the parameters of all target networks: f i ′←ξφ i +(1-ξ)φ i ′; ψ′←ξψ+(1―ξ)ψ′; Among them, ξ is the soft update coefficient of the target network.

Images