Edge server deployment scheme of mobile edge network

Abstract

The invention discloses an edge server deployment scheme for a mobile edge network, and the scheme comprises the steps: constructing a mobile edge computing network model which comprises a remote cloud computing center, a plurality of macro base stations and a plurality of edge servers which are cooperatively deployed with the macro base stations; and obtaining an optimal edge server deployment scheme based on a Q-Learning server deployment algorithm in combination with a user request received by each macro base station in the edge network and the computing power of the edge server. According to the method, a three-layer edge network model is considered, a plurality of base stations are selected in a base station layer to cooperatively deploy the edge server, and computing resources are sunk into the edge network closer to the user, so that the mobile user is prevented from sending a request to a remote cloud computing center, the waiting delay of the mobile user is reduced, and the transmission pressure of each level of network can be relieved. In addition, transmission time delay among the base stations in the network is represented through reinforcement learning and introduction of a delay matrix, and an invalid or large-delay distribution scheme can be avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-layer network architecture and deployment in edge networks, and specifically relates to an edge server deployment solution for a mobile edge network. Background Art

[0002] With the development of the Internet of Things and 5G networks in smart cities, mobile communication traffic has exploded, and mobile smart devices have become increasingly important as tools for entertainment, learning, social interaction, and a smarter life. The explosive growth of terminal devices such as various smartphones, smart watches, tablets, smart home devices, sensors, etc. are widely used in industries such as industry, school, medical care, VR (Virtual Reality), smart home, and Internet of Vehicles. The rapid development of these new fields in which the Internet is deeply involved has posed new challenges to the transmission capacity, transmission rate, data distribution and processing capabilities, and data security of the entire network. According to Cisco's forecast, the monthly traffic consumption of smartphones in 2022 will be as high as 11GB, of which about 80% will be generated by watching videos. With the development of cloud computing, the method of sending the requests of the above terminals directly to the cloud computing center has indeed improved the user experience. However, deploying all applications in the cloud computing center will increase the pressure on the transmission network at all levels, resulting in increased access delays. In addition, the energy consumption cost of transmitting data from mobile devices to the cloud computing center is too high.

[0003] In order to solve the above problems, academia and industry have begun to optimize computing models, such as micro data centers, mobile edge computing (MEC), fog computing (FC), and cloudlet. The rise of the concept of the Internet of Everything and the growth of mobile terminals have prompted the development of edge processing mode, namely edge computing model. The European Telecommunications Standards Institute (ETSI) established the Mobile Edge Computing Standardization Working Group in 2014. The main idea is to sink the MEC platform from the cloud computing center to the edge of the mobile access network, and ultimately achieve elastic utilization of computing and storage resources.

[0004] MEC has attracted widespread attention as an effective solution to reduce user access latency and improve the current network architecture. In traditional cloud computing, computing resources and storage management are performed in the cloud; in mobile edge computing, computing resources are deployed to the edge of the mobile access network with the edge server, and user requests can be processed at a physical location closer to the user. Such a network architecture can bring the following benefits: From the user's perspective, edge servers are equivalent to cloud service providers, with computing resources closer to users and lower latency in requesting services. For the core network, MEC can effectively alleviate network transmission congestion. In some high-bandwidth, low-latency innovative services, such as augmented reality (AR), deep learning (DL), smart cities and other application scenarios, many scholars have conducted real-world deployment analysis.

[0005] At present, in the research on deploying micro-clouds or cloudlets in MEC scenarios, many scholars use user access delay or the energy consumption of these deployed devices as optimization targets, and then express these optimization targets as single-objective optimization problems or multi-objective optimization problems, and use optimization algorithms such as mixed integer programming and particle swarm optimization to solve the best solution. The common point of these optimization algorithms is that the deployment strategy is fixed. Although a better deployment solution can be solved, there are generally problems such as large amount of calculation and long solution time. Another problem is that when determining the allocation relationship between base stations and edge servers, many scholars directly use random allocation, and then evaluate the quality of this allocation method. This random allocation will result in too many allocation methods. Summary of the invention

[0006] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide an edge server deployment solution for a mobile edge network. The deployment solution is designed by introducing the idea of ​​reinforcement learning, and a reward function is designed to evaluate the quality of an execution action, which is used to adjust the subsequent deployment strategy. Through learning from multiple episodes, an optimal deployment strategy is finally obtained, thereby minimizing the total waiting delay of the base station in the edge network.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: One aspect of the present invention provides an edge server deployment solution for a mobile edge network, comprising the following steps: Construct a mobile edge computing network model, including a remote cloud computing center, multiple macro base stations, and multiple edge servers deployed in coordination with the macro base stations; the edge server is deployed at the macro base station to process service requests from multiple macro base station users nearby, and the edge server communicates with the remote cloud computing center to obtain more computing resources; Combining the user requests received by each macro base station in the edge network and the computing power of the edge server, the optimal edge server deployment solution is obtained based on the Q-Learning server deployment algorithm.

[0008] As a preferred technical solution, the optimal edge server deployment solution is obtained by using the Q-Learning server deployment algorithm, specifically: Based on Q-Learning server deployment algorithm includes N Time period i , each period i Complete a complete server deployment plan. i Divided into k Time slots t , and perform the following steps: In each time slot t At the beginning, a macro base station with the largest Q value is selected to coordinate the deployment of edge servers; In time slot t According to the allocation strategy, several nearby macro base stations are allocated to the deployed edge servers, and the reward value of this deployment action is calculated according to the reward function; Use the action value function to update the Q value in the next time slot t At the beginning, the macro base station with the largest next-state Q value is selected to deploy the edge server.

[0009] As a preferred technical solution, the total global network delay in a period of time T It is expressed as: ; in k represents the number of time slots, i.e., the number of deployed edge servers; T j Represents edge server s j The latency of the macro base station responsible for processing is expressed as follows: ; in B j Indicates that it is assigned to the edge server s j A collection of macro base stations, D i,j Indicates a macro base station b i To the edge server s j The transmission delay of is obtained through the delay matrix; w bi Represents a macro base station in an edge network b i The load, that is, the amount of computing power received from the user request; f Represents the maximum processing capacity of the edge server. Due to the limited computing resources of the edge server, the task processing of the edge server is regarded as a queuing system. μ j is the average service rate of the edge server, ρ jis the average service density.

[0010] As a preferred technical solution, the time slot t Several nearby macro base stations are allocated to the deployed edge servers according to the allocation strategy, specifically: Using undirected graph G ( V , E ) represents the topology of the edge network, where V is a collection of macro base stations, | V |= m , E It is a collection of links, each link is represented by an edge, and each edge has a weight to represent the transmission delay of this edge; Obtain the delay matrix between macro base stations and macro base stations through the shortest path algorithm D ; After determining that a certain edge server is located at a certain macro base station, the macro base station with a smaller delay is allocated to the edge server using the delay matrix.

[0011] As a preferred technical solution, based on the characteristics of the network model, the state, action, reward and update function in reinforcement learning are defined on the basis of the Q-Leaning server deployment algorithm; The state is defined as: the location of a certain edge server at a certain moment, and each state is represented by a vector state j =[ c j,1 ,…, c j,i ,…, c j,m ], in vector c j,i is a binary variable representing the edge server s j Whether deployed on a macro base station b i Department, c j,i =1 indicates edge server s j With macro base station b i Collaborative deployment, otherwise c j,i =0; an edge server can only be deployed with one macro base station, subject to the following constraints: ; In time slot t System status state j t It is expressed as: state j t =[ c j,1 t ,…, c j,i t ,…, c j,m t ]; in, c j,i t Indicates time slot t Internal Edge Server s j Whether it is connected to the macro base station b i Collaborative deployment: Each state represents the deployment status of an edge server, and the state space is represented as: ; That is, updating the state space once represents a complete deployment plan; The action is defined as: determining the location of an edge server and assigning some macro base stations to this edge server. The action is represented by a vector: a j =[ state j , B j ]; the action space represents the set of actions that the system can perform, expressed as: ; The system goal is to obtain a deployment solution that minimizes the waiting time of the macro base station, specifically: ; The goal of reinforcement learning is to maximize the long-term cumulative reward, so the reward for an adjustment action is set to a function that is negatively correlated with the delay. The reward function under the reinforcement learning framework is expressed as: reward j = e -Tj ; The action value function update formula is: Q ( state t , a t )← Q ( state t , a t )+ α [reward + γ maxQ ( state t+1 , a t+1 )- Q ( state t , a t )]; in, state t , a t Indicates the current state and action. state t+1 , a t+1 Indicates the state and action of the next time slot, reward Indicates execution of an action a t The reward value obtained after α represents the learning rate, γ ∈[0,1] is the reward attenuation factor. If the reward attenuation factor γ When it is equal to 0, it is considered that the corresponding scheme only focuses on the reward of the current time step; if the reward attenuation factor γ The closer it is to 0, the more likely the corresponding solution is to consider immediate rewards. γ The closer it is to 1, the more likely it is that the corresponding solution can balance future rewards and rewards at the current time step; if the environment state is known, the reward decay factor γ Set to 1.

[0012] As a preferred technical solution, the delay matrix between the macro base station and the macro base station is obtained by the shortest path algorithm. D , specifically: Generate an undirected graph based on a normal distribution G ( V , E ) and use Dijkstra's algorithm to calculate the weighted value of each edge in the undirected graph G ( V , E ) Zhonghong Base Station b i and b j The delay between D i,j .

[0013] Another aspect of the present invention further provides an edge server deployment system for a mobile edge network, which is applied to the above-mentioned edge server deployment solution for a mobile edge network, including a mobile edge computing network model and an edge server deployment module; The mobile edge computing network model includes a remote cloud computing center, multiple macro base stations, and multiple edge servers deployed in coordination with the macro base stations; the edge server is deployed at the macro base station to process service requests from multiple macro base station users nearby, and the edge server communicates with the remote cloud computing center to obtain more computing resources; The edge server deployment module is used to combine the user requests received by each macro base station in the edge network and the computing power of the edge server to obtain the best edge server deployment solution based on the Q-Learning server deployment algorithm.

[0014] Another aspect of the present invention further provides a storage medium storing a program, which, when executed by a processor, implements the above-mentioned edge server deployment solution for a mobile edge network.

[0015] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The edge server deployment of the mobile edge computing of the present invention considers a three-layer edge network model. At the base station layer, several base stations are selected to collaboratively deploy edge servers with computing, storage, and management functions. The base stations can receive user requests and forward them to nearby edge servers. By sinking computing resources to the edge network closer to the user, mobile users avoid sending requests to remote cloud computing centers, which can reduce the waiting delay of mobile users and alleviate the transmission pressure of networks at all levels.

[0016] (2) The present invention uses a reinforcement learning algorithm to design a deployment strategy. Reinforcement learning can learn a strategy that maximizes the cumulative reward value through interaction with the environment. The algorithm first defines the concepts of state, action, reward function, and action value function in the reinforcement learning framework of this system; when processing the allocation relationship between base stations and edge servers, this system introduces a delay matrix to represent the transmission delay between base stations in the network, which can avoid invalid or large-delay allocation schemes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a three-layer edge network model diagram consisting of a cloud computing center, an edge server (Edge Server), and a macro base station (MBS) according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0019] Example: This embodiment proposes a Figure 1 The three-layer mobile edge computing network model is shown in the figure, and a solution for edge server deployment based on the reinforcement learning Q-Learning algorithm is given under this network model. The implementation of this solution includes the following steps: S1. Build a three-layer mobile edge computing network model, which includes a remote cloud computing center, multiple macro base stations (MBS), and multiple edge servers (ES) deployed in collaboration with the macro base stations.

[0020] S2. Combining the user requests received by each macro base station in the edge network and the computing power of the edge server, the optimal edge server deployment solution is obtained based on the Q-Learning server deployment algorithm.

[0021] In step S1 of this embodiment, each macro base station in the edge network has a certain coverage range, and the macro base stations are connected to each other through wired links. Edge servers will be deployed at certain macro base stations to process service requests from nearby macro base stations. At the same time, edge servers can communicate with cloud computing centers to obtain more computing resources. In the edge network, a macro base station can only forward requests to one edge server, and all edge servers happen to serve all macro base stations in the edge network. Each edge server has computing, storage and other functions. Each macro base station is responsible for collecting requests from mobile users within the coverage area and forwarding them to the edge server. Each macro base station can forward requests to the edge server directly connected to it, or forward them to another edge server via other macro base stations. Each macro base station will allocate certain resources to forward requests, so user requests will not generate waiting delays at the macro base station during the forwarding process, and only the transmission delay needs to be considered.

[0022] The edge network consists of a group of macro base stations connected via the Internet. B ={ b 1,…, b i ,…, b m} and a set of edge servers that provide computing resources S= { s1 ,…, s j ,…, s k}. Using an undirected graph G ( V , E ) is used to represent the link relationship between the macro base station and the edge server. Figure 1 There are two types of edges, macro base stations b i and macro base stations b o The edge between b i , b o )∈ E express b i and b o There is one hop distance between the macro base stations. b i and edge servers s j The edge ( b i , s j )∈ E Indicates a macro base station b i You can use the edge ( b i , s j ) and edge servers s j Direct communication. We assume that G It is fully connected, which means that all macro base stations can be connected to each other through communication links. In addition, each edge server can directly access the remote cloud computing center through the core network.

[0023] In step S2 of this embodiment, the optimal edge server deployment solution is obtained by using the Q-Learning server deployment algorithm, specifically: S21, based on Q-Learning server deployment algorithm includes N Period (episode) i , each period i Complete a complete server deployment plan. i Divided into k Time slots t , and perform the following steps: S22, in each time slot tAt the beginning, a macro base station with the largest Q value is selected to coordinate the deployment of edge servers; S23, in the time slot t According to the allocation strategy, several nearby macro base stations are allocated to the deployed edge servers, and the reward value of this deployment action is calculated according to the reward function; S24, use the action value function to update the Q value, in the next time slot t At the beginning, the macro base station with the largest next-state Q value is selected to deploy the edge server.

[0024] Reinforcement learning is a process of continuous interactive learning until the cumulative reward value of each period converges, and each period ends with a complete deployment plan.

[0025] As a preferred technical solution, in step S21, the total global network delay T in a period is expressed as: ; in k represents the number of time slots, i.e., the number of deployed edge servers; T j Represents edge server s j The latency of the macro base station responsible for processing is expressed as follows: ; in B j Indicates that it is assigned to the edge server s j A collection of macro base stations, D i,j Indicates a macro base station b i To the edge server s j The transmission delay of is obtained through the delay matrix; w bi Represents a macro base station in an edge network b i The load, that is, the amount of computing power received from the user request; f Represents the maximum processing capacity of the edge server. Due to the limited computing resources of the edge server, the task processing of the edge server is regarded as a queuing system. μ j is the average service rate of the edge server, ρ j is the average service density.

[0026] As a preferred technical solution, in step S23, the tSeveral nearby macro base stations are allocated to the deployed edge servers according to the allocation strategy, specifically: Using undirected graph G ( V , E ) represents the topology of the edge network, where V is a collection of macro base stations, | V |= m , E It is a collection of links, each link is represented by an edge, and each edge has a weight to represent the transmission delay of this edge; Obtain the delay matrix between macro base stations and macro base stations through the shortest path algorithm D ; After determining that a certain edge server is located at a certain macro base station, the macro base station with a smaller delay is allocated to the edge server using the delay matrix.

[0027] To construct the network delay matrix In this embodiment, the edge network topology diagram G A weighted value is set for each edge in the network, which represents the delay between the two macro base stations connected by this edge. The weighted value of each edge is generated according to the normal distribution, with an average value of 0.2, a variance of 0.1, and a limited range of values ​​between 0.1 and 0.4. Finally, the Dijkstra algorithm is used to calculate the network undirected graph G ( V , E ) Zhonghong Base Station b i and b j The delay between D i,j .

[0028] In order to enable those skilled in the art to better understand the solution of this embodiment, the logic implementation code of the macro base station allocation strategy is provided below: Input: G, B, D, k, m , base station number q deployed with edge server s j ; Output: B j ; B j ←∅, R ← m / k , index ←-1, x q ←1; Fori = 1 to R do / / Number of base stations allocated to edge server; minDelay ← Float.MAX_VALUE ; For v = 1 to m do; If D v,q < minDelay ; minDelay ← D v,q ; index ← v ; End if; End for; y ( index, j )←1, B j ← B j ∪ b index ; For v = 1 to m do; D index,v ← Float.MAX_VALUE ; End for; End for; Return B j .

[0029] This embodiment uses the Q-ESP algorithm to combine the characteristics of this system with the Q-Learning algorithm, and uses the thinking of reinforcement learning to learn an optimal deployment plan. First, it is necessary to determine the relevant concepts such as state, action, and reward function of the edge server deployment problem in reinforcement learning.

[0030] The state of this system under reinforcement learning is defined as: the location of a certain edge server at a certain moment, and each state can be represented by a vector state j =[ c j,1 ,…, c j,i ,…, c j,m ], in vector cj,i is a binary variable representing the edge server s j Whether deployed on a macro base station b i Department, c j,i =1 indicates edge server s j With macro base station b i Collaborative deployment, otherwise c j,i = 0. An edge server can only be deployed with one macro base station. Obviously, there are the following constraints: ; In time slot t System status state j t It is expressed as: state j t =[ c j,1 t ,…, c j,i t ,…, c j,m t ]; in, c j,i t Indicates time slot t Internal Edge Server s j Whether it is connected to the macro base station b i Collaborative deployment: Each state represents the deployment status of an edge server, and the state space is represented as: ; It can be seen that updating the state space once represents a complete deployment plan.

[0031] The action of this system under reinforcement learning consists of two parts: determining the location of an edge server and assigning some macro base stations to this edge server. The action can be represented by a vector a j =[ state j , B j ]. The action space represents the set of actions that can be performed by this system, expressed as: ; To determine the reward function, we first need to define the optimization goal of the system model. The main goal of the present invention is to reduce the waiting delay of all macro base stations in the edge network. Since the macro base station forwards the request to the edge server for processing, the total delay can be expressed by the processing delay of the server, which mainly includes the following parts: the transmission delay of the macro base station forwarding the request to the server; the processing delay of the request at the server; the waiting delay of the request generating a waiting queue at the server, which is expressed as follows: ; in B j Indicates that it is assigned to the edge server s j A collection of macro base stations, D i,j Indicates a macro base station b i To the edge server s j The transmission delay of is obtained through the delay matrix; w bi Represents a macro base station in an edge network b i The load, that is, the amount of computing power received from the user request; f Represents the maximum processing capacity of the edge server. Due to the limited computing resources of the edge server, the task processing of the edge server is regarded as a queuing system. μ j is the average service rate of the edge server, ρ j is the average service density.

[0032] The proposed edge server deployment problem is dedicated to obtaining a deployment strategy that minimizes the waiting time of the macro base station. The goal of reinforcement learning is to maximize the long-term cumulative reward, so the reward for an adjustment action is set to a function that is negatively correlated with the delay. The reward function under the reinforcement learning framework of this system is expressed as: reward j = e -Tj ; Action-Value Function Q ( state t , a t ) is used to indicate that the state t Next select action a t The Q-Learning algorithm belongs to the non-fixed strategy algorithm of time difference control, and its action value function update formula is as follows: Q ( state t , a t )← Q ( state t , a t )+ α [ reward + γ maxQ ( state t+1 , a t+1 )- Q ( state t , a t )]; in, state t , a t Indicates the current state and action. state t+1 , a t+1 Indicates the state and action of the next time slot, reward Indicates execution of an action a t The reward value obtained after α represents the learning rate, γ ∈[0,1] is the reward attenuation factor. If the reward attenuation factor γ When it is equal to 0, it is considered that the corresponding scheme only focuses on the reward of the current time step; if the reward attenuation factor γ The closer it is to 0, the more likely the corresponding solution is to consider immediate rewards. γ The closer it is to 1, the more likely it is that the corresponding solution can balance future rewards and rewards at the current time step; if the environment state is known, the reward decay factor γ Set to 1.

[0033] In order to enable those skilled in the art to better understand the solution of this embodiment, the following provides a logic implementation code of the Q-Learning-based edge server deployment algorithm (Q-ESP): Input: G, D, W, B, k, m, ε, α, γ, N ; Output: Q - table, X, U ; For episode i = 1 to N do; randomly select a state state 1 ; For slott = 1 to k do; generate p at random; If p ≤ ε ; randomly select an action a t ; Else; select action argmax at Q ( state t , a t ); End if; execute action a t , obtain reward reward t ( state t , a t ) according to equation(3-11) and observe the next state state t+1 ; calculate Q-value and update in Q-table according to equation (3-7); End for; End for.

[0034] Furthermore, this embodiment also provides a specific experimental scenario simulation. The experimental scenario simulates a rectangular area of ​​2000m*2000m, and 50 macro base stations are deployed in the area to receive user requests. The communication range of each macro base station is 200m, and the topology between macro base stations is generated by the Barabasi-Albert model. The task data that needs to be calculated for each macro base station is 200-500MB, and the number of CPU cycles required to process 1bit is 1000cycles / bit. Select k (1≤k≤10) macro base station locations to collaboratively deploy edge servers to process requests from macro base stations. The service range of each edge server is 500m. In each experiment, the average arrival rate of edge server tasks is λ j Determine λ from the normal distributionj ~N(0.2,0.5), with a mean of 0.2, a variance of 0.5 and 0≤λ j ≤0.4, the average service rate of the edge server is 0.5, and the processing capacity of the edge server is 10GHz / s. In addition, the learning rate α of Q learning is 0.8, the experience reward decay factor γ is 0.1, the number of iterations is set N to 1000, and ε in the ε-greedy algorithm is set to 0.01.

[0035] In order to evaluate the specific performance of the Q-ESP algorithm proposed in this embodiment in the edge server deployment problem, several common algorithms are provided: Top-K, K-means, Random, and MIP algorithms as comparison benchmarks, among which the Top-K algorithm selects K macro base stations with the highest workload to deploy edge servers; the K-means algorithm divides all macro base stations into K clusters, and then deploys K edge servers at the centroid of these K clusters; the Random algorithm randomly selects K macro base stations to deploy servers; the MIP algorithm is an optimization algorithm proposed by other scholars to solve the edge server deployment problem. Mainly from the perspective of the total waiting delay of the macro base station, the performance of this solution is measured by comparing with several common algorithms.

[0036] In another embodiment of the present application, a system for deploying an edge server in a mobile edge network is provided, the system comprising a mobile edge computing network model and an edge server deployment module; The mobile edge computing network model includes a remote cloud computing center, multiple macro base stations, and multiple edge servers deployed in coordination with the macro base stations; the edge server is deployed at the macro base station to process service requests from multiple macro base station users nearby, and the edge server communicates with the remote cloud computing center to obtain more computing resources; The edge server deployment module is used to combine the user requests received by each macro base station in the edge network and the computing power of the edge server to obtain the best edge server deployment solution based on the Q-Learning server deployment algorithm.

[0037] It should be noted here that the system provided in the above embodiment is only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above. The system is an edge server deployment solution for a mobile edge network applied to the above embodiment.

[0038] In another embodiment of the present application, a storage medium is further provided, storing a program, and when the program is executed by a processor, an edge server deployment solution of a mobile edge network of the above embodiment is implemented, specifically: Construct a mobile edge computing network model, including a remote cloud computing center, multiple macro base stations, and multiple edge servers deployed in coordination with the macro base stations; the edge server is deployed at the macro base station to process service requests from multiple macro base station users nearby, and the edge server communicates with the remote cloud computing center to obtain more computing resources; Combining the user requests received by each macro base station in the edge network and the computing power of the edge server, the optimal edge server deployment solution is obtained based on the Q-Learning server deployment algorithm.

[0039] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0040] The above embodiments are preferred implementation schemes of the present invention, but the implementation schemes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An edge server deployment solution for a mobile edge network, characterized in that: The steps include: Build a mobile edge computing network model, including a remote cloud computing center, multiple macro base stations, and multiple edge servers deployed in coordination with the macro base stations; The edge server is deployed at the macro base station to process service requests from multiple macro base station users nearby. At the same time, the edge server communicates with the remote cloud computing center to obtain more computing resources. Combining the user requests received by each macro base station in the edge network and the computing power of the edge server, the optimal edge server deployment solution is obtained based on the Q-Learning server deployment algorithm.

2. The edge server deployment solution of a mobile edge network according to claim 1, characterized in that: The optimal edge server deployment solution is obtained by using the Q-Learning server deployment algorithm, specifically: Based on Q-Learning server deployment algorithm includes N Time period i , each period i Complete a complete server deployment plan. i Divided into k Time slots t , and perform the following steps: In each time slot t At the beginning, a macro base station with the largest Q value is selected to coordinate the deployment of edge servers; In time slot t According to the allocation strategy, several nearby macro base stations are allocated to the deployed edge servers, and the reward value of this deployment action is calculated according to the reward function; Use the action value function to update the Q value in the next time slot t At the beginning, the macro base station with the largest next-state Q value is selected to deploy the edge server.

3. The edge server deployment solution of a mobile edge network according to claim 2, characterized in that: Total global network delay in a period of time T It is expressed as: ； in k represents the number of time slots, i.e., the number of deployed edge servers; T j Represents edge server s j The latency of the macro base station responsible for processing is expressed as follows: ； in B j Indicates that it is assigned to the edge server s j A collection of macro base stations, D i,j Indicates a macro base station b i To the edge server s j The transmission delay of is obtained through the delay matrix; w bi Represents a macro base station in an edge network b i The load, that is, the amount of computing power received from the user request; f Represents the maximum processing capacity of the edge server. Due to the limited computing resources of the edge server, the task processing of the edge server is regarded as a queuing system. μ j is the average service rate of the edge server, ρ j is the average service density.

4. The edge server deployment solution of a mobile edge network according to claim 2, characterized in that: In the time slot t Several nearby macro base stations are allocated to the deployed edge servers according to the allocation strategy, specifically: Using undirected graph G ( V , E ) represents the topology of the edge network, where V is a collection of macro base stations, | V |= m , E It is a collection of links, each link is represented by an edge, and each edge has a weight to represent the transmission delay of this edge; Obtain the delay matrix between macro base stations and macro base stations through the shortest path algorithm D ; After determining that a certain edge server is located at a certain macro base station, the macro base station with a smaller delay is allocated to the edge server using the delay matrix.

5. The edge server deployment solution of a mobile edge network according to claim 2, characterized in that: According to the characteristics of the network model, the state, action, reward and update function in reinforcement learning are defined based on the Q-Leaning server deployment algorithm; The state is defined as: the location of a certain edge server at a certain moment, and each state is represented by a vector state j =[ c j,1 ,…, c j,i ,…, c j,m ], in vector c j,i is a binary variable representing the edge server s j Whether deployed on a macro base station b i Department, c j,i =1 indicates edge server s j With macro base station b i Collaborative deployment, otherwise c j,i =0; an edge server can only be deployed with one macro base station, subject to the following constraints: ； In time slot t System status state j t It is expressed as: state j t =[ c j,1 t ,…, c j,i t ,…, c j,m t ]； in, c j,i t Indicates time slot t Internal Edge Server s j Whether it is connected to the macro base station b i Collaborative deployment: Each state represents the deployment status of an edge server, and the state space is represented as: ； That is, updating the state space once represents a complete deployment plan; The action is defined as: determining the location of an edge server and assigning some macro base stations to this edge server. The action is represented by a vector: a j =[ state j , B j ]; the action space represents the set of actions that the system can perform, expressed as: ； The system goal is to obtain a deployment solution that minimizes the waiting time of the macro base station, specifically: ； The goal of reinforcement learning is to maximize the long-term cumulative reward, so the reward for an adjustment action is set to a function that is negatively correlated with the delay. The reward function under the reinforcement learning framework is expressed as: reward j = e -Tj ； The action value function update formula is: Q ( state t , a t )← Q ( state t , a t )+ α [ reward+γmaxQ ( state t+1 , a t+1 )- Q ( state t , a t )]； in, state t , a t Indicates the current state and action. state t+1 , a t+1 Indicates the state and action of the next time slot, reward Indicates execution of an action a t The reward value obtained after α represents the learning rate, γ ∈[0,1] is the reward attenuation factor. If the reward attenuation factor γ When it is equal to 0, it is considered that the corresponding scheme only focuses on the reward of the current time step; if the reward attenuation factor γ The closer it is to 0, the more likely the corresponding solution is to consider immediate rewards. γ The closer it is to 1, the more likely it is that the corresponding solution can balance future rewards and rewards at the current time step; if the environment state is known, the reward decay factor γ Set to 1.

6. The method for deploying an edge server of a mobile edge network according to claim 4, characterized in that: The delay matrix between the macro base stations is obtained by the shortest path algorithm. D , specifically: Generate an undirected graph based on a normal distribution G ( V , E ) and use Dijkstra's algorithm to calculate the weighted value of each edge in the undirected graph G ( V , E ) Zhonghong Base Station b i and b j The delay between D i,j .

7. An edge server deployment system for a mobile edge network, characterized in that: An edge server deployment solution for a mobile edge network applied to any one of claims 1-6, comprising a mobile edge computing network model and an edge server deployment module; The mobile edge computing network model includes a remote cloud computing center, multiple macro base stations, and multiple edge servers deployed in coordination with the macro base stations; The edge server is deployed at the macro base station to process service requests from multiple macro base station users nearby. At the same time, the edge server communicates with the remote cloud computing center to obtain more computing resources. The edge server deployment module is used to combine the user requests received by each macro base station in the edge network and the computing power of the edge server to obtain the best edge server deployment solution based on the Q-Learning server deployment algorithm.

8. A storage medium storing a program, characterized in that: When the program is executed by a processor, an edge server deployment solution for a mobile edge network as described in any one of claims 1 to 6 is implemented.

Images