A service migration decision method, device, equipment and medium based on MEC environment

Abstract

The present invention relates to the field of mobile communications technology, and more specifically to a method, apparatus, device, and medium for service migration decision-making based on an MEC environment. The method specifically comprises: constructing a user mobility prediction model based on a Bidirectional LSTM model for many-to-many prediction using known human activity mobility data; establishing a DRL framework, wherein the DRL framework includes a state s, an action a, and a reward r, wherein the state s represents the distance between the user and each MEC server and the user's current location, the action a represents an MEC server that the user can select, and the reward r represents the reward for taking action a under state s; training the DRL framework using a DQN algorithm, and obtaining a service migration decision model based on the user mobility prediction model. The present invention addresses the problem of providing mobility services by combining the proposed DRL framework and the user mobility prediction model.

Description

Technical Field

[0001] The present invention relates to the field of mobile communication technologies, and in particular to a service migration decision method, apparatus, device, and medium based on an MEC environment. Background Art

[0002] Driven by 5G communications and the Internet of Things (IoT), mobile computing is shifting from centralized mobile cloud computing (MCC) to mobile edge computing (MEC). Proposed by the European Telecommunications Standards Institute (ETSI), MEC provides an IT service environment and cloud computing capabilities at the edge of mobile networks. One of the most challenging aspects of MEC is service provision that takes user mobility into account. Mobility prediction is a key function for enabling effective service migration in mobile service provision. Predicting a user's future location and time series allows for estimating the expected latency they may experience and selecting the optimal MEC server for them. In the early days of trajectory prediction, Markov models and hidden Markov models (HMMs) were the primary methods used. However, Markov models have limited performance in predicting complex human motion trajectories. With the rise and rapid development of deep learning technology in recent years, trajectory prediction using neural networks has become a new research approach. However, traditional recurrent neural networks (RNNs) are unable to address long-term dependencies. Summary of the Invention

[0003] The purpose of the present invention is to provide a service migration decision method, apparatus, device and medium based on the MEC environment. A BL-DRL framework is proposed to address the problem of mobile service provision in the MEC environment. Bidirectional LSTM is used for mobility prediction, and DRL is used for migration decision making. The problem of mobility service provision is addressed by combining the proposed DRL framework with a user mobility prediction model.

[0004] The present invention provides a service migration decision method based on an MEC environment, the method specifically comprising:

[0005] Based on the Bidirectional LSTM model for many-to-many prediction, a user mobility prediction model is constructed using known human activity mobility data;

[0006] Establish a DRL framework, which includes state s, action a, and reward r. State s represents the distance between the user and each MEC server and the user's current location, action a represents the MEC server that the user can choose, and reward r represents the reward for taking action a under state s.

[0007] The DRL framework is trained by the DQN algorithm, and a service migration decision model is obtained according to the user mobility prediction model. The service migration decision model is used to predict the future location of the user during the migration process and select the optimal MEC server for the user.

[0008] Furthermore, the Bidirectional LSTM model based on many-to-many prediction uses known human activity mobility data to build a user mobility prediction model. The specific steps include:

[0009] Divide the known human activity mobility data into training and test sets according to a preset ratio, and set the prediction window size;

[0010] Based on a many-to-many prediction Bidirectional LSTM model, the training set is differentiated to obtain a first differential sequence, the first differential sequence is normalized and then trained to obtain a model for predicting differential values;

[0011] Differentiating the test set to obtain a second difference sequence, and then inputting the second difference sequence into the model for predicting difference values ​​to obtain predicted difference values;

[0012] Adding the predicted difference value to the user's current location to obtain the user's predicted location within multiple time steps in the future;

[0013] An average prediction error is obtained by summing and averaging the absolute values ​​of the differences between the predicted position and the actual position;

[0014] The model of the prediction difference value is optimized according to the average prediction error to obtain a user movement prediction model.

[0015] Furthermore, the reward r satisfies the function value r=K-μ, where K represents the number of user requests and μ represents a fixed value.

[0016] Furthermore, the number of user requests is the quotient of a time step and the time it takes for the MEC server to process each user request, and the time step is the time interval for each collection of user location data.

[0017] Furthermore, the time taken by the MEC server to process each user request meets

[0018] T all =T trans +T comp +T mig (1)

[0019] or

[0020] T all =T trans +Tcomp (2)

[0021] Among them, T all T is the time it takes for the MEC server to process each user request. trans is the data transmission time, T comp MEC server computing time, T mig The migration time of the virtual machine in the MEC server.

[0022] Furthermore, the data transmission time is the quotient of the transmission data size and the communication capacity, the MEC server computing time is the quotient of the task load size and the MEC server computing power, and the virtual machine migration time within the MEC server is the quotient of the virtual machine size and the network bandwidth.

[0023] Furthermore, the DRL framework is trained using the DQN algorithm, specifically including:

[0024] Based on the DQN algorithm, a first neural network and a second neural network are set;

[0025] The first neural network is used to train the state s, action a and reward r of the DRL framework and obtain a predicted value. The first neural network adopts a greedy strategy to extract action a and stores each step state s, action a and reward r in the experience pool in an experience replay manner;

[0026] The second neural network is used to obtain the data of the experience pool for training and copy the data of the first neural network and update it in each first preset time period. The second neural network obtains the target value according to formula (3), and the target value satisfies

[0027] TargetQ=r+γmax a′ Q(s′,a′;θ) (3)

[0028] Among them, γ represents the attenuation factor, and θ represents the mean square error loss weight of the neural network parameters;

[0029] The first neural network is updated in every second preset time period according to the loss function, wherein the loss function is

[0030] L(e)=E[(TargetQ-Q(s,a;θ)) 2 ] (4)

[0031] Where TargetQ is the target value, Q(s, a, θ) is the predicted value, and formula (4) is the mean square error between the target value and the predicted value. The loss is minimized by updating θ through gradient descent.

[0032] The first neural network and the second neural network set action constraints to shield illegal actions when outputting actions.

[0033] The present invention also provides a service migration decision-making device based on an MEC environment, the device specifically comprising:

[0034] The first processing module is used to build a user mobility prediction model based on a Bidirectional LSTM model based on many-to-many prediction and using known human activity mobility data;

[0035] The second processing module is used to establish a DRL framework. The DRL framework includes a state s, an action a, and a reward r. The state s represents the distance between the user and each MEC server and the user's current location. The action a represents the MEC server that the user can select. The reward r represents the reward for taking action a under state s.

[0036] The third processing module is used to train the DRL framework through the DQN algorithm to obtain a service migration decision model based on the MEC environment. The service migration decision model based on the MEC environment is used to select the optimal MEC server for the user during the migration process.

[0037] The present invention also provides a computer device comprising: a memory and a processor and a computer program stored in the memory. When the computer program is executed on the processor, it implements the service migration decision method based on the MEC environment as described in any one of the above methods.

[0038] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a service migration decision method based on an MEC environment as described in any one of the above methods.

[0039] Compared with the prior art, the present invention has at least one of the following technical effects:

[0040] 1. We design a BL-DRL model that can jointly solve the provision of mobile services, i.e., migration problems in MEC environments.

[0041] 2. We extend the Bidirectional LSTM prediction model, which provides useful coarse-grained mobility information for M-DRL (mobility-aware deep reinforcement learning) training. We propose an independent training algorithm for this mobility model to achieve more accurate mobility prediction and model reuse.

[0042] 3. Differential processing of the mobile trajectory sequence eliminates the autocorrelation of the time series, solves the lag problem of LSTM prediction, and greatly improves the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Figure 1 This is a flow chart of a service migration decision method based on an MEC environment provided by an embodiment of the present invention;

[0045] Figure 2 This is a structural diagram of a service migration decision-making device based on an MEC environment provided by an embodiment of the present invention;

[0046] Figure 3 is a structural diagram of a computer device provided by an embodiment of the present invention;

[0047] Figure 4 Schematic diagram comparing the predicted trajectory and the actual trajectory of the Bidirectional LSTM model provided by an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram comparing the rewards and iterations of the Bidirectional LSTM model provided by an embodiment of the present invention and three other algorithms;

[0049] Figure 6 This figure is a schematic diagram comparing the delay and iteration of the Bidirectional LSTM model provided by an embodiment of the present invention and three other algorithms. DETAILED DESCRIPTION

[0050] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0051] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0052] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0053] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0054] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0055] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0056] Reference Figure 1 The present invention provides a service migration decision method based on an MEC environment, the method specifically comprising:

[0057] S101: Based on the Bidirectional LSTM model for many-to-many prediction, a user mobility prediction model is constructed using known human activity mobility data.

[0058] In this embodiment, user movement can lead to frequent location changes, resulting in service interruptions or suspensions. Therefore, analyzing user movement trajectories and predicting future trajectories has attracted extensive research in the communications field. The goal of mobility prediction is to use pedestrian trajectories from past time periods to build a model that can accurately predict pedestrians' future locations. By building a model that learns from human movement patterns, pedestrian trajectories can be predicted within a short period of time.

[0059] In some embodiments, the Bidirectional LSTM model based on many-to-many prediction uses known human activity mobility data to build a user mobility prediction model, specifically including the following steps:

[0060] Divide the known human activity mobility data into training and test sets according to a preset ratio, and set the prediction window size;

[0061] Based on a many-to-many prediction Bidirectional LSTM model, the training set is differentiated to obtain a first differential sequence, the first differential sequence is normalized and then trained to obtain a model for predicting differential values;

[0062] Differentiating the test set to obtain a second difference sequence, and then inputting the second difference sequence into the model for predicting difference values ​​to obtain predicted difference values;

[0063] Adding the predicted difference value to the user's current location to obtain the user's predicted location within multiple time steps in the future;

[0064] An average prediction error is obtained by summing and averaging the absolute values ​​of the differences between the predicted position and the actual position;

[0065] The model of the prediction difference value is optimized according to the average prediction error to obtain a user movement prediction model.

[0066] In this embodiment, the CRAWDAD NCSU dataset (I. Rhee, M. Shin, S. Hong, K. Lee, S. Kim, and S. Chong, “CRAW-DAD dataset ncsu / mobilitymodels (v. 2009-07-23),” Downloaded from https: / / crawdad.org / ncsu / mobilitymodels / 20090723, 2009) is used to simulate human mobility for real human traces. Mobility data from New York City in the dataset is used to train a mobility prediction model. A path containing both motion and stop modes is selected for training and testing, and user data in the dataset is divided into a training set and a test set in a ratio of 3:1. The prediction window size is typically set between 2 and 6. When the prediction window size is set to 2, the predicted value lags behind the true value. When the prediction window size is set to 6, the predicted data is mixed with irrelevant other data due to the large prediction window size. When the prediction window size is set to 5, it is more appropriate, but the problem of the predicted value lagging behind the true value still occurs. This is because user mobility data is a time series, and this time series is non-stationary (i.e., a series containing trends, seasonality, or cyclical characteristics). Observations in non-stationary time series exhibit trends, seasonal effects, or other structures that depend on time exponentially. Summary statistics such as mean and variance may change over time, distorting the concepts that the model is trying to capture. Classic time series analysis and forecasting methods are based on identifying trends and then eliminating them to make non-stationary time series data stationary. Therefore, differencing can be used to eliminate the series dependence on time. Differentiation is performed by subtracting the previous observation from the current observation. When the forecast needs to be converted back to the original scale, the difference needs to be reversed. This process can be reversed by adding the observation at the previous time step to the difference. In this way, a series of differences and contrasts can be calculated.

[0067] A difference operation is performed on both the training and test sets. This involves taking the difference between time t in the dataset and its previous time t-1, yielding a first difference sequence and a second difference sequence, respectively. Because users move a limited distance in a short period of time and their locations rarely change, the first difference sequence needs to be normalized before training the network using a Bidirectional LSTM model for many-to-many prediction. Otherwise, the network loss will be difficult to reduce, resulting in poor results. After training, a model for predicted difference values ​​is obtained. This model takes the difference sequence as input and outputs predicted difference values. Therefore, the second difference sequence is input into this model to obtain the user's predicted location. Furthermore, to improve the accuracy of this model, the absolute values ​​of the differences between the predicted and actual locations are summed and averaged to obtain the average prediction error. This model is then optimized based on the average prediction error, ultimately yielding a user movement prediction model.

[0068] Furthermore, compared to the shortcomings of recurrent neural networks (RNNs), such as difficulty in training and the tendency to cause gradient explosions, LSTMs, thanks to their unique internal structure: forget gate, input gate, and output gate, effectively avoid these training difficulties. Furthermore, thanks to the presence of these special gates, they also effectively control the ability to remember historical data. A typical Bidirectional LSTM performs a many-to-one prediction, using the previous n time steps to predict the next time step. This is done by expanding the predictions. At the current time t, when the data from time t-4 to time t is used to predict time t+1, the data from time t-3 to time t+1, which was predicted before new data is received, is used to predict the data for time t+2. This means that the window continues to shift using the feedback of the predicted data until the data for time t+5 is obtained, thus expanding the original LSTM's many-to-one prediction to a many-to-many prediction.

[0069] S102: Establish a DRL framework, which includes state s, action a, and reward r. State s represents the distance between the user and each MEC server and the user's current location, action a represents the MEC server that the user can select, and reward r represents the reward for taking action a under state s.

[0070] In some embodiments, the reward r satisfies the function value r=K-μ, where K represents the number of user requests and μ represents a fixed value.

[0071] Specifically, the number of user requests is the quotient of a time step and the time it takes for the MEC server to process each user request, and the time step is the time interval for each collection of user location data.

[0072] Specifically, the time for the MEC server to process each user request meets

[0073] T all =T trans +T comp +T mig (1)

[0074] or

[0075] T all =T trans +T comp (2)

[0076] Among them, T all T is the time it takes for the MEC server to process each user request. trans is the data transmission time, T comp MEC server computing time, T mig The migration time of the virtual machine in the MEC server.

[0077] Specifically, the data transmission time is the quotient of the transmission data size and the communication capacity, the MEC server computing time is the quotient of the task load size and the MEC server computing power, and the virtual machine migration time within the MEC server is the quotient of the virtual machine size and the network bandwidth.

[0078] In this embodiment, the DRL framework consists of a state s, an action a, and a reward r. The state s is represented by (L, d), where L represents the user's current location, including the horizontal and vertical coordinates, and is a two-dimensional vector. d represents the distance between the user and each server, and the action a represents the server that the user can choose. Therefore, the action a is the index of the server selected by the user. The best action at each moment is found through learning. The DRL algorithm is an algorithm that finds the maximum reward. In this DRL framework, the goal is to increase the total number of requests that can be processed, that is, to minimize the total delay. Therefore, a reward function r = K-μ is designed. Its purpose is to optimize the reward value, which is equivalent to achieving low delay. By subtracting the number of user requests K from a fixed value μ, the result is used as the value of the reward r. If the number of processed requests is higher than the threshold, it is a positive reward, and the more processed requests, the higher the reward value. If the number of processed requests is less than the threshold, it is a negative reward.

[0079] The number of user requests is the quotient of a time step and the time it takes for the MEC server to process each user request. The time step is the time interval for each collection of user location data. For example, in the above-mentioned CRAWDAD NCSU dataset, user location information is collected every 30 seconds, so the time step is 30 seconds. The time it takes for the MEC server to process each user request is equal to the sum of the data transmission time, the MEC server calculation time, and the virtual machine migration time within the MEC server, or the sum of the data transmission time and the MEC server calculation time (since the user location may not have changed or the change is not significant, there may be a situation where the virtual machine within the MEC service period does not need to be migrated).

[0080] The problem of providing mobile services in a MEC environment can be formulated as a mathematical problem. This environment consists of three components: MEC servers, user devices, and a task model. Assume that the MEC environment has a time counter that represents the environment's time, starting from 0 and ending at T. There are n MEC servers in the environment, processing user requests. Each MEC server has two properties: computing resources and computing power. Computing power, determined by the MEC server's CPU, determines how quickly the server can process requests. Data can be transmitted between MEC servers via connections, with the bandwidth of the links connecting MEC servers being B. Each MEC server is located at a different location in space. When a user sends a request to a server, the virtual machine on the MEC server processes the request. Upon receiving the request, the MEC server immediately performs computations on the request, with the computational load being the task. After the computation is complete, the MEC server sends the result back to the user's mobile device. Furthermore, when changing the MEC server providing services to a user, the virtual machines on the original MEC server need to be migrated to the new server.

[0081] The optimal solution to the mobile service provision problem is to minimize the average delay by properly migrating servers. The task delay consists of three parts: transmission delay, computation delay, and migration delay. Transmission delay depends on two factors: the data size and the distance between the user and the selected server. Shannon capacity is used to simulate wireless transmission in communication. Since the results generated by the MEC server are usually small, the transmission delay from the MEC server to the user can usually be ignored. By dividing the transmitted data size by the communication capacity, the transmission delay T can be calculated. trans ; Divide the task load size by the computing power C of the MEC server to get the computing delay T comp ; Migration delay T mig Depends on the size of the virtual machine and the network bandwidth.

[0082] S103: The DRL framework is trained using a DQN algorithm, and a service migration decision model is obtained based on the user mobility prediction model. The service migration decision model is used to predict the future location of the user during the migration process and select the optimal MEC server for the user.

[0083] In some embodiments, training the DRL framework based on the DQN algorithm specifically includes:

[0084] Based on the DQN algorithm, a first neural network and a second neural network are set;

[0085] The first neural network is used to train the state s, action a and reward r of the DRL framework and obtain a predicted value. The first neural network adopts a greedy strategy to extract action a and stores each step state s, action a and reward r in the experience pool in an experience replay manner;

[0086] The second neural network is used to obtain the data of the experience pool for training and copy the data of the first neural network and update it in each first preset time period. The second neural network obtains the target value according to formula (3), and the target value satisfies

[0087] TargetQ=r+γmax a′ Q(s′,a′;θ) (3)

[0088] Among them, γ represents the attenuation factor, and θ represents the mean square error loss weight of the neural network parameters;

[0089] The first neural network is updated in every second preset time period according to the loss function, wherein the loss function is

[0090] L(θ)=E[(TargetQ-Q(s,a;θ)) 2 ] (4)

[0091] Where TargetQ is the target value, Q(s, a; θ) is the predicted value, and formula (4) is the mean square error between the target value and the predicted value. The loss is minimized by updating θ through gradient descent.

[0092] The first neural network and the second neural network set action constraints to shield illegal actions when outputting actions.

[0093] In this embodiment, the first neural network can be used to input state s and output Q(s, a) corresponding to various actions a, and adopt a greedy strategy to select one of the various actions a. In order to ensure a certain amount of exploration, it is likely to select the action with the largest Q value, and input the action into the environment to obtain the next state s' and reward r, obtain an experience (experience): (s, a, s', r), and put the experience into the experience pool (Experience Replay Buffer). Assume that the second neural network obtains the experience from the experience pool (Experience Replay Buffer), inputs the state s' and outputs and selects action a', and finally uses Q(s, a; θ) as the network's predicted value, with TargetQ=r+γmax a′ Q(s′, a′; θ) is the target value of the network. Error backpropagation is then performed to update the first neural network. The loss function uses the mean square error. The gradient is calculated from formula (4) and the first neural network Q is updated once every second preset time period. The second neural network Target Q is updated once every first preset time period by setting Target Q = Q, where the second preset time period is a small constant and the first preset time period is a large constant.

[0094] During the inference phase, the network model selects the action with the highest probability. This model can only predict the migration of a single user. Since this embodiment only considers the single-user scenario, and virtual machines typically migrate to the original server or a server adjacent to the user, some actions are unreasonable. Furthermore, considering that the action space is too large to converge, an action mask is set at the end of the model to block illegal actions when outputting them, preventing them from interacting with the environment.

[0095] Reference Figure 2 The present invention provides a service migration decision device 2 based on an MEC environment, and the device 2 specifically includes: a first processing module 201, which is used to build a user mobility prediction model based on a Bidirectional LSTM model of many-to-many prediction using known human activity mobility data; a second processing module 202, which is used to establish a DRL framework, and the DRL framework includes a state s, an action a and a reward r, wherein the state s represents the distance between the user and each MEC server and the user's current location, the action a represents the MEC server that the user can select, and the reward r represents the reward for taking action a under the state s; a third processing module 203, which is used to train the DRL framework through a DQN algorithm to obtain a service migration decision model based on the MEC environment, and the service migration decision model based on the MEC environment is used to select the optimal MEC server for the user during the migration process.

[0096] It is understandable that if Figure 1 The contents of the embodiment of the service migration decision method based on the MEC environment shown in the embodiment are applicable to the embodiment of the service migration decision device based on the MEC environment. The functions specifically implemented by the embodiment of the service migration decision device based on the MEC environment are the same as those in the embodiment of the MEC environment. Figure 1 The embodiment of the service migration decision method based on the MEC environment shown is the same as that shown in FIG. Figure 1 The beneficial effects achieved by the embodiment of the service migration decision method based on the MEC environment shown are also the same.

[0097] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0099] Reference Figure 3 The present invention provides a computer device 3, comprising: a memory 302 and a processor 301 and a computer program 303 stored on the memory 302. When the computer program 303 is executed on the processor 301, the service migration decision method based on the MEC environment as described in any one of the above methods is implemented.

[0100] The computer device 3 can be a desktop computer, a notebook computer, a PDA, a cloud server or other computing devices. The computer device may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will understand that Figure 3 This is merely an example of the computer device 3 and does not constitute a limitation on the computer device 3 . The computer device 3 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device 3 may also include input and output devices, network access devices, etc.

[0101] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0102] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the computer device 3. Furthermore, the memory 302 may include both an internal storage unit of the computer device 3 and an external storage device. The memory 302 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 302 may also be used to temporarily store data that has been output or is about to be output.

[0103] The present invention also provides a computer-readable storage medium, characterized in that a computer program is stored thereon, and when the computer program is executed by a processor, the service migration decision method based on the MEC environment as described in any one of the above methods is implemented.

[0104] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0105] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0106] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] In the embodiments disclosed in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0108] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Reference Figure 4 , Figure 4 The figure shows a comparison between the predicted trajectory and the actual trajectory of the Bidirectional LSTM model provided by an embodiment of the present invention. The difference between the predicted trajectory and the actual trajectory is shown in the figure. It can be seen that the Bidirectional LSTM can accurately predict most trajectory points.

[0110] Table 1 Comparison of the results of the four algorithms in the dataset

[0111]

[0112] Table 1 Comparison of error results of the Bidirectional LSTM model provided by the embodiment of the present invention and the other three algorithms

[0113] Table 1 compares the single-step average prediction error, multi-step average prediction error, and average precision (AP) of Simple LSTM, Stack LSTM, seq2seq, and Bidirectional LSTM on the dataset. The results show that before differencing, seq2seq can achieve optimal predictions with a larger number of units, but this is often undesirable due to the large number of units. The other three algorithms achieve similar results with a smaller number of units. After differencing, Bidirectional LSTM outperforms the others, achieving superior prediction results with a smaller number of units. The single-step prediction error of Bidirectional LSTM is approximately 18m, while that of Simple LSTM is 19.3m, that of Seq2seq is 22.1m, and that of Stack LSTM is 25m. For multi-step predictions, the extended predictions of the other three algorithms all improve prediction accuracy, with Bidirectional LSTM achieving the best performance, reaching 14.9m.

[0114] Reference Figures 5 and 6, respectively, compared the rewards and iterations, and the delay and iterations of the Bidirectional LSTM model provided by the embodiment of the present invention and the other three algorithms.

[0115] exist Figure 5 and Figure 6 In this paper, after integrating the user mobility prediction model into the DQN algorithm, the prediction results of no prediction, linear prediction, sequence-to-sequence prediction, perfect prediction, and bidirectional LSTM prediction were compared. Linear prediction fits the user's movement trajectory with a linear function, resulting in large errors. Perfect prediction, which knows the user's precise future location without error, was chosen as the lower bound to demonstrate that the introduced user mobility prediction model can provide effective information for migration decisions. The average rewards of each benchmark (no prediction, linear prediction, bidirectional LSTM prediction, seq2seq prediction, and perfect prediction) were 279.42, 292.61, 318.89, 320.98, and 337.1, respectively, and the average latencies were 20.0393ms, 20.035ms, 20.0253ms, 20.0252ms, and 20.0189ms, respectively. Aside from the impossibility of achieving perfect prediction, the seq2seq prediction model performs better than linear and no prediction. However, seq2seq performs worse than the algorithm proposed in this embodiment of the present invention. The above data demonstrates that the reward function proposed in this embodiment of the present invention is effective for training. Although rewards are increased through training iterations, waiting time is reduced. The above results indicate that the algorithm proposed in this embodiment of the present invention performs slightly better than seq2seq. This demonstrates that, given the same user mobility prediction model, the extended Bidirectional LSTM provides more accurate predictions than seq2seq and more effective information for MEC server migration.

[0116] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A service migration decision method based on MEC environment, characterized in that: The method specifically includes: Based on the many-to-many prediction Bidirectional LSTM model, we use known human activity mobility data to build a user mobility prediction model. The specific steps include: Divide the known human activity mobility data into training and test sets according to a preset ratio, and set the prediction window size; Based on a many-to-many prediction Bidirectional LSTM model, the training set is differentiated to obtain a first differential sequence, the first differential sequence is normalized and then trained to obtain a model for predicting differential values; Differentiating the test set to obtain a second difference sequence, and then inputting the second difference sequence into the model for predicting difference values ​​to obtain predicted difference values; Adding the predicted difference value to the user's current location to obtain the user's predicted location within multiple time steps in the future; An average prediction error is obtained by summing and averaging the absolute values ​​of the differences between the predicted position and the actual position; Optimizing the prediction difference value model according to the average prediction error to obtain a user movement prediction model; Establish a DRL framework, which includes state s, action a, and reward r. State s represents the distance between the user and each MEC server and the user's current location. Action a represents the MEC server that the user can choose. Reward r represents the reward for taking action a under state s. The reward r satisfies the function r=K-µ, where K represents the number of user requests and µ represents a fixed value. The DRL framework is trained using the DQN algorithm, specifically including: Based on the DQN algorithm, a first neural network and a second neural network are set; The first neural network is used to train the state s, action a and reward r of the DRL framework and obtain a predicted value. The first neural network adopts a greedy strategy to extract action a and stores each step state s, action a and reward r in the experience pool in an experience replay manner; The second neural network is used to obtain the data of the experience pool for training and copy the data of the first neural network and update it in each first preset time period. The second neural network obtains the target value according to formula (3), and the target value satisfies （3） Among them, γ represents the attenuation factor, and θ represents the mean square error loss weight of the neural network parameters; The first neural network is updated in every second preset time period according to the loss function, wherein the loss function is （4） in, is the target value, is the predicted value, formula (4) is the mean square error between the target value and the predicted value, and the loss is minimized by updating θ through gradient descent; The first neural network and the second neural network set action constraints to shield illegal actions when outputting actions; A service migration decision model is obtained according to the user mobility prediction model, and the service migration decision model is used to predict the future location of the user during the migration process and select the optimal MEC server for the user.

2. The method according to claim 1, characterized in that The number of user requests is the quotient of a time step and the time it takes for the MEC server to process each user request. The time step is the time interval for each collection of user location data.

3. The method according to claim 2, characterized in that The time taken by the MEC server to process each user request meets （1） or （2） in, The time it takes for the MEC server to process each user request. is the data transmission time, Compute time for MEC servers, The migration time of the virtual machine in the MEC server.

4. The method according to claim 3, characterized in that The data transmission time is the quotient of the transmission data size and the communication capacity, the MEC server computing time is the quotient of the task load size and the MEC server computing power, and the virtual machine migration time within the MEC server is the quotient of the virtual machine size and the network bandwidth.

5. A service migration decision-making device based on MEC environment, characterized in that: The device specifically includes: The first processing module is used to build a user mobility prediction model based on a Bidirectional LSTM model for many-to-many prediction, using known human activity mobility data. The specific steps include: Divide the known human activity mobility data into training and test sets according to a preset ratio, and set the prediction window size; Based on a many-to-many prediction Bidirectional LSTM model, the training set is differentiated to obtain a first differential sequence, the first differential sequence is normalized and then trained to obtain a model for predicting differential values; Differentiating the test set to obtain a second difference sequence, and then inputting the second difference sequence into the model for predicting difference values ​​to obtain predicted difference values; Adding the predicted difference value to the user's current location to obtain the user's predicted location within multiple time steps in the future; An average prediction error is obtained by summing and averaging the absolute values ​​of the differences between the predicted position and the actual position; Optimizing the prediction difference value model according to the average prediction error to obtain a user movement prediction model; The second processing module is used to establish a DRL framework. The DRL framework includes a state s, an action a, and a reward r. The state s represents the distance between the user and each MEC server and the user's current location. The action a represents the MEC server that the user can select. The reward r represents the reward for taking action a under state s. The reward r satisfies the function r=K-µ, where K represents the number of user requests and µ represents a fixed value. The third processing module is used to train the DRL framework using the DQN algorithm, specifically including: Based on the DQN algorithm, a first neural network and a second neural network are set; The first neural network is used to train the state s, action a and reward r of the DRL framework and obtain a predicted value. The first neural network adopts a greedy strategy to extract action a and stores each step state s, action a and reward r in the experience pool in an experience replay manner; The second neural network is used to obtain the data of the experience pool for training and copy the data of the first neural network and update it in each first preset time period. The second neural network obtains the target value according to formula (3), and the target value satisfies （3） Among them, γ represents the attenuation factor, and θ represents the mean square error loss weight of the neural network parameters; The first neural network is updated in every second preset time period according to the loss function, wherein the loss function is （4） in, is the target value, is the predicted value, formula (4) is the mean square error between the target value and the predicted value, and the loss is minimized by updating θ through gradient descent; The first neural network and the second neural network set action constraints to shield illegal actions when outputting actions; A service migration decision model based on an MEC environment is obtained, where the service migration decision model based on the MEC environment is used to select an optimal MEC server for a user during a migration process.

6. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory, which, when executed on the processor, implements the service migration decision method based on the MEC environment as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the service migration decision method based on the MEC environment according to any one of claims 1 to 4 is implemented.

Images