Hotspot file edge distribution method and device, server and storage medium

Abstract

The application discloses a hotspot file edge distribution method and device, a server and a storage medium. The method comprises the following steps: acquiring file states corresponding to each file and a running state of a file server in a preset time period; performing normalization preprocessing on the file states and the running state to obtain processed multi-state time sequence data; judging the processed multi-state time sequence data through a hotspot file edge distribution model of multi-state DQN to obtain an edge distribution position adjustment action corresponding to each file; and adjusting a file position corresponding to the file according to the edge distribution position adjustment action, so that the decision of the distribution position adjustment action is made according to the collected file and the state information of the server, the distribution of the file is flexibly adjusted, the uneven utilization of server resources caused by the storage of the fixed file position is avoided, and the full utilization of the server is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, and particularly relates to a hot file edge distribution method and device, a server and a storage medium. BACKGROUND

[0002] In general, the storage location of a 5G message file is fixed, and cannot be automatically adjusted according to the heat of the file, so when a certain file has high concurrent reading, it is easy to cause the file download time delay to be lengthened, affecting the user's perception of 5G message use, and the resources of each file storage server cannot be balanced and fully utilized. SUMMARY

[0003] The main purpose of the present application is to provide a hot file edge distribution method, device, server and storage medium, which aims to solve the technical problem of how to improve the resource balance of the file storage server.

[0004] To achieve the above purpose, the present application provides a hot file edge distribution method, which comprises the following steps:

[0005] Obtain the file state corresponding to each file and the running state of the file server in a preset time period;

[0006] The file state and the running state are normalized and pretreated to obtain processed multi-state time series data;

[0007] The processed multi-state time series data is judged by a hot file edge distribution model of multi-state DQN to obtain an edge distribution position adjustment action corresponding to each file;

[0008] Adjust the file position corresponding to the file according to the edge distribution position adjustment action.

[0009] Optionally, before the step of adjusting the file position corresponding to the file according to the edge distribution position adjustment action, the method further comprises:

[0010] Judging whether the file position corresponding to the file needs to be adjusted according to the edge distribution position adjustment action;

[0011] When the file position corresponding to the file needs to be adjusted, the step of adjusting the file position corresponding to the file according to the edge distribution position adjustment action is executed.

[0012] Optionally, after the step of adjusting the file position corresponding to the file according to the edge distribution position adjustment action, the method further comprises:

[0013] Obtain the file state corresponding to the adjusted file and the running state of the file server;

[0014] a reward value is obtained according to the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution position adjustment action;

[0015] The hotspot file edge distribution model of the multi-state DQN is updated according to the reward value.

[0016] Optionally, the reward value is obtained according to the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution position adjustment action, and includes:

[0017] a preset strategy is determined according to the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution position adjustment action through a Q function;

[0018] a reward function is obtained;

[0019] a reward value is obtained according to the reward function, the preset strategy, the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution position adjustment action.

[0020] Optionally, the hotspot file edge distribution model of the multi-state DQN is updated according to the reward value, and includes:

[0021] an adjusted Q value is obtained according to the reward value, the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution position adjustment action;

[0022] an adjustment error is determined according to the adjusted Q value and a target Q value;

[0023] The hotspot file edge distribution model of the multi-state DQN is updated according to the adjustment error.

[0024] Optionally, before the processed multi-state time series data is judged by the hotspot file edge distribution model of the multi-state DQN to obtain the edge distribution position adjustment action corresponding to each file, the method further includes:

[0025] a historical file state set, a historical running state set of the file server, and a corresponding historical file distribution action set are obtained;

[0026] an input layer, a long short-term memory layer, a full connection layer, a merging layer, a random discard layer, and an output layer are established;

[0027] a judgment model composed of a long short-term memory neural network and a full connection neural network is established according to the input layer, the long short-term memory layer, the full connection layer, the merging layer, the random discard layer, and the output layer;

[0028] The historical file state set, the historical running state set of the file server and the corresponding historical file distribution action set are input into the evaluation model for training to generate a hot file edge distribution model of the multi-state DQN.

[0029] Optionally, the inputting the historical file state set, the historical running state set of the file server and the corresponding historical file distribution action set into the evaluation model for training to generate a hot file edge distribution model of the multi-state DQN comprises:

[0030] The historical file state set and the historical running state set of the file server are input into an input layer and a long short-term memory layer in the evaluation model respectively for feature extraction to obtain historical state vectors;

[0031] The historical file distribution action set is input into an input layer and a fully connected layer in the evaluation model for feature extraction to obtain a historical action vector;

[0032] The historical state vector and the historical action vector are trained through a merging layer, a fully connected layer and a random discard layer to output a historical Q value corresponding to a historical file distribution action;

[0033] A historical reward value is determined according to a historical file state in the historical file state set, a historical running state in the historical running state set of the file server and a historical file distribution action in the corresponding historical file distribution action set;

[0034] A historical target Q value is determined according to the historical file state, the historical running state, the corresponding historical file distribution action and the historical reward value;

[0035] The evaluation model is updated according to the historical Q value and the historical target Q value to generate a hot file edge distribution model of the multi-state DQN.

[0036] In addition, to achieve the above object, the application further provides a hot file edge distribution device, which comprises:

[0037] An acquisition module is configured to acquire file states corresponding to respective files and a running state of a file server in a preset time period;

[0038] A preprocessing module is configured to perform normalization preprocessing on the file states and the running state to obtain processed multi-state time series data;

[0039] An evaluation module is configured to evaluate the processed multi-state time series data through a hot file edge distribution model of the multi-state DQN to obtain an edge distribution position adjustment action corresponding to each file;

[0040] An adjusting module is configured to adjust a file position corresponding to the file according to the edge distribution position adjusting action.

[0041] In addition, to achieve the above object, the application further provides a hotspot file edge distribution server, which comprises a memory, a processor and a hotspot file edge distribution program stored in the memory and executable on the processor, and the hotspot file edge distribution program is configured to implement the hotspot file edge distribution method as described above.

[0042] In addition, to achieve the above object, the application further provides a storage medium, which stores a hotspot file edge distribution program, and the hotspot file edge distribution program is executable on a processor to implement the hotspot file edge distribution method as described above.

[0043] The hotspot file edge distribution method provided by the application obtains file states corresponding to each file and running states of a file server in a preset time period, performs normalization preprocessing on the file states and the running states to obtain processed multi-state time sequence data, judges the processed multi-state time sequence data through a hotspot file edge distribution model of multi-state DQN to obtain an edge distribution position adjusting action corresponding to each file, and adjusts a file position corresponding to the file according to the edge distribution position adjusting action, so that the distribution position adjusting action is determined according to the collected file and server state information, the file distribution is flexibly adjusted, the server resource utilization imbalance caused by the storage of fixed file positions is avoided, and the server is fully utilized. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a hardware running environment of an embodiment scheme of the application, and is a schematic structural diagram of a hotspot file edge distribution method device;

[0045] Figure 2 is a flowchart of a first embodiment of the hotspot file edge distribution method of the application;

[0046] Figure 3 is a whole flowchart of hotspot file distribution of an embodiment of the hotspot file edge distribution method of the application;

[0047] Figure 4 is a flowchart of a second embodiment of the hotspot file edge distribution method of the application;

[0048] Figure 5 is a flowchart of a third embodiment of the hotspot file edge distribution method of the application;

[0049] Figure 6A hot spot file edge distribution model network diagram of the multi-state DQN of the embodiment of the hot spot file edge distribution method of the application;

[0050] Figure 7 A function module diagram of the first embodiment of the hot spot file edge distribution device of the application.

[0051] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are merely intended to explain the application and are not intended to limit the application.

[0053] Reference Figure 1 , Figure 1 A device structure diagram of the hardware running environment involved in the embodiment scheme of the application.

[0054] As Figure 1 shown, the device can include a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen (Display), an input unit such as a key, and an optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface). The memory 1005 can be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a magnetic disk memory. The memory 1005 can also be an optional storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art can understand that the device structure of the hot spot file edge distribution method shown in the Figure 1 application does not constitute a limitation on the hot spot file edge distribution method device, and can include more or fewer components than the diagram, or combine certain components, or different component arrangements.

[0056] As Figure 1 shown, the memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and a hot spot file edge distribution method program.

[0057] In Figure 1The network interface 1004 is mainly used for connecting a server and communicating data with the server, and the user interface 1003 is mainly used for connecting a terminal and communicating data with the terminal. The hotspot file edge distribution method device of the present application calls the hotspot file edge distribution method program stored in the memory 1005 through the processor 1001, and executes the hotspot file edge distribution method provided by the embodiment of the present application.

[0058] Based on the above hardware structure, the embodiment of the hotspot file edge distribution method of the present application is proposed.

[0059] Reference Figure 2 , Figure 2 The flowchart of the first embodiment of the hotspot file edge distribution method of the present application is shown.

[0060] In the first embodiment, the hotspot file edge distribution method comprises the following steps:

[0061] Step S10, obtaining the file state corresponding to each file and the running state of the file server in a preset time period.

[0062] It should be noted that the execution subject of the present embodiment can be a hotspot file edge distribution server, and the hotspot file edge distribution server is provided with a hotspot file edge distribution method program, and can also be other devices that can realize the same or similar functions, and the present embodiment does not limit this. In the present embodiment, the hotspot file edge distribution server is taken as an example for description, and the hotspot file edge distribution application program is provided on the hotspot file edge distribution, and the message distribution can be carried out according to the hotspot file edge distribution application program.

[0063] In the embodiment, the file status includes the number of file calls and the file download time consumption, the file server includes a cloud server and an edge server, the file server status includes the average number of service requests per second and the average service request delay, and other status information can also be included, which is not limited in the embodiment, and the application scenario of the embodiment is that when 5G message files are stored, the storage location of the 5G message files is relatively fixed, and automatic adjustment of the storage location cannot be performed according to the heat of the files, so when a certain file has high concurrency reading, the file download delay is easily lengthened, the user's 5G message use perception is affected, and the resource balance and full use of each file storage server cannot be achieved. Therefore, the embodiment utilizes the advantage of deep reinforcement learning DQN (Deep Q-network) in processing high-dimensional status and discrete actions, adds multi-object status to the existing DQN model, the multi-object status includes the 5G message file status and the status of each cloud server and each edge server, and through learning of the multi-object status, the 5G message file is distributed to the best server location, so as to maximize the user's experience of downloading the 5G message file, wherein the preset time period can be 10 minutes, and other parameters can also be used, which is not limited in the embodiment, and the preset time period can be flexibly adjusted according to requirements.

[0064] 5G message provides enhanced personal and application message services for industry customers, realizes "message as a service", and introduces a new message interaction mode Chatbot chat robot. Through Chatbot, users can intuitively and conveniently enjoy various 5G application services such as payment and recharge, ticket ordering, hotel reservation, logistics inquiry, catering reservation, and takeout ordering in the message window. Chatbot is a service provided by an industry customer to a terminal user in the form of a dialogue. The service is usually based on artificial intelligence software, simulates human intelligent dialogue, and provides specific service functions to users.

[0065] 5G message service is based on the terminal native short message portal and provides sending and receiving of text, picture, audio, video, location, and contact media content for users, including point-to-point message, group message, group chat message, and point-to-application message. Compared with the single-function traditional short message, 5G message not only broadens the breadth of information transmission and reception, supports users to use multimedia content such as text, audio and video, cards, and location, but also extends the depth of interactive experience. Users can complete service search, discovery, interaction, and payment in the message window to build a one-stop service information window.

[0066] 5G message system, including 5G message center (5GMC), MaaP system (including MaaP platform management module and MaaP platform) and group chat server and other devices. The 5G message center is the core network element of the 5G message service. It has access, routing module and function, as a whole virtualized network function (Virtualized Network Function, VNF) for deployment, also has the processing capacity and external interface of the short message center. The network element will provide processing, sending, storage and forwarding functions for short messages and basic multimedia messages; the MaaP system is the core network element of the industry 5G message service, which provides 5G commercial message (MaaP) service access and message uplink and downlink capabilities for industry users, and provides industry chat robot search, detail query and message uplink and downlink functions; the group chat server provides group chat function for 5G message, including group chat message transmission and group information management functions.

[0067] The 5G message application open platform can help industry customers to realize A2P communication in multiple scenarios on demand. Enterprises can quickly complete the deployment of message applications through the platform without complex code development, helping industry customers to create their own 5G message applications simply and conveniently.

[0068] Since the storage locations of 5G message hot files and non-hot files are randomly stored on multiple cloud servers, there is currently no edge distribution scheme for 5G message hot files. The embodiment utilizes the advantage of deep reinforcement learning DQN (DeepQ-network) in processing high-dimensional states and discrete actions, adds multiple object states to the existing DQN model, and distributes 5G message files to the best server location through learning multiple object states.

[0069] Step S20, normalizing and preprocessing the file state and running state to obtain processed multi-state time series data.

[0070] In a specific implementation, the 5G message open platform obtains the file i state in the recent T time period, including the number of calls and download time consumption, and the state of each cloud server and edge server, including the average number of service requests per second and the average service request delay. The file i state and the state of each cloud server and edge server are input to the state data preprocessing module for time series data normalization preprocessing, as shown in the overall flowchart of hot file distribution. Figure 3 The file i state and the state of each cloud server and edge server are preprocessed to obtain multi-state time series data, so that the best server location can be determined according to the multi-object state, to improve the effectiveness of message storage.

[0071] Step S30, the processed multi-state time series data is judged by the hotspot file edge distribution model of multi-state DQN to obtain the edge distribution position adjustment action corresponding to each file.

[0072] In a specific implementation, the preprocessed file i state in the latest T time period and the state data of each file server are input into the 5G message hotspot file edge distribution model (agent) based on multi-state DQN. The 5G message hotspot file edge distribution model uses a long short-term memory neural network to extract the time series trend features of the file i state in the latest T time period and the state of each file server, respectively. The extracted multi-state features are combined into a feature vector. After passing through multiple fully connected layers and a random dropout layer, the 5G message file edge distribution position adjustment action is finally output. Thus, the determination of the message storage server position is realized by using a deep learning model according to the multi-object state.

[0073] Step S40, adjusting the file position according to the edge distribution position adjustment action.

[0074] It should be noted that the file state and the server state (state) include the user usage data of the message k issued by the chatbot i in the zth time period t; the edge distribution position adjustment action, i.e., the action (action), is the 5G message button setting modification action selected at time t corresponding to the message user usage data, which belongs to the discrete action space type. After the model selects the button optimization action is implemented through the 5G message open platform, the 5G message terminal user usage state is changed from s t to s t+1 . That is, the file state and the running state of the file server are judged by the hotspot file edge distribution model of multi-state DQN to obtain the position of the best server for storing the file, and the corresponding edge distribution position adjustment action is determined according to the position of the best server for storing the file, so as to adjust the stored file from the original position to the best server storage position, thereby realizing the balance of file server resources.

[0075] In the embodiment, the file state corresponding to each file and the running state of the file server in a preset time period are acquired; the file state and the running state are normalized and pretreated to obtain processed multi-state time sequence data; the processed multi-state time sequence data is judged by a hot file edge distribution model of multi-state DQN to obtain an edge distribution position adjustment action corresponding to each file; and the file position corresponding to the file is adjusted according to the edge distribution position adjustment action, so that the distribution position adjustment action is determined according to the collected file and server state information, the distribution of the file is flexibly adjusted, the uneven utilization of server resources caused by the storage of fixed file positions is avoided, and the full utilization of the server is improved.

[0076] In an embodiment, as shown in Figure 4 The second embodiment of the hot file edge distribution method of the application is proposed based on the first embodiment, and before the step S40, the method further includes:

[0077] In the step S401, it is judged whether the file position corresponding to the file needs to be adjusted according to the edge distribution position adjustment action.

[0078] When the file position corresponding to the file needs to be adjusted, the step S40 is executed.

[0079] Continuing as shown in Figure 3 The 5G message open platform compares the output action with the original file position to judge whether the file position needs to be adjusted. If the file position needs to be adjusted, the adjustment action is implemented. If the file position does not need to be adjusted, the adjustment action is not implemented, and the file server state after the action is implemented is fed back to the 5G message open platform.

[0080] In a specific implementation, after the hot file edge distribution model of multi-state DQN is judged to obtain the edge distribution position adjustment action corresponding to each file, the position of the server on which the file is best distributed is determined according to the edge distribution position adjustment action. The position of the server on which the file is best distributed is compared with the position of the server on which the file is stored. If the position of the server on which the file is best distributed is the same as the position of the server on which the file is stored, no adjustment is needed. If the position of the server on which the file is best distributed is different from the position of the server on which the file is stored, the action is implemented to adjust the server on which the file is stored, so that the flexible adjustment of the file distribution position is realized.

[0081] In an embodiment, after the step S40, the method further includes:

[0082] Obtain the file state corresponding to the adjusted file and the running state of the file server; obtain the reward value according to the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution location adjustment action; and update the hotspot file edge distribution model of the multi-state DQN through the reward value.

[0083] After the T time period of action implementation, the 5G message open platform calculates a reward function according to the state data of the adjusted file i and each file server, feeds back the value of the reward function to the 5G message hotspot file edge distribution model, and the model optimizes the model according to the reward value this time. The 5G message file is distributed to the best server location, and the user experience of downloading the 5G message file is maximized. The reward function can be determined by the user side slice instance using state, and can also be determined in other ways, which are not limited in the embodiment. The reward function is a Q function composed of a regular state and an action. The expectation of the discounted cumulative reward is reflected according to the Q function. The corresponding relationship between the expectation of the discounted cumulative reward and the discounted cumulative reward is determined.

[0084] In a specific implementation, the reward value is obtained according to the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution location adjustment action, specifically, the preset strategy is determined through a Q function according to the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution location adjustment action; a reward function is obtained; and the reward value is obtained according to the reward function, the preset strategy, the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution location adjustment action.

[0085] It should be noted that the reinforcement learning is an important machine learning method, which includes three elements of state, action, and reward. The agent needs to take action according to the current state, obtain the corresponding reward, and then improve the action, so that the agent can make a better action next time when it is in the same state. Q-Learning is a value-based algorithm in reinforcement learning algorithm, that is, the focus is to train a critic. Q is Q(s, a), which is the expected return of taking action a (a e A) in s state (s e S) at a certain time. The environment will feedback the corresponding reward according to the action of the agent, and r represents the reward. DQN (Deep Q-network) is a combination of Q-Learning and deep learning, which uses a deep neural network to represent the value function to predict the Q value, and learns the optimal action path by continuously updating the neural network.

[0086] The goal of DQN is to learn a policy π that maximizes the discounted cumulative reward over T time steps: γ represents the discount factor, r represents the single-step reward value, and R represents maximizing the discounted cumulative reward over T time steps. The single-step reward value is obtained from the discounted cumulative reward according to formula (I).

[0087]

[0088] The Q-function can be defined as the expected value of the discounted cumulative reward, which is based on the current state and the chosen action, with all subsequent actions made according to policy π:

[0089] Q π (s,a)=E s,a [R]; Formula (II)

[0090] We need to find a strategy that allows the Q function to reach its maximum value:

[0091]

[0092] DQN uses two neural networks: a target-net with relatively fixed parameters to obtain the Q-target value, and an eval-net to obtain the Q-eval value. The Q-value is updated according to the following rules:

[0093]

[0094] states t This includes user usage data for messages k sent by chatbot i within the t-th time period z; and action a. t The action selected at time t corresponds to the 5G message button setting modification action used by the user for that message, and belongs to the discrete action space type. After the optimized button action selected by the model is implemented through the 5G messaging open platform, the usage state of the 5G messaging terminal user changes from s t Convert to s t+1 ;r(s t ,c t ,a t The function is s t Under condition c t The following action a is to be executed t Then, the returned single-step reward value, the specific reward function of which is determined by the user-side slice instance using state s. t+1 To decide; R tis the weighted sum of reward values obtained by all behaviors from the current state to a certain future state, that is, the policy is determined according to formula (three), then the Q value is determined according to formula (two) and the policy, and the corresponding return value is obtained according to the Q value and formula (one), so that the hotspot file edge distribution model of the multi-state DQN is updated according to the return value.

[0095] In a specific implementation, updating the hotspot file edge distribution model of the multi-state DQN according to the return value includes: obtaining an adjusted Q value according to the return value, a file state corresponding to the adjusted file, a running state of the file server, and an edge distribution position adjustment action; determining an adjustment error according to the adjusted Q value and a target Q value; and updating the hotspot file edge distribution model of the multi-state DQN according to the adjustment error, wherein the adjusted Q value is Q(s1, s2, s3, a), that is, determined by the multi-object state and the edge distribution position adjustment action corresponding to each adjusted file, the target Q value is determined according to the multi-object state and the edge distribution position adjustment action corresponding to each file before adjustment, the value Q(s1, s2, s3, a) of the action selected this time is adjusted, the Q value output by the evaluator is compared with the target Q value Qtarget(s1, s2, s3, a) and the error is calculated, and the error signal is fed back to the deep neural network, so as to gradually improve the model accuracy.

[0096] In this embodiment, the file state corresponding to the adjusted file and the running state of the file server are obtained; the return value is obtained according to the file state corresponding to the adjusted file, the running state of the file server, and the edge distribution position adjustment action; and the hotspot file edge distribution model of the multi-state DQN is updated according to the return value, so as to update the hotspot file edge distribution model of the multi-state DQN according to the return value corresponding to the adjusted action, to improve the accuracy of the model.

[0097] In an embodiment, as shown in Figure 5 The third embodiment of the hotspot file edge distribution method of the present application is based on the first embodiment or the second embodiment, and will be described taking the first embodiment as an example. Before step S30, the method further includes:

[0098] In step S301, a historical file state set, a historical running state set of the file server, and a corresponding historical file distribution action set are obtained.

[0099] In this embodiment, in order to improve the accuracy of the sampling data, the sampling data is normalized. That is, the status of file i (number of times the file is called, file download time) and the status of each cloud server and edge server (average number of service requests per second, average service request latency) are collected from the 5G Message Open Platform every T time period in history. The corresponding file status and the 5G message file distribution actions under each server status are manually labeled as a total dataset. The multi-state time series data is then normalized and preprocessed.

[0100] The state of file i (s1) within time period T: including the number of times the file is accessed, can be represented as d1 = {d1} 1 d1 2 d1 3 、…、d1 T The download time for this file can be expressed as d2 = {d2} 1 d2 2 d2 3 , ..., d2 T};

[0101] The status of each cloud server within time period T (s2): including the average number of concurrent service requests per second (x1), can be expressed as x1 = {x1 1 x1 2 x1 3 ... x1 T The average concurrent service request latency (x2) can be expressed as x2 = {x2} 1 x2 2 x2 3 ... x2 T};

[0102] The status of each edge server within time period T (s3): including the average number of concurrent service requests per second (z1), can be represented as z1 = {z1} 1 z1 2 z1 3 ... z1 T The average concurrent service request latency (z2) can be expressed as z2 = {z2} 1 z2 2 z2 3 ... z2 T};

[0103] The 5G message file distribution actions (a) correspond to the file status and server status of each server: This includes m+n actions, corresponding to m cloud file servers and n edge file servers. All actions need to be pre-encoded; the value of each action is its corresponding code.

[0104] Then all attributes are standardized: (X-mean) / std. When calculating, each dimension is calculated separately, and the data is subtracted from the mean value according to the attribute (according to the column) and divided by the variance. After standardization, the convergence speed of the boosting model and the accuracy of the boosting model are improved.

[0105] Finally, the total data set is divided into training data and test data, and 80% of the entire data set is taken as training data, and the remaining 20% is taken as test data. The training set is used for training, and the test set is used to evaluate the model.

[0106] Step S302, an input layer, a long short-term memory layer, a full connection layer, a merging layer, a random discard layer and an output layer are established.

[0107] An improved evaluator composed of a long short-term memory neural network and a full connection neural network is built to estimate the Q function for evaluating the 5G message file distribution action. The preprocessed file i state s1, cloud server state s2, edge server state s3 and corresponding artificial annotation of the 5G message file distribution action in the T time period are input into the 5G message hotspot file edge distribution model based on multi-state DQN, and the value Q(s1, s2, s3, a) of the action selection is output. The Q value output by the evaluator is compared with the target Q value Q target (s1, s2, s3, a) and the error is calculated, and the error signal is fed back to the deep neural network, so as to gradually improve the model accuracy.

[0108] Step S303, a judgment model based on long short-term memory neural network and full connection neural network is established according to the input layer, long short-term memory layer, full connection layer, merging layer, random discard layer and output layer.

[0109] Step S304, the historical file state set, the historical running state set of the file server and the corresponding historical file distribution action set are input into the judgment model for training, and the multi-state DQN hotspot file edge distribution model is generated.

[0110] In order to realize the building of the multi-state DQN hotspot file edge distribution model, as Figure 6 The network diagram of the multi-state DQN hotspot file edge distribution model.

[0111] Branch 1: the number of times of calling the file i state (s1) in the T time period and the time consumed for downloading the file are input, and are respectively passed through two layers of long short-term memory layers (LSTM), 32 and 16 neurons are respectively set, and the activation function is "relu";

[0112] Branch 2: the average number of service requests per second and the average service request delay of each cloud server state (s2) in the T time period are input, and are respectively passed through two layers of long short-term memory layers (LSTM), 32 and 16 neurons are respectively set, and the activation function is "relu";

[0113] Branch 3: the average number of service requests per second and the average service request delay of each edge server state (s3) in the T time period are input, and are respectively passed through two layers of long short-term memory layers (LSTM), 32 and 16 neurons are respectively set, and the activation function is "relu";

[0114] Branch 4: the 5G message file distribution action (a) corresponding to the file state and the server state is input, and is passed through two layers of dense connection layers (Dense), 16 and 8 neurons are respectively set, and the activation function is "relu";

[0115] Then, the s1, s2, s3 and a passed through the feature extraction are merged through a merge layer, and are passed through two dense connection layers (Dense), 64 and 32 neurons are respectively set, and the activation function is "relu"; a dropout layer is respectively set after the two dense connection layers: the dropout probability is set to 0.2, the input neurons are randomly disconnected at a certain probability (20%) during the parameter update each time in the training process, and are used for preventing overfitting;

[0116] The output layer is composed of one dense connection neuron, outputs the Q value of the selected 5G message file distribution action, so as to realize the construction of the hotspot file edge distribution model of the multi-state DQN combining the long short-term memory neural network and the dense connection neural network.

[0117] In an embodiment, the step S304 comprises:

[0118] The historical file state set and the historical running state set of the file server are respectively input into the input layer and the long short-term memory layer of the evaluation model for feature extraction to obtain a historical state vector; the historical file distribution action set is input into the input layer and the full connection layer of the evaluation model for feature extraction to obtain a historical action vector; the historical state vector and the historical action vector are trained through a merging layer, a full connection layer and a random discard layer to output a historical Q value corresponding to the historical file distribution action;

[0119] A historical reward value is determined according to the historical file state in the historical file state set, the historical running state in the historical running state set of the file server and the historical file distribution action in the corresponding historical file distribution action set; a historical target Q value is determined according to the historical file state, the historical running state, the corresponding historical file distribution action and the historical reward value; the evaluation model is updated according to the historical Q value and the historical target Q value to generate a hot file edge distribution model of the multi-state DQN.

[0120] In this embodiment, the training of the hot file edge distribution model of the multi-state DQN is specifically described. An improved evaluator composed of a long short-term memory neural network and a full connection neural network is built to estimate the Q function for evaluating the 5G message file distribution action. The preprocessed file i state s1, cloud server state s2, edge server state s3 and the corresponding artificial annotation of the 5G message file distribution action in a T time period are input into the 5G message hot file edge distribution model based on the multi-state DQN together, and the value Q(s1, s2, s3, a) of the action selected this time is output. The Q value output by the evaluator is compared with the target Q value Q target (s1, s2, s3, a) and the error is calculated, and the error signal is fed back to the deep neural network, so as to gradually improve the model accuracy.

[0121] The training data is randomly extracted from the memory bank, which records the action, reward and next state result (s, a, r, s') under each state. The memory bank has a limited size, and when the data is full, the next data will overwrite the first data in the memory bank. Experience replay is used to save all stages of (s, a, r, s') to a replay memory. When training the neural network, a small batch is randomly selected from it for updating, rather than using the latest one, thereby solving the problem of mutual correlation between the sampled sample data, which will greatly improve the stability of the system.

[0122] To avoid the limitation of action selection and enrich the data collection, the greedy algorithm (epsilon greedy) is introduced to select the action, and the multi-object state is added in arg(maxQ(s, a)): arg(maxQ(s1, s2, s3, a)). The action is randomly selected with the probability of epsilon, and the currently known optimal action is selected with the probability of 1-epsilon. With the deepening of learning, the value of epsilon can become smaller and smaller, and the learning mode changes from full exploration to deep research.

[0123] The specific training process is to initialize the Q function with random weights, so that the target Q function Q target = Q. At each time step t in each round, given the state s 1t of the 5G message file i in the initialization time period T, the state s 2t of the cloud server, the state s 3t of the edge server, the 5G message file distribution action a t is given based on the greedy algorithm, the reward r t is obtained, and the new file state s 1(t+1) , the new cloud server state s 2(t+1) , and the new edge server state s s3(t+1) are reached. Store (s 1t , s 2t , s 3t , a t , r t , s 1(t+1) , s 2(t+1) , s 3(t+1) ) in the replay buffer; extract (s 1i , s 2i , s 3i , a t , r t , s 1(i+1) , s 2(i+1) , s 3(i+1) ) from the replay buffer, calculate the target value y = r i + maxQ target (s 1(i+1) , s 2(i+1) , s 3(i+1) , a), that is, determine the historical target Q value according to the historical file state, historical running state, corresponding historical file distribution action and historical reward value, update the parameters of the Q function neural network so that Q(s 1i , s 2i , s 3i , a i ) is closer to the target value y, and assign the updated Q function neural network weight to Q target = Q.

[0124] The error calculation method is as follows:

[0125]

[0126] The model will be trained for 1000 rounds, the batch size is set to 32 (batch_size=32), and the replay buffer size is set to 50000. The mean absolute error MSE (Mean Squared Error) is selected as the loss function, i.e. the objective function (loss='mse'), and the gradient descent optimization algorithm selects the adam optimizer to improve the learning speed of the traditional gradient descent (optimizer='adam'). The neural network can find the optimal weight value that minimizes the objective function through gradient descent, and as the number of training rounds increases, the training error also gradually decreases, and the model gradually converges. After offline training is completed, the derived neural network weight is calculated, i.e. the hot file edge distribution model of the multi-state DQN is obtained.

[0127] In the present embodiment, the historical file states in the historical file state set, the historical running states in the historical running state set of the file server, and the historical file distribution actions in the corresponding historical file distribution action set are trained based on the combination of long short-term memory neural network and fully connected neural network to generate the hot file edge distribution model of the multi-state DQN, realize the building and training of the model, and improve the accuracy of the model due to the use of multi-object state information for model training.

[0128] The application further provides a hot file edge distribution device.

[0129] Referring to Figure 7 , Figure 7 It is a functional module schematic diagram of the first embodiment of the hot file edge distribution device of the application.

[0130] In the first embodiment of the hot file edge distribution device of the application, the hot file edge distribution device comprises:

[0131] The acquisition module 10 is configured to acquire the file states corresponding to each file and the running state of the file server in a preset time period.

[0132] In the embodiment, the file status includes the number of file calls and the file download time consumption, the file server includes a cloud server and an edge server, the file server status includes the average number of service requests per second and the average service request delay, and other status information can also be included, and the embodiment is not limited thereto. The application scenario of the embodiment is that when 5G message files are stored, the storage position of the 5G message files is relatively fixed, and automatic adjustment of the storage position cannot be performed according to the heat of the files. Therefore, when a certain file has high concurrency reading, the file download delay is easily lengthened, the user's 5G message use perception is affected, and the resource balance and full use of each file storage server cannot be achieved. Therefore, the embodiment utilizes the advantage of deep reinforcement learning DQN in processing high-dimensional status and discrete actions, adds a multi-object status to the existing DQN model, the multi-object status includes the 5G message file status and the status of each cloud server and each edge server, and the 5G message file is distributed to the best server position through learning of the multi-object status, so as to maximize the user's experience of downloading the 5G message file. The preset time period can be 10 minutes, and other parameters can also be used, and the embodiment is not limited thereto. The preset time period can be flexibly adjusted according to requirements.

[0133] The 5G message service is based on a terminal native short message portal, and provides sending and receiving of text, picture, audio, video, location, and contact media content for users, including point-to-point messages, group message, group chat messages, and point-to-application messages. Compared with the single-function traditional short message, the 5G message not only widens the breadth of information transmission and supports users to use multimedia content such as text, audio and video, cards, and location, but also extends the depth of interactive experience. Users can complete service search, discovery, interaction, and payment in the message window, and build an information window of one-stop service.

[0134] The 5G message system includes a 5G message center (5GMC), a MaaP system (including a MaaP platform management module and a MaaP platform), and a group chat server. The 5G message center is a core network element of the 5G message service. It has an access and routing module and functions, is deployed as a whole VNF, and has the processing capacity and external interface of a short message center. The network element will provide processing, sending, storage, and forwarding functions for short messages and basic multimedia messages; the MaaP system is a core network element of the industry 5G message service. The network element will provide 5G commercial message (MaaP) service access and message uplink and downlink capabilities for industry users, and provide industry chat robot search, detail query, and message uplink and downlink functions for users; the group chat server provides group chat functions for the 5G message, including group chat message transmission and group information management.

[0135] The 5G message application open platform can help industry customers to realize A2P communication in multiple scenes on demand, and enterprises can quickly complete the deployment of message applications through the platform without complex code development, helping industry customers to simply and conveniently create their own 5G message applications.

[0136] Since the storage locations of the 5G message hot file and the non-hot file are randomly stored on multiple cloud servers, there is currently no edge distribution scheme for the 5G message hot file. The embodiment utilizes the advantage of deep reinforcement learning DQN in processing high-dimensional states and discrete actions, adds multi-object state to the existing DQN model, and distributes the 5G message file to the best server location through learning multi-object state.

[0137] The preprocessing module 20 is configured to normalize and preprocess the file state and the running state to obtain processed multi-state time series data.

[0138] In a specific implementation, the 5G message open platform obtains the file i state in the recent T time period, including the number of invocations and download time consumption, and the state of each cloud server and edge server, including the average number of service requests per second and the average service request delay. The file i state and the state of each cloud server and edge server are input to the state data preprocessing module for time series data normalization preprocessing, as shown in the overall flowchart of hot file distribution. Figure 3 The file i state and the state of each cloud server and edge server are preprocessed to obtain multi-state time series data, so that the best server location can be determined according to the multi-object state to improve the effectiveness of message storage.

[0139] The evaluation module 30 is configured to evaluate the processed multi-state time series data through the multi-state DQN hot file edge distribution model to obtain the edge distribution location adjustment action corresponding to each file.

[0140] In a specific implementation, the file i state and the file server state data (state) in the recent T time period after preprocessing are input to the 5G message hot file edge distribution model (agent) based on multi-state DQN. The 5G message hot file edge distribution model utilizes a long short-term memory neural network to extract the time series trend features of the file i state and the state of each file server in the recent T time period. The extracted multi-state features are combined into a feature vector, and after passing through multiple fully connected layers and random dropout layers, the 5G message file edge distribution location adjustment action (action) is finally output. Thus, the determination of the message storage server location is realized by using the deep learning model according to the multi-object state.

[0141] The adjusting module 40 is configured to adjust the file position corresponding to the file according to the edge distribution position adjustment action.

[0142] It should be noted that the file state and the server state (state) include the message k usage data of the user issuing the chatbot i in the tth time period z; the edge distribution position adjustment action, i.e., the action, is a 5G message button setting modification action selected at the tth time point corresponding to the message user usage data, which belongs to the discrete action space type. After the button optimization action selected by the model is implemented through the 5G message open platform, the 5G message terminal user usage state is s t converted into s t+1 . That is, the file storage best server position is obtained by judging the hot file edge distribution model of the multi-state DQN according to the file state and the running state of the file server, the corresponding edge distribution position adjustment action is determined according to the file storage best server position, the stored file is adjusted from the original position to the best server storage position, and the balance of the file server resources is realized.

[0143] In the embodiment, the file state and the running state of the file server corresponding to each file in a preset time period are obtained; the file state and the running state are normalized and preprocessed to obtain processed multi-state time sequence data; the processed multi-state time sequence data are judged by the hot file edge distribution model of the multi-state DQN to obtain the edge distribution position adjustment action corresponding to each file; and the file position corresponding to the file is adjusted according to the edge distribution position adjustment action, so that the distribution position adjustment action is decided according to the collected file and server state information, the file distribution is flexibly adjusted, the storage of the fixed file position is avoided to cause the unbalanced utilization of the server resources, and the full utilization of the server is improved.

[0144] In an embodiment, the adjusting module 40 is further configured to determine whether the file position corresponding to the file needs to be adjusted according to the edge distribution position adjustment action.

[0145] In an embodiment, the hot file edge distribution device further includes an updating module.

[0146] The updating module is configured to obtain the file state and the running state of the file server corresponding to the adjusted file.

[0147] A reward value is obtained according to the file state, the running state of the file server, and the edge distribution position adjustment action corresponding to the adjusted file.

[0148] The hot file edge distribution model of the multi-state DQN is updated through the reward value.

[0149] In an embodiment, the updating module is further configured to determine a preset policy according to the adjusted file corresponding file state, file server running state and edge distribution location adjustment action through a Q function;

[0150] obtain a reward function;

[0151] obtain a reward value according to the reward function, the preset policy, the adjusted file corresponding file state, file server running state and edge distribution location adjustment action.

[0152] In an embodiment, the updating module is further configured to obtain an adjusted Q value according to the reward value, the adjusted file corresponding file state, file server running state and edge distribution location adjustment action;

[0153] determine an adjustment error according to the adjusted Q value and a target Q value;

[0154] update the hotspot file edge distribution model of the multi-state DQN according to the adjustment error.

[0155] In an embodiment, the hotspot file edge distribution device further comprises a model establishing module;

[0156] The model establishing module is configured to obtain a historical file state set, a historical file server running state set and a corresponding historical file distribution action set;

[0157] establish an input layer, a long short-term memory layer, a full connection layer, a merging layer, a random discard layer and an output layer;

[0158] establish a judgment model based on a long short-term memory neural network and a full connection neural network according to the input layer, the long short-term memory layer, the full connection layer, the merging layer, the random discard layer and the output layer;

[0159] input the historical file state set, the historical file server running state set and the corresponding historical file distribution action set into the judgment model for training to generate a hotspot file edge distribution model of a multi-state DQN.

[0160] In an embodiment, the model establishing module is further configured to input the historical file state set and the historical file server running state set into the input layer and the long short-term memory layer in the judgment model respectively for feature extraction to obtain a historical state vector;

[0161] input the historical file distribution action set into the input layer and the full connection layer in the judgment model for feature extraction to obtain a historical action vector;

[0162] The historical state vector and the historical action vector are trained through a merging layer, a full connection layer and a random discard layer, and a historical Q value corresponding to a historical file distribution action is outputted;

[0163] A historical reward value is determined according to a historical file state in the historical file state set, a historical running state in the historical running state set of the file server and a historical file distribution action in the corresponding historical file distribution action set;

[0164] A historical target Q value is determined according to the historical file state, the historical running state, the corresponding historical file distribution action and the historical reward value;

[0165] The evaluation model is updated according to the historical Q value and the historical target Q value, and a hot file edge distribution model of the multi-state DQN is generated.

[0166] In addition, to achieve the above object, the application further provides a hot file edge distribution server, which comprises a memory, a processor and a hot file edge distribution program stored in the memory and executable on the processor, and the hot file edge distribution program is configured to implement the hot file edge distribution method as described above.

[0167] In addition, the application further provides a storage medium, which stores a hot file edge distribution program, and the hot file edge distribution program is executed by a processor to implement the hot file edge distribution method as described above.

[0168] Since the storage medium adopts all the technical solutions of the above-mentioned embodiments, it at least has all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here.

[0169] It should be noted that in this paper, the term "including", "containing" or any other variant thereof is intended to cover non-exclusive containing, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0170] The above-mentioned application embodiment serial numbers are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0171] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned example method can be realized by means of software and a necessary general hardware platform, and of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a computer readable storage medium (such as a ROM / RAM, a magnetic disk, or an optical disc) as described above, and includes a plurality of instructions for causing an intelligent terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the method described in each embodiment of the present application.

[0172] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for edge distribution of hotspot files, characterized in that, The hotspot file edge distribution method comprises: obtaining file states corresponding to each file and running states of a file server in a preset time period, wherein the file states comprise file calling times and file download time consumption, and the running states of the file server comprise average service request numbers per second and average service request time delays; normalizing and preprocessing the file states and the running states to obtain processed multi-state time sequence data; judging the processed multi-state time sequence data by using a multi-state DQN hotspot file edge distribution model to obtain edge distribution position adjustment actions corresponding to each file; adjusting file positions corresponding to the files according to the edge distribution position adjustment actions; obtaining file states corresponding to the adjusted files and running states of the file server; obtaining reward values according to the file states corresponding to the adjusted files, the running states of the file server and the edge distribution position adjustment actions; updating the multi-state DQN hotspot file edge distribution model by using the reward values; before the step of judging the processed multi-state time sequence data by using the multi-state DQN hotspot file edge distribution model to obtain edge distribution position adjustment actions corresponding to each file, the method further comprises: obtaining a historical file state set, a historical running state set of a file server and a corresponding historical file distribution action set; establishing an input layer, a long short-term memory layer, a full connection layer, a merging layer, a random discard layer and an output layer; establishing a judgment model based on a long short-term memory neural network and a full connection neural network according to the input layer, the long short-term memory layer, the full connection layer, the merging layer, the random discard layer and the output layer; inputting the historical file state set and the historical running state set of the file server into the input layer and the long short-term memory layer of the judgment model respectively to extract features to obtain historical state vectors; inputting the historical file distribution action set into the input layer and the full connection layer of the judgment model to extract features to obtain a historical action vector; training the historical state vector and the historical action vector by using the merging layer, the full connection layer and the random discard layer to output historical Q values corresponding to the historical file distribution actions; determining historical reward values according to historical file states in the historical file state set, historical running states in the historical running state set of the file server and historical file distribution actions in the corresponding historical file distribution action set; determining historical target Q values according to the historical file states, the historical running states, the corresponding historical file distribution actions and the historical reward values; updating the judgment model according to the historical Q values and the historical target Q values to generate a multi-state DQN hotspot file edge distribution model.

2. The method of claim 1, wherein the hotspot file edge delivery method is characterized by, before the step of adjusting file positions corresponding to the files according to the edge distribution position adjustment actions, the method further comprises: determining whether the file positions corresponding to the files need to be adjusted according to the edge distribution position adjustment actions; when the file positions corresponding to the files need to be adjusted, performing the step of adjusting the file positions corresponding to the files according to the edge distribution position adjustment actions.

3. The method of claim 1, wherein the hotspot file edge delivery method is characterized by, The reward value is obtained according to the adjusted file state corresponding to the file, the running state of the file server and the edge distribution position adjustment action, and the reward value comprises: The preset strategy is determined by a Q function according to the adjusted file state corresponding to the file, the running state of the file server and the edge distribution position adjustment action, wherein the Q function is defined as the expectation of the discounted cumulative reward, and the expectation is obtained based on the current state and the selected action, and the action is made according to the strategy; An reward function is obtained; The reward value is obtained according to the reward function, the preset strategy, the adjusted file state corresponding to the file, the running state of the file server and the edge distribution position adjustment action.

4. The method of claim 1, wherein the hotspot file edge delivery method is characterized by, The hotspot file edge distribution model of the multi-state DQN is updated by the reward value, and the updating comprises: An adjusted Q value is obtained according to the reward value, the adjusted file state corresponding to the file, the running state of the file server and the edge distribution position adjustment action; An adjustment error is determined according to the adjusted Q value and a target Q value; The hotspot file edge distribution model of the multi-state DQN is updated according to the adjustment error.

5. A hot spot file edge distribution apparatus, characterized by, The hotspot file edge distribution device comprises: An acquisition module is configured to acquire the file state corresponding to each file and the running state of the file server in a preset time period; A preprocessing module is configured to normalize and preprocess the file state and the running state to obtain processed multi-state time sequence data; A judgment module is configured to judge the processed multi-state time sequence data by the hotspot file edge distribution model of the multi-state DQN to obtain an edge distribution position adjustment action corresponding to each file; An adjustment module is configured to adjust the file position corresponding to the file according to the edge distribution position adjustment action. The hotspot file edge distribution device further comprises an update module, which is configured to acquire the file state corresponding to the adjusted file and the running state of the file server, obtain a reward value according to the adjusted file state corresponding to the file, the running state of the file server and the edge distribution position adjustment action, and update the hotspot file edge distribution model of the multi-state DQN by the reward value. The hotspot file edge distribution device further comprises a model establishment module, which is configured to acquire a historical file state set, a historical running state set of the file server and a corresponding historical file distribution action set; An input layer, a long short-term memory layer, a full connection layer, a merging layer, a random discard layer and an output layer are established; A judgment model based on a long short-term memory neural network and a full connection neural network is established according to the input layer, the long short-term memory layer, the full connection layer, the merging layer, the random discard layer and the output layer; The historical file state set and the historical running state set of the file server are respectively input into the input layer and the long short-term memory layer in the judgment model for feature extraction to obtain a historical state vector; The historical file distribution action set is input into the input layer and the full connection layer in the judgment model for feature extraction to obtain a historical action vector; and The history state vector and the history action vector are trained through a merging layer, a full connection layer and a random discard layer, and a history Q value corresponding to a history file distribution action is output; A history reward value is determined according to a history file state in the history file state set, a history running state in a history running state set of the file server and a history file distribution action in a corresponding history file distribution action set; A history target Q value is determined according to the history file state, the history running state, the corresponding history file distribution action and the history reward value; The evaluation model is updated according to the history Q value and the history target Q value, and a hot file edge distribution model of the multi-state DQN is generated.

6. A Hotspot File Edge Distribution Server, characterized by The hot file edge distribution server comprises a memory, a processor and a hot file edge distribution program stored on the memory and executable on the processor, and the hot file edge distribution program is configured to implement the hot file edge distribution method in any one of claims 1 to 4.

7. A storage medium, characterized by The storage medium stores a hot file edge distribution program, and the hot file edge distribution program is executed by the processor to implement the hot file edge distribution method in any one of claims 1 to 4.

Images