Time delay optimization method and device, electronic equipment and nonvolatile storage medium

Abstract

The invention discloses a time delay optimization method and device, electronic equipment and a nonvolatile storage medium. The method comprises: determining a user access decision when a user accesses a cellular network, the user access decision being used for determining a target content request time delay for performing a content request to an accessed base station, the target content request time delay being the minimum content request time delay in a target time slot, and the target content request time delay being the minimum content request time delay in the target time slot; the target time slot is any time slot in the target time frame; determining a target bandwidth allocation decision based on the user access decision, the target bandwidth allocation decision being used for indicating bandwidth allocation of a base station accessed by the user to the user; and determining a cache decision for time delay optimization in the target time frame based on the user access decision and the target bandwidth allocation decision. The technical problems of large long-term content distribution delay, low content distribution efficiency and low caching efficiency during multi-time-scale caching in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a method, device, electronic device and non-volatile storage medium for delay optimization. Background Art

[0002] Edge caching includes two stages: cache placement and content distribution. In order to improve cache efficiency, there are several algorithms in related technologies: Belief Propagation algorithm (BP algorithm for short): In each time slot, the Belief Propagation algorithm calculates the optimal cache placement matrix and user access matrix based on the system state of the time slot to implement cache placement content update and user access reset. The BP algorithm needs to update cache placement, user access and bandwidth allocation decisions in each time slot, which means that there will be cache update delay in each time slot, which brings a larger time overhead compared to other algorithms. Twin Delayed Deep Deterministic Policy Gradient (TD3 algorithm for short) updates cache placement, user access and bandwidth allocation decisions at the same time on a long time scale of the time frame, which brings a larger content transmission delay while obtaining a smaller content update delay. The least frequently used strategy (LFU) and the maximum carrier to interference ratio (Max C / I) algorithm calculate the optimal cache placement vector in each time frame, and determine the user access vector in each time slot based on the maximum transmission speed between the user and the base station. The LFU and Max C / I algorithms lack a learning mechanism for long-term problems and cannot try to reduce the content update delay overhead caused by the large difference in cache placement decisions between the previous time frame and the current time frame.

[0003] Therefore, when the above method is used for caching at multiple time scales, there are problems such as long-term content distribution delay, low content distribution efficiency, and low caching efficiency.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method, device, electronic device and non-volatile storage medium for delay optimization, so as to at least solve the technical problems of long-term content distribution delay, low content distribution efficiency and low caching efficiency existing in the related art when performing multi-time scale caching.

[0006] According to one aspect of an embodiment of the present application, a method for latency optimization is provided, comprising: determining a user access decision when a user accesses a cellular network, wherein the user access decision is used to determine a target content request delay for making content requests to the accessed base station, the target content request delay being the minimum content request delay within a target time slot, and the target time slot being any time slot in a target time frame; determining a target bandwidth allocation decision based on the user access decision, wherein the target bandwidth allocation decision is used to indicate bandwidth allocation to the user by the base station accessed by the user; and determining a cache decision for latency optimization within the target time frame based on the user access decision and the target bandwidth allocation decision.

[0007] In some embodiments of the present application, the content request delay of the user is determined in the following manner: when there is cached content in the first base station accessed by the user in the target time slot, the first requested content is obtained from the first base station; the content request delay is determined based on the transmission rate from the first base station to the user, the request result of the first requested content, and the file size of the first requested content; when there is no cached content in the first base station accessed by the user in the target time slot, and there is cached content in the adjacent base station of the first base station, the second requested content is obtained from the adjacent base station; the content request delay is determined based on the transmission rate from the first base station and the adjacent base station to the user, the request result of the second requested content, and the file size of the second requested content; when there is no cached content in the first base station accessed by the user in the target time slot and the adjacent base station of the first base station, the third requested content is obtained from the core network; the content request delay is determined based on the transmission rate from the first base station to the user, the average delay in transmitting the third requested content, the request result of the third requested content, and the file size of the third requested content.

[0008] In some embodiments of the present application, determining a user access decision when a user accesses a cellular network includes: solving a first objective function using a first algorithm within a target time slot, wherein the first objective function is used to indicate a minimum content request delay when all users make content requests to the connected base station; determining the minimum content request delay as a target content request delay, and determining the target content request delay and the user corresponding to the target content request delay and the base station accessed by the user as the user access decision of the user.

[0009] In some embodiments of the present application, a first algorithm is used to solve a first objective function within a target time slot, including: establishing an initial matching state between each base station and the user based on a preset preference list, wherein the preference list is used to indicate the priority of each user accessing each base station and the priority preference of each base station for the accessed user; performing exchange matching of user access to the base station based on the initial matching state until a preset matching condition is reached to obtain a solution to the first objective function, wherein the preset matching condition is that the content request delay of the user exchanged after any exchange matching is higher than the content request delay before the exchange and the total delay after the exchange is higher than the total delay before the exchange.

[0010] In some embodiments of the present application, determining a target bandwidth allocation decision based on a user access decision includes: solving a second objective function using a second algorithm within a target time slot, wherein the second objective function is used to indicate an optimal bandwidth allocation decision allocated to the user by the base station to which the user accesses under the user access decision; and determining a minimum bandwidth obtained by solving the second objective function using the second algorithm as the target bandwidth allocation decision.

[0011] In some embodiments of the present application, a second objective function is solved using a second algorithm within a target time slot, including: determining a second-order derivative of a target content request delay based on a user access decision within the target time slot; when the second-order derivative is greater than 0, determining that the target content request delay is a convex function; and using a convex optimization algorithm to solve the second objective function to obtain a minimum bandwidth, and determining the minimum bandwidth as a target bandwidth allocation decision.

[0012] In some embodiments of the present application, a cache decision for delay optimization is determined based on a user access decision and a target bandwidth allocation decision, including: solving a third objective function using a third algorithm within a target time frame to determine a cache decision for delay optimization, wherein the third objective function is determined based on a cache update delay of a previous time frame of the target time frame as well as the user access decision and the target bandwidth allocation decision.

[0013] In some embodiments of the present application, a third objective function is solved using a third algorithm within a target time frame to determine a cache decision for latency optimization, including: initializing parameters of a comment network and an actor network in the third algorithm, wherein the comment network is used to evaluate the value of a given state and action, and the actor network is used to obtain a cache placement strategy based on a given state; updating a first content popularity to a second content popularity within a time frame, wherein the content popularity represents the change in the probability of a user requesting different content over time, the first content popularity is used to represent the content popularity at a previous moment, and the second content popularity is used to represent the content popularity at a next moment to the previous moment; based on the second content popularity, determining an initial cache strategy using a third algorithm, and selecting an action through the actor network to update the initial cache strategy to obtain a current cache strategy; when the change between the current cache strategy and the initial cache strategy is less than a preset threshold, determining the current cache strategy as the cache decision for latency optimization.

[0014] According to another aspect of an embodiment of the present application, a delay optimization device is also provided, including a first determination module, used to determine a user access decision when a user accesses a cellular network, wherein the user access decision is used to determine a target content request delay for making a content request to the accessed base station, the target content request delay is the minimum content request delay within a target time slot, and the target time slot is any time slot in a target time frame; a second determination module, used to determine a target bandwidth allocation decision based on the user access decision, wherein the target bandwidth allocation decision is used to indicate the bandwidth allocation of the base station to which the user is accessed to the user; a third determination module, used to determine a cache decision for delay optimization within the target time frame based on the user access decision and the target bandwidth allocation decision.

[0015] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, in which a program is stored, wherein when the program is running, a device where the non-volatile storage medium is located is controlled to execute the above-mentioned delay optimization method.

[0016] According to another aspect of an embodiment of the present application, an electronic device is further provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the above-mentioned delay optimization method is executed when the program is running.

[0017] According to another aspect of an embodiment of the present application, a computer program product is also provided, including computer instructions, which implement the above-mentioned delay optimization method when executed by a processor.

[0018] In an embodiment of the present application, a user access decision is determined when a user accesses a cellular network, wherein the user access decision is used to determine a target content request delay for making a content request to the accessed base station, the target content request delay being the minimum content request delay within a target time slot, and the target time slot being any time slot in a target time frame; a target bandwidth allocation decision is determined based on the user access decision, wherein the target bandwidth allocation decision is used to indicate bandwidth allocation of the base station accessed by the user to the user; a cache decision for delay optimization within a target time frame is determined based on the user access decision and the target bandwidth allocation decision, by respectively obtaining the user access decision and the target bandwidth allocation decision in the target time slot, and then obtaining the cache decision for delay optimization in the target time frame, thereby achieving the purpose of delay optimization when caching at a time scale, thereby solving the technical problems of large long-term content distribution delay, low content distribution efficiency, and low cache efficiency when caching at multiple time scales in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing a method for delay optimization provided in an embodiment of the present application;

[0021] Figure 2 It is a flowchart of a delay optimization method provided according to an embodiment of the present application;

[0022] Figure 3 It is a framework diagram of a many-to-one exchange matching algorithm provided according to an embodiment of the present application;

[0023] Figure 4 It is a structural diagram of a delay optimization device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or reject automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0028] Collaborative caching: Base station multi-hop transmission expands the base station cache space. When the base station itself has no cache, it can seek help from neighboring base stations and form a collaborative cache with the neighboring base stations.

[0029] Multiple time scales: The time scales for updating content cache placement decisions and content requests are different, so time scale separation is used to form multiple time scales. This application considers three scales: the short time scale for user access decisions, the short time scale for bandwidth allocation decisions, and the long time scale for content cache decisions.

[0030] In the related art, when caching at multiple time scales, there is a long-term content distribution delay, low content distribution efficiency, and low caching efficiency. In order to solve this problem, the embodiments of the present application provide a related solution.

[0031] According to an embodiment of the present application, a method embodiment of latency optimization is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a method for delay optimization. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0033] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0034] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the delay optimization method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the delay optimization method described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0037] In the above operating environment, the present application embodiment provides a flowchart of a method for delay optimization, such as Figure 2 The method comprises the following steps:

[0038] Step S202, determining a user access decision of a user when accessing a cellular network, wherein the user access decision is used to determine a target content request delay for making a content request to the accessed base station, the target content request delay being a minimum content request delay within a target time slot (e.g., t time slot, t greater than 0), the target time slot being any time slot in a target time frame (e.g., T time frame, a T time frame consisting of multiple t time slots).

[0039] In the technical solution provided in step S202, there are multiple ways to determine the user access decision when the user accesses the cellular network, for example: using the first algorithm to solve the first objective function in the target time slot, wherein the first objective function is used to indicate the minimum content request delay when all users make content requests to the accessed base station; determining the minimum content request delay as the target content request delay, and determining the target content request delay and the user corresponding to the target content request delay and the base station accessed by the user as the user access decision of the user. This method can achieve the optimal match between the user and the base station, especially in a complex network environment with multiple users and multiple base stations, and improve the overall performance of the network and user experience.

[0040] In the above steps, solving the first objective function using the first algorithm within the target time slot can be achieved in a variety of ways, for example: establishing an initial matching state between each base station and the user based on a preset preference list, wherein the preference list is used to indicate the priority of each user accessing each base station and the priority preference of each base station for the connected user; performing exchange matching of user access to the base station based on the initial matching state until the preset matching condition is reached, and obtaining the solution of the first objective function, wherein the preset matching condition is that the content request delay of the user exchanged after any exchange matching is higher than the content request delay before the exchange and the total delay after the exchange is higher than the total delay before the exchange. Through iterative optimization, dynamic adjustment of user access decisions is achieved, effectively responding to dynamic changes in the network environment, and is particularly suitable for scenarios with high user mobility and frequent changes in network conditions, such as urban bus and subway network coverage.

[0041] It should be noted that the content request delay of the user in the above steps is determined in the following manner: when there is cache content in the first base station accessed by the user in the target time slot, the first request content is obtained from the first base station; the content request delay is determined based on the transmission rate from the first base station to the user, the request result of the first request content, and the file size of the first request content; when there is no cache content in the first base station accessed by the user in the target time slot, and there is cache content in the adjacent base station of the first base station, the second request content is obtained from the adjacent base station; the content request delay is determined based on the transmission rate from the first base station and the adjacent base station to the user, the request result of the second request content, and the file size of the second request content; when there is no cache content in the first base station accessed by the user in the target time slot and the adjacent base station of the first base station, the third request content is obtained from the core network; the content request delay is determined based on the transmission rate from the first base station to the user, the average delay of transmitting the third request content, the request result of the third request content, and the file size of the third request content. In this way, the best content acquisition path can be intelligently selected, and the content request delay is further shortened, especially in the case of network congestion or limited cache resources, and the resource utilization efficiency is improved.

[0042] The following is a specific embodiment:

[0043] Since the time scale for updating content cache placement decisions is very different from the time scale for content requests, this application uses a time scale separation approach to implement user access decisions, bandwidth allocation decisions, and cache placement decisions. Using time scale separation, the optimization problem can be split into the following three sub-problems (short time scale user access solution, short time scale bandwidth allocation solution, and long time scale cache placement solution) to solve. The short time scale user access solution is a complex global problem with multiple discrete variables, which can be solved by an exhaustive algorithm. However, the complexity of the exhaustive algorithm is exponential and is only applicable to small-scale networks. For this sub-problem, a many-to-one exchange matching algorithm (i.e., the first algorithm mentioned above) is used to solve it.

[0044] like Figure 3 As shown, a framework diagram of a many-to-one exchange matching algorithm is provided. The base station-user matching (BS-usermatching) adopts a many-to-one exchange matching algorithm to solve the user access sub-problem. The number of base stations (BSs) is 1-M, the number of users (users) is 1-K, and the base station set and the user set are two disjoint sets. A base station can access multiple users, and a user can access at most one base station, so this problem meets the many-to-one matching problem.

[0045] The first objective function is solved in the t time slot using a many-to-one exchange matching algorithm. For example, the first objective function can be expressed by the following formula:

[0046]

[0047] in, represents the minimum content request delay, K represents the total number of users, represents the content request delay of user k requesting content from base station m in time slot t, n represents the requested content, and there are N contents in the cellular network, respectively. represents, st represents the constraint condition, m represents the base station, M represents the total number of base stations, Indicates that user k accesses base station m in time slot t, and vice versa It means that user k does not access base station m in time slot t.

[0048] Content request latency It can be determined in the following ways: According to whether base station m has cached content n, it can be divided into the following three cases:

[0049] Case 1: The base station m accessed by the user in time slot t has cached content n, and user k directly obtains the requested content from the base station (that is, when the first base station accessed by the user in the target time slot has cached content, the first requested content is obtained from the first base station). Whether user k requests content n is determined by (i.e. the request result of the first request content above) indicates that, indicates that user k requests content n, and vice versa Indicates that user k accesses base station m in time slot t, and vice versa With matrix X T Indicates the base station active cache strategy, Indicates that in time frame T, base station m has content n in cache. Indicates that there is no content n in base station m in time frame T, and the content request delay of time slot t is:

[0050]

[0051] in, Indicates the content request latency. represents the request result of the requested content, f represents the file size of the requested content (here, the file size of the first requested content), represents the transmission rate from user k to base station m in time slot t.

[0052] Case 2: In time slot t, the user requests content that is not stored in base station m, but is stored in the neighboring base station b. Base station m obtains the content from base station b and transmits it to the user (i.e., in the above case where there is no cached content in the first base station accessed by the user in the target time slot, but there is cached content in the neighboring base station of the first base station, the second requested content is obtained from the neighboring base station). is the transmission rate from base station b to base station m in time slot t, and the content request delay in time slot t is:

[0053]

[0054] in, represents the transmission rate from base station b to base station m in time slot t, represents the transmission rate from user k to base station m in time slot t, Indicates the content request latency. represents the request result of the requested content, f represents the file size of the requested content (here, the file size of the second requested content), It indicates that in time frame T, base station b has content n cached.

[0055] Case 3: In time slot t, a user requests content, but base station m does not have any storage, and neither does a neighboring base station. Base station m obtains the content from the core network and transmits it to the user. The average delay for transmitting file n of size f through the wired backhaul link is:

[0056]

[0057] Among them, λ b and λ gare the densities of base stations and gateways respectively, and a, b and κ are constants reflecting the processing capabilities of nodes. The content acquisition delay of time slot t is:

[0058]

[0059] Among them, D 0 Indicates the average delay of transmitting the third request content.

[0060] The delay of user k requesting file n in time slot t can be expressed as:

[0061]

[0062] Where b∈M\m means b is any base station in M ​​that is not the same as m, R 0 Indicates the preset initial transmission rate. Indicates that there is cache content n in base station b in time slot T. This can be determined by:

[0063]

[0064] in, represents the transmission rate from base station m to user k in time slot t, represents the signal-to-noise ratio in time slot t.

[0065] It can be expressed by the following formula:

[0066]

[0067] in, q ∈M\{m} means q and m are two different base stations, P m represents the average transmission power of base station m, P q represents the average transmission power of base station q, σ 2 represents additive white Gaussian noise, represents the path loss between user k and base station m in time slot t, represents the distance between user k and base station m at time t, represents the distance between user k and base station q at time t, α represents the path loss index, represents the channel gain of base station m in time slot t, represents the channel gain of base station q at time slot t. The random channel between the base station and the user obeys Rayleigh fading, that is, the channel gain of time slot t The average power is 1. Assuming that the downlink channels from the base station to the user are independent and identically distributed, the average content request delay of the user in the T time frame is

[0068] The process of solving the first objective function using the first algorithm within the target time slot is as follows:

[0069] The Gale-Shapley (GS) algorithm is used to establish the initial matching state between the base station and the user. In the initialization process based on the GS algorithm, it is assumed that Thus, the impact of other users' access changes on the user's transmission rate is ignored. During the initialization process, based on the preset preference list, each user applies for access to the base station with the highest priority according to the preference list, and the base station accepts the user with the highest priority among the requesting users according to the preference list. When each user accesses the base station, the GS algorithm ends and the initialization is completed. Based on this state, the exchange is performed.

[0070] Define function μ to represent the access matching between users and base stations, which is defined as follows:

[0071] Definition: A many-to-one 2D matching function μ is established based on the preference list of the user and the base station. The definition of μ is as follows

[0072] (1) Among them, μ(k) represents the matching on the user side, k represents the user, and K represents the total number of users.

[0073] (2) Wherein, μ(m) represents the matching on the base station side, m represents a base station, and M represents the total number of base stations.

[0074] (3)

[0075] In this problem, the user's goal is to obtain a smaller content request latency. The goal of the base station is to reduce the total delay of access users. represents the total delay of base station m in time slot t.

[0076] Therefore, for any user k and two base stations Two matchings μ, μ', m = μ(k), m' = μ'(k) have the following properties This indicates that compared with the request delay obtained by user k accessing base station m' under matching μ' When user k accesses base station m under matching μ, he can get a smaller request delay Similarly, for any base station m and two groups of users Α, Α'∈K, Α≠Α', two matching Has the following properties This indicates that base station m prefers user set A, that is, the match μ that can obtain a smaller content request delay.

[0077] When the user accessed by base station m changes, it may affect the request delay of user k∈μ(m). Externality is an external influence of one subject on another subject. In this problem, there is a certain influence between user accesses, so this problem is a many-to-one matching problem with external characteristics. Many-to-one exchange matching is used to solve this problem. The definition of exchange matching is as follows:

[0078] Exchange match definition:

[0079]

[0080] m=μ(k), m'=μ(k'), m≠m', k≠k'.

[0081] In exchange matching, a matching pair is considered stable if and only if it satisfies the definition of a bilaterally stable matching, which is defined as follows:

[0082] Definition: A matching μ is a bilaterally stable matching if and only if there is no such pair of elements (m,m'). Where m=μ(k), m'=μ(k'), (m,m') satisfies:

[0083] (1) represents the content request delay of base station m after switching, represents the content request delay of base station m' after switching, represents the content request delay of base station m before switching, represents the content request delay of base station m' before switching.

[0084] (2) The exchange matching base stations m and m' exchange the users accessed, and other base stations and users remain unchanged. Reduce the total request delay of the base station without sacrificing the delay of each user. The base station exchanges users and the users agree to this exchange. Such a match is unstable. If any exchange match is irrelevant to the user or base station, and there is no delay reduction, a stable match is found, and the cycle between equivalent matches can be avoided, because it ensures that the delay of each terminal is not increased and the total delay of accessing the base station is strictly reduced.

[0085] The behavior of users in many-to-one matching can be described as follows: any two users can be arranged by the access base station to form a candidate exchange matching pair. The access base station checks whether it can benefit from exchanging matched users without damaging its own interests. Through a series of exchange operations, the matching can reach a stable state and a minimum content request delay can be obtained. First, randomly select any base station and a user accessing the base station, and find another user from the adjacent access node. In order to reduce the user's delay and, the access base stations m and m' observe whether their user delays are reduced after matching and exchanging. If the delay is reduced, the access base station m sends the exchange application to m'. Users k and k' calculate the exchange delay, and only if the request delay of each user is not increased, will each user agree to perform the exchange operation. Otherwise, the exchange will be interrupted. Update their preference lists according to the changes in the matching pairs. Repeat the above process until a stable match is achieved. At the end of the iteration, the preset matching condition is met, and the preset matching condition is that the content request delay of the user exchanged after any exchange matching is higher than the content request delay before the exchange and the total delay after the exchange is higher than the total delay before the exchange. The solution process of the above first algorithm is summarized in Table 1.

[0086] Table 1

[0087]

[0088] Step S204: determining a target bandwidth allocation decision based on the user access decision, wherein the target bandwidth allocation decision is used to indicate bandwidth allocation of the user by the base station accessed by the user.

[0089] In the technical solution provided in step S204, there are multiple ways to solve the second objective function using the second algorithm in the target time slot, for example: determining the second-order derivative of the target content request delay based on the user access decision in the target time slot; when the second-order derivative is greater than 0, determining the target content request delay as a convex function; using a convex optimization algorithm to solve the second objective function to obtain the minimum bandwidth, and determining the minimum bandwidth as the target bandwidth allocation decision. It is possible to intelligently allocate bandwidth resources according to user access decisions to ensure that each user can obtain sufficient bandwidth support, especially during periods of high traffic demand, such as lunch breaks on weekdays and evening entertainment peaks, avoiding network congestion and improving user experience.

[0090] The following is a specific embodiment: For the short-time scale bandwidth allocation problem, within the t time slot, the second objective function can be expressed by the following formula:

[0091]

[0092] in, represents the bandwidth allocated by base station m to user k in time slot t, Indicates the minimum bandwidth.

[0093] The process of solving the second objective function using the second algorithm in the target time slot is as follows: the base station cache placement matrix X T and the user access matrix (i.e. user access decision) L t Considered as a constant value, the content request delay varies according to the different conditions of the base station cache. There are different forms of expression:

[0094] Case 1: The local base station has cached:

[0095] right Find the first-order derivative

[0096]

[0097] Taking the second-order derivative we get:

[0098]

[0099] It can be seen from this

[0100] Case 2: The local base station has no cache, but the neighboring base station has cached it according to the formula:

[0101] By derivation, we can get

[0102] Case 3: Neither the local nor the neighboring base station has a cache, and the cache is obtained from the core network: According to the formula:

[0103] By derivation, we can get

[0104] In summary, it can be concluded that And the domain is a convex set, so this subproblem is a convex problem. For this convex problem, a convex optimization algorithm is used to solve it: the CVXPY toolbox is used to solve the optimal bandwidth allocation strategy to obtain the minimum bandwidth, and the minimum bandwidth is determined as the target bandwidth allocation decision. The above bandwidth allocation ensures that all users can obtain reasonable resource allocation, avoiding some users from being in a low service quality state for a long time due to poor location or infrequent content requests. By optimizing bandwidth allocation, especially by utilizing the cache resources of neighboring base stations, direct requests to the core network can be reduced, thereby reducing the load on the core network and improving the robustness and scalability of the network.

[0105] Step S206: determining a cache decision for delay optimization within a target time frame based on the user access decision and the target bandwidth allocation decision.

[0106] In the technical solution provided in step S206, there are multiple ways to implement the cache decision for delay optimization based on the user access decision and the target bandwidth allocation decision, for example: using a third algorithm to solve the third objective function within the target time frame to determine the cache decision for delay optimization, wherein the third objective function is determined based on the cache update delay of the previous time frame of the target time frame, the user access decision and the target bandwidth allocation decision. By utilizing the convex optimization theory, the optimal solution can be found quickly, especially in large-scale networks, which improves the computational efficiency of the algorithm and the accuracy of the decision.

[0107] For example, the third objective function can be solved by the third algorithm within the target time frame to determine the cache decision for delay optimization in the following manner: initialize the parameters of the comment network and the actor network in the third algorithm, wherein the comment network is used to evaluate the value of a given state and action, and the actor network is used to obtain a cache placement strategy according to a given state; update the first content popularity to the second content popularity within a time frame, wherein the content popularity represents the change in the probability of a user's request for different content over time, the first content popularity is used to represent the content popularity at the previous moment, and the second content popularity is used to represent the content popularity at the next moment of the previous moment; based on the second content popularity, use the third algorithm to determine the initial cache strategy, and select an action through the actor network to update the initial cache strategy to obtain the current cache strategy; when the change between the current cache strategy and the initial cache strategy is less than a preset threshold, determine the current cache strategy as the cache decision for delay optimization. It can intelligently predict and adjust the cache strategy based on historical data and the current network status, which is particularly suitable for scenarios with frequent content updates and changing user needs, such as video on demand and online games, to improve the cache hit rate and reduce the latency of content requests.

[0108] The following is a specific embodiment:

[0109] For the sub-problem of solving the long-time-scale cache placement, the cache decision is updated every T time frame, so the delay is determined by the backhaul link download delay. If the content required within the T time frame (i.e. the above target time frame) has been cached in the base station within T-1 time frame, there is no need to re-download; if the content required within the T time frame is not cached by the base station within T-1 time frame, it is necessary to re-download the cache. Therefore, the cache update delay within the T time frame (i.e. the third objective function mentioned above) is

[0110]

[0111] Among them, H takes the value of 0 and The maximum value in It indicates that the base station m has content n cached in the T-1 time frame.

[0112] Using TD3 (i.e., the third algorithm mentioned above) to solve the third objective function using the third algorithm, the parameters of the comment network and the actor network in the third algorithm are initialized, wherein the comment network is used to evaluate the value of a given state and action, and the actor network is used to obtain a cache placement strategy based on a given state; updating the first content popularity to the second content popularity within a time frame, wherein the content popularity represents the change in the probability of a user's request for different content over time, the first content popularity is used to represent the content popularity at the previous moment, and the second content popularity is used to represent the content popularity at the next moment of the previous moment; based on the second content popularity, determining an initial cache strategy using the third algorithm, and selecting an action through the actor network to update the initial cache strategy to obtain a current cache strategy; when the change between the current cache strategy and the initial cache strategy is less than a preset threshold, determining the current cache strategy as a cache decision for latency optimization.

[0113] When using TD3 (i.e., the third algorithm mentioned above) to determine the cache decision for latency optimization, the above problem is modeled as a Markov decision process, using<A,S,P,r,γ> express.

[0114] 1. S is a set of states, state s j ∈S. State characteristics s T =E T .

[0115] 2. A represents the action set, state a T =X T The action space size is 2 MN .

[0116] 3. P(s'|s,a) represents the transition probability function.

[0117] 4. r represents the immediate reward, which is the content distribution delay in the system over a long time scale. Specifically, according to the given system state, the optimal user access L for each time slot can be obtained respectively. t and bandwidth allocation W t (That is, the above user receives and then calculates the content request delay Cache update delay Based on this, the content distribution delay within each time frame is calculated

[0118] In the reinforcement learning algorithm based on value learning, function approximation error is the main reason for the overestimation of Q value and suboptimal strategy in TD3 algorithm. On the basis of dual Q learning, by selecting the smaller value of the two valuation functions, the overestimation of Q value is reduced, and the delayed update strategy is used to reduce the error of each update. By considering the mutual influence between the function approximation error between the policy function and the valuation function in the AC framework (Actor-Critic architecture), the AC network (reinforcement learning network under the Actor-Critic architecture) is optimized on the basis of Deep Deterministic Policy Gradient (DDPG) and the TD3 algorithm is proposed. Improvements to the TD3 algorithm:

[0119] (1) Due to its essential characteristics, Q-value learning will inevitably lead to the problem of Q-value overestimation. That is, when maximizing the noisy Q-value, the valuation function will continue to overestimate the Q-value. On the one hand, due to the existence of noise, although the possibility of an action being overestimated and underestimated is the same, after the maximization operation, the overestimated curve is more likely to be selected. The greater the noise, the greater the possibility of overestimation and the degree of overestimation. On the other hand, due to the approximation error, the overestimation bias begins to occur. This phenomenon is more common in algorithms such as DDPG with continuous action space.

[0120] The method of intercepting the dual Q value learning gradient is used to alleviate the problem of overestimation of Q value in the AC framework. θ 1 is the parameter estimation network, θ 2 is a parameter estimation network, s and a are states and actions respectively, and the same memory is used for simultaneous estimation. Selecting the less overestimated value as the estimated value can alleviate the overestimate phenomenon. The specific formula is:

[0121]

[0122] Where r represents the reward, γ is the discount factor, is the action selection strategy.

[0123] In this way, overestimation is avoided. Although it may lead to underestimation bias, the underestimation error will not be propagated through policy updates.

[0124] (2) Another common problem in determining policy gradient algorithms is overfitting of the valuation function.

[0125] To this end, a regularization strategy is introduced to smooth the target strategy. In actual use, a small variance noise is added to the target strategy, and the expected value of the action is updated in small batches on average:

[0126]

[0127] in, ε is random noise, c is the parameter of the noise distribution, Represents the distribution of a.

[0128] In the TD3 network, the review network parameters are trained by minimizing the mean squared error:

[0129]

[0130] in, Y represents the sample (s i ,a i ,r i ,s i+1 ), is the Q value of the comment network i. The actor network is trained according to the policy gradient, and the parameters φ of the actor network are improved according to the deterministic policy gradient to obtain a better Q value. The actor network updates the parameters according to the following formula:

[0131]

[0132] Where φ is the parameter of the actor-target network, a = π φ (s) represents a deterministic behavior strategy. The above steps are summarized in Table 2:

[0133] Table 2

[0134]

[0135]

[0136] User access decision refers to the strategy of deciding which base station each user should access in each short-time slot. In a dynamic cellular network environment, due to the change of user location and the instantaneous nature of content requests, user access decision must be able to quickly adapt to changes in network status to ensure that users can obtain services from the nearest base station that caches their requested content, thereby reducing the transmission delay of the content. This decision is jointly determined by the preferences between the base station and the user, as well as the current cache status and transmission capacity of the network. The target bandwidth allocation decision is the strategy for each base station to decide how to allocate its limited bandwidth resources among the connected users after the user access decision is determined. The purpose of bandwidth allocation is to maximize the network transmission efficiency and reduce the content transmission delay while satisfying user content requests. Due to the limited bandwidth resources and the dynamic changes in user needs, the bandwidth allocation decision needs to take into account the cache status and user access status while ensuring fairness to optimize the use of resources. The cache decision is the strategy of deciding which content each base station should cache at the beginning of the long-time frame. This decision is based on the popularity distribution of the content, the user request probability, and the transmission characteristics of the network. The goal of caching decisions is to store the content that is most likely to be requested within a limited cache space, so that it can be directly obtained from local or neighboring base stations when requested by users, reducing or avoiding the delay of downloading from the core network. Caching decisions need to take into account the limitations of cache space, the frequency of content updates, and the size of the content to minimize long-term content distribution delays. These three decisions are closely related. User access decisions and target bandwidth allocation decisions are made on a short time scale, aiming to respond to changes in network status as immediately as possible, while caching decisions are made on a longer time scale, taking into account longer-term content distribution efficiency and latency optimization. By jointly optimizing these three decisions, the algorithm can dynamically adjust on multiple time scales to improve the performance of edge caches in cellular networks.

[0137] Through the above steps, the waiting time of users requesting content in cellular networks can be significantly reduced, especially in high-density user areas such as stadiums, concert venues or commercial centers, improving user satisfaction and network efficiency. A cache architecture for cellular networks under multiple time scales is proposed. The long-term content distribution delay is minimized by jointly optimizing cache placement and user access bandwidth allocation. The user access algorithm based on many-to-one exchange matching, the bandwidth allocation algorithm based on convex optimization and the cache placement algorithm based on TD3 are respectively used to solve each sub-problem. Finally, the suboptimal solution is obtained through iteration, which significantly improves the content distribution efficiency, effectively reduces the user acquisition delay and content update delay, and improves the performance of the cache network. The above steps constitute a joint optimization algorithm, which decomposes the optimization problem that is difficult to solve directly into three sub-problems, and proposes corresponding solution algorithms respectively, specifically solving the short-time scale user access strategy based on many-to-one exchange matching, solving the short-time scale bandwidth allocation based on convex optimization, and solving the long-time scale cache placement strategy based on TD3. The three sub-algorithms iterate to obtain the final result. Since the TD3 reinforcement learning algorithm requires a period of offline training, it cannot be directly applied to online problem solving. The following Table 3 summarizes the joint optimization algorithm:

[0138] Table 3

[0139]

[0140] The proposed joint optimization algorithm provided by this application uses two time scales, making cache placement decisions on a long time scale and making user access and bandwidth allocation decisions on a short time scale. By performing cache placement updates on a long time scale, the update delay overhead caused by cache content updates in the base station is significantly reduced, while user access and bandwidth allocation decisions on a short time scale ensure low content transmission delay. Therefore, the proposed joint optimization algorithm provided by this application can still guarantee relatively superior performance as the time scale increases.

[0141] The present application also provides a structure diagram of a delay optimization device, such as Figure 4 As shown, including:

[0142] The first determination module 402 is used to determine the user access decision when the user accesses the cellular network, wherein the user access decision is used to determine the target content request delay for making content requests to the accessed base station, the target content request delay is the minimum content request delay within the target time slot, and the target time slot is any time slot in the target time frame.

[0143] The first determination module 402 is also used to solve the first objective function using a first algorithm within the target time slot, wherein the first objective function is used to indicate the minimum content request delay when all users make content requests to the connected base station; the minimum content request delay is determined as the target content request delay, and the target content request delay and the user corresponding to the target content request delay and the base station accessed by the user are determined as the user access decision of the user.

[0144] The first determination module 402 is also used to establish an initial matching state between each base station and the user based on a preset preference list, wherein the preference list is used to indicate the priority of each user accessing each base station and the priority preference of each base station for the connected user; based on the initial matching state, exchange matching of user access to the base station is performed until a preset matching condition is reached to obtain a solution to the first objective function, wherein the preset matching condition is that the content request delay of the user exchanged after any exchange matching is higher than the content request delay before the exchange and the total delay after the exchange is higher than the total delay before the exchange.

[0145] The second determination module 404 is used to determine a target bandwidth allocation decision based on the user access decision, wherein the target bandwidth allocation decision is used to indicate bandwidth allocation to the user by the base station accessed by the user.

[0146] The second determination module 404 is also used to solve the second objective function using the second algorithm within the target time slot, wherein the second objective function is used to indicate the optimal bandwidth allocation decision allocated to the user by the base station to which the user accesses under the user access decision; and the minimum bandwidth obtained by solving the second objective function using the second algorithm is determined as the target bandwidth allocation decision.

[0147] The second determination module 404 is also used to determine the second-order derivative of the target content request delay based on the user access decision within the target time slot; when the second-order derivative is greater than 0, determine the target content request delay as a convex function; use a convex optimization algorithm to solve the second objective function to obtain the minimum bandwidth, and determine the minimum bandwidth as the target bandwidth allocation decision.

[0148] The third determination module 406 is configured to determine a cache decision for delay optimization within a target time frame based on the user access decision and the target bandwidth allocation decision.

[0149] The third determination module 406 is also used to solve the third objective function using a third algorithm within the target time frame to determine a cache decision for delay optimization, wherein the third objective function is determined based on the cache update delay of the previous time frame of the target time frame as well as the user access decision and the target bandwidth allocation decision.

[0150] The third determination module 406 is also used to initialize the parameters of the comment network and the actor network in the third algorithm, wherein the comment network is used to evaluate the value of a given state and action, and the actor network is used to obtain a cache placement strategy based on a given state; updating the first content popularity to the second content popularity within a time frame, wherein the content popularity represents the change in the probability of a user's request for different content over time, the first content popularity is used to represent the content popularity at the previous moment, and the second content popularity is used to represent the content popularity at the next moment of the previous moment; based on the second content popularity, using the third algorithm to determine the initial cache strategy, and selecting an action through the actor network to update the initial cache strategy to obtain the current cache strategy; when the change between the current cache strategy and the initial cache strategy is less than a preset threshold, determining the current cache strategy as a cache decision for latency optimization.

[0151] It should be noted that the various modules in the above-mentioned delay optimization device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0152] The embodiment of the present application also provides a non-volatile storage medium, the non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above delay optimization method. For example, determine the user access decision of the user when accessing the cellular network, wherein the user access decision is used to determine the target content request delay for making content requests to the accessed base station, the target content request delay is the minimum content request delay within the target time slot, and the target time slot is any time slot in the target time frame; determine the target bandwidth allocation decision based on the user access decision, wherein the target bandwidth allocation decision is used to indicate the bandwidth allocation of the base station to which the user is connected to the user; determine the cache decision for delay optimization within the target time frame based on the user access decision and the target bandwidth allocation decision.

[0153] The embodiment of the present application also provides an electronic device, the electronic device includes a processor, the processor is used to run a program, wherein the above delay optimization method is executed when the program is running. For example, a user access decision is determined when a user accesses a cellular network, wherein the user access decision is used to determine a target content request delay for making a content request to the accessed base station, the target content request delay is the minimum content request delay within a target time slot, and the target time slot is any time slot in a target time frame; a target bandwidth allocation decision is determined based on the user access decision, wherein the target bandwidth allocation decision is used to indicate the bandwidth allocation of the base station to which the user is connected to the user; a cache decision for delay optimization within the target time frame is determined based on the user access decision and the target bandwidth allocation decision.

[0154] According to another aspect of the embodiment of the present application, a computer program product is also provided, including a computer program, which implements the above delay optimization method when executed by a processor. For example, determining a user access decision when a user accesses a cellular network, wherein the user access decision is used to determine a target content request delay for making a content request to the accessed base station, the target content request delay is the minimum content request delay within a target time slot, and the target time slot is any time slot in a target time frame; determining a target bandwidth allocation decision based on the user access decision, wherein the target bandwidth allocation decision is used to indicate the bandwidth allocation of the base station to which the user is accessed to the user; determining a cache decision for delay optimization within the target time frame based on the user access decision and the target bandwidth allocation decision.

[0155] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0160] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for delay optimization, characterized in that: include: Determine a user access decision when the user accesses the cellular network, wherein the user access decision is used to determine a target content request delay for making a content request to the accessed base station, the target content request delay being a minimum content request delay within a target time slot, the target time slot being any time slot in a target time frame; Determine a target bandwidth allocation decision based on the user access decision, wherein the target bandwidth allocation decision is used to indicate bandwidth allocation of the base station accessed by the user to the user; A caching decision for latency optimization within the target time frame is determined based on the user access decision and the target bandwidth allocation decision.

2. The method according to claim 1, characterized in that The content request latency of the user is determined by: In a case where cached content exists in a first base station accessed by the user in the target time slot, obtaining first requested content from the first base station; Determining the content request delay according to a transmission rate from the first base station to the user, a request result of the first request content, and a file size of the first request content; In the case where there is no cache content in the first base station accessed by the user in the target time slot, and there is cache content in a neighboring base station of the first base station, obtaining the second request content from the neighboring base station; Determine the content request delay according to transmission rates from the first base station and the neighboring base station to the user, respectively, a request result of the second request content, and a file size of the second request content; When there is no cache content in the first base station accessed by the user and the neighboring base stations of the first base station in the target time slot, obtaining the third request content from the core network; The content request delay is determined according to a transmission rate from the first base station to the user, an average delay in transmitting the third requested content, a request result of the third requested content, and a file size of the third requested content.

3. The method according to claim 1, characterized in that Determine the user access decision when the user accesses the cellular network, including: Solving a first objective function using a first algorithm within the target time slot, wherein the first objective function is used to indicate a minimum content request delay when all users make content requests to the connected base station; The minimum content request delay is determined as the target content request delay, and the target content request delay, the user corresponding to the target content request delay, and the base station accessed by the user are determined as the user access decision of the user.

4. The method according to claim 3, characterized in that Solving a first objective function using a first algorithm within the target time slot includes: Establishing an initial matching state between each base station and the user based on a preset preference list, wherein the preference list is used to indicate the priority of each user to access each base station and the priority preference of each base station to the accessed user; Based on the initial matching state, exchange matching of user access to the base station is performed until a preset matching condition is reached to obtain a solution to the first objective function, wherein the preset matching condition is that the content request delay of the user exchanged after any exchange matching is higher than the content request delay before the exchange and the total delay after the exchange is higher than the total delay before the exchange.

5. The method according to claim 1, characterized in that Determining a target bandwidth allocation decision based on the user access decision includes: Solving a second objective function using a second algorithm in the target time slot, wherein the second objective function is used to indicate an optimal bandwidth allocation decision allocated to the user by the base station accessed by the user under the user access decision; The minimum bandwidth obtained by solving the second objective function using the second algorithm is determined as the target bandwidth allocation decision.

6. The method according to claim 5, characterized in that Solving the second objective function using a second algorithm within the target time slot includes: determining a second-order derivative of a target content request delay based on the user access decision within the target time slot; When the second-order derivative is greater than 0, determining that the target content request delay is a convex function; The second objective function is solved by using a convex optimization algorithm to obtain a minimum bandwidth, and the minimum bandwidth is determined as the target bandwidth allocation decision.

7. The method according to claim 1, characterized in that Determining a cache decision for delay optimization based on the user access decision and the target bandwidth allocation decision includes: A third objective function is solved using a third algorithm within the target time frame to determine a cache decision for delay optimization, wherein the third objective function is determined based on a cache update delay of a previous time frame of the target time frame, the user access decision, and the target bandwidth allocation decision.

8. The method according to claim 7, characterized in that Solving a third objective function using a third algorithm within the target time frame to determine the cache decision for delay optimization includes: Initializing parameters of a critic network and an actor network in a third algorithm, wherein the critic network is used to evaluate the value of a given state and action, and the actor network is used to obtain a cache placement strategy according to the given state; Updating the first content popularity to the second content popularity within a time frame, wherein the content popularity represents the change of the probability of users requesting different contents over time, the first content popularity is used to represent the content popularity at a previous moment, and the second content popularity is used to represent the content popularity at a next moment of the previous moment; Based on the second content popularity, determine an initial caching strategy using the third algorithm, and select an action through the actor network to update the initial caching strategy to obtain a current caching strategy; When the amount of change between the current cache strategy and the initial cache strategy is less than a preset threshold, the current cache strategy is determined as a cache decision for latency optimization.

9. A device for delay optimization, characterized in that: include: A first determination module, configured to determine a user access decision when a user accesses a cellular network, wherein the user access decision is used to determine a target content request delay for making a content request to the accessed base station, wherein the target content request delay is a minimum content request delay within a target time slot, and the target time slot is any time slot in a target time frame; A second determination module is used to determine a target bandwidth allocation decision based on the user access decision, wherein the target bandwidth allocation decision is used to indicate bandwidth allocation of the base station accessed by the user to the user; The third determination module is used to determine a cache decision for delay optimization within the target time frame based on the user access decision and the target bandwidth allocation decision.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the delay optimization method described in any one of claims 1 to 8.

11. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the delay optimization method described in any one of claims 1 to 8 is executed when the program is run.

12. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the delay optimization method described in any one of claims 1 to 8 is implemented.

Images