A data intelligent caching method based on collaborative perception

Abstract

The present invention relates to a data intelligent caching method based on collaborative perception, and belongs to the field of wireless communication technology. The method first updates the global popularity and local popularity on two time scales, long and short, by analyzing historical request information and the hit status of the request. Then, the device request file transmission delay in the industrial Internet scenario is modeled. Finally, a multi-agent reinforcement learning model based on the Actor-Critic framework is used to update the cache status at the micro base station, and bandwidth resources are adaptively allocated to maximize the rate weighted sum objective function. The present invention can significantly reduce the delay in devices in the industrial Internet obtaining key data files and improve application performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of wireless communications and relates to a data intelligent caching method based on collaborative perception. Background Art

[0002] With the growing demand for smart manufacturing, leveraging technologies such as artificial intelligence, edge computing, and cloud-edge collaboration to enhance traditional industrial production has become a hot topic of research in the international industry. The Industrial Internet (IIoT) will become a new type of infrastructure that deeply integrates next-generation information and communications technologies with the industrial economy, providing integrated connectivity for people, objects, and networks. In industrial manufacturing scenarios, devices are extremely diverse, making it difficult to achieve intelligent and comprehensive connectivity across the entire IIoT ecosystem using only traditional cellular and fiber-optic networks. Unmanned intelligent inspections, energy and environmental monitoring, remote equipment control, and on-site intelligent testing all place new demands on the networks they support. As services diversify, data collected by industrial equipment and sensors is uploaded to cloud centers for centralized processing, and the resulting data is fed back into various industrial applications. However, the large number of device nodes accessing popular data generates a significant amount of redundant data within the transmission network, and retrieving processed data files from the cloud each time makes it difficult to reduce latency in device data acquisition. Edge Cache, on the other hand, can cache popular data in small base stations (SBSs) near devices, enabling rapid responses to large numbers of device requests and reducing data file retrieval latency. However, due to the limited coverage of SBSs, some devices in suburban areas or areas with significant interference may have difficulty accessing service. Macro base stations (MBSs) offer a wider coverage area, and working in conjunction with SBSs can provide comprehensive coverage for smart devices across the entire Industrial Internet landscape.

[0003] Due to the low latency requirements of smart industrial scenarios, it's difficult to accurately track the popularity of various types of content based solely on historical data file requests. Traditional caching strategies, such as Least Recently Used (LRU) and Least Frequently Used (LFU), are based on simple statistics and often struggle to achieve effective results in rapidly changing scenarios. On the other hand, machine learning solutions combined with AI, such as deep learning and reinforcement learning, can help uncover changing trends in content popularity within large amounts of data. This provides a more effective solution for improving base station caching efficiency and reducing device latency in accessing content.

[0004] In a large number of existing studies, there are certain studies on using a single SBS to analyze historical data to make caching decisions, analyzing the mobility of devices, and using long-term and short-term caches to track the popularity of regional content. However, they all focus on the mobile content level and have certain similar characteristics in different regions. On the other hand, the popularity trends of content in the areas covered by different SBSs are often different, but because the distribution of similar industrial production is often concentrated together, there may be certain similarities between adjacent SBSs. In addition, the size of the data requests for a region often affects the cache efficiency of a region. Therefore, how to efficiently cache popular files and transmit them to devices remains an open problem.

[0005] Therefore, further research is still needed to consider the request volume of regional content and introduce collaboration between different base stations to make caching decisions to reduce the device acquisition latency cost in industrial Internet scenarios. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a data intelligent caching method based on collaborative perception to address the problem that industrial equipment in industrial Internet scenarios has a long delay in obtaining file data required for applications and traditional caching methods are difficult to ensure efficient utilization of cache space. The method takes into account the changing characteristics of cache content at different time scales, introduces the Actor-Critic machine learning framework, uses the collaborative caching architecture to improve the utilization efficiency of cache space and the diversity of file content, and adaptively associates industrial equipment with base stations to reduce the overall delay in devices in the network obtaining files.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A data intelligent caching method based on collaborative perception first updates the global popularity and local popularity on two time scales, long and short, by analyzing historical request information and request hit status. Then, the file transmission delay of device requests in industrial Internet scenarios is modeled. Finally, a multi-agent reinforcement learning model based on the Actor-Critic framework is used to update the cache status at the micro base station, and bandwidth resources are adaptively allocated to maximize the rate weighted sum objective function.

[0009] The method specifically comprises the following steps:

[0010] S1: System cold start: Initialize and determine relevant file library information, industrial equipment location and channel conditions, cache system weight parameters, long-term cache update time slots, and cache file acquisition costs at different locations;

[0011] S2: Popularity update: Based on the information in step S1, the global popularity is updated in each time slot t according to the long-term update time slot set and the cache hit of the requested file. and local popularity

[0012] S3: File cache decision: Based on the popularity information obtained in step S2, first update the long-term cache matrix at the macro base station according to the global popularity in each long-term cache update time slot T. To ensure that the overall popular files can be served at least at the macro base station; then use the multi-agent reinforcement learning model based on the Actor-Critic framework to optimize the short-term cache matrix at the micro base station. Update to obtain all cached decision models at the current moment;

[0013] S4: Information upload collection: collect the request volume of each area in time slot t And the cache decision information of the previous time slot Uploaded to the macro base station, and after being processed by the macro base station, fed back to the micro base station for caching decision in the next time slot;

[0014] S5: Determine the allocation of transmission resources: Based on the industrial equipment information obtained in step S1 and the cache status obtained in step S3, the transmission rate requirements of different industrial equipment are combined, and the time transmission benefit results are uploaded to the cloud (Critic). Then, they are fed back to each micro base station (Actors) and the benefit feedback (Rewards) is obtained to update the local model. When the relevant model parameters need to be modified, it can be stopped; otherwise, the cache file will continue to be updated.

[0015] Furthermore, step S1 specifically includes the following steps:

[0016] S11: Get the relevant file library And according to the position relationship between the industrial equipment and the micro base station, the transmission equivalent rate per unit bandwidth is initialized, which is specifically expressed as:

[0017]

[0018] in, Indicates the equivalent transmission rate per unit bandwidth; Indicates the standard bandwidth size allocated by the micro base station to the device, P k represents the transmission power at micro base station k, represents the channel gain between micro base station k and device u at time t, and N0 represents the noise power spectral density;

[0019] S12: Set the weight parameters α∈[0,1] and β∈[0,1] in the cache system, which respectively represent the influence of the neighbor file popularity on the local popularity of the micro base station and the influence of the latest file request ratio in the previous time slot on the local popularity; then set the long-term cache update time slot T; in addition, set the delay cost parameter λ for obtaining files at different locations s and λ c , which represent the latency cost of neighboring micro base stations collaborating to obtain files and the latency cost of obtaining files from the cloud, respectively. The specific relationship is expressed as λ c >>λ s .

[0020] Furthermore, step S2 specifically includes the following steps:

[0021] S21: Cold start the entire system. At this time, there are no historical file requests, so the average initial popularity is set according to the number of files in the file library. The expression is:

[0022]

[0023]

[0024] in, They represent the initial popularity of content f at micro base station k and the overall network, respectively, and F represents the total number of contents in the content library;

[0025] S22: Update the file popularity at different scales in each time slot t; when the current time slot t corresponds to n times the long-term cache update time slot T, update the long-term global popularity, which is expressed as:

[0026]

[0027] in, and They represent the real-time popularity of content f at micro base station k and the overall network at time t, respectively, and K represents the number of micro base stations. The corresponding local popularity will be updated in the long-term time slot to ensure that it does not deviate too much from the overall popularity trend. The expression is:

[0028]

[0029] in, represents the number of micro base stations adjacent to micro base station k, γ k represents the set of neighboring micro base stations of micro base station k, α∈[0,1] represents the neighbor popularity influence factor;

[0030] S23: In normal time slots, that is, when the long-term cache update time slot is not an integer multiple of T, the long-term popularity is no longer updated but only the local popularity is updated, and it is only related to the file hit situation; the specific update method is:

[0031]

[0032] in, and They represent the request volume of file f in the previous time slot and the request volume in the entire area, respectively, and β represents the influence factor of popularity update weight.

[0033] Furthermore, step S3 specifically includes the following steps:

[0034] S31: When in the long-term cache update time slot, cached files are first updated at the macro base station using the popularity-related information obtained in step S2. The long-term popularity sequences corresponding to all files are first analyzed. Considering that long-term popularity corresponds to the long-term preference characteristics of the entire network scenario, files are cached in descending order of popularity. Existing content in the cache remains unchanged, and only some less popular files are replaced with more popular files to speed up the replacement process.

[0035] S32: To further reduce the latency for industrial equipment to obtain the required files, we cache more regionally specific content at the corresponding micro base stations during short-term caching timeslots, ensuring that real-time changes can be tracked. First, we model the optimization problem as follows:

[0036]

[0037]

[0038]

[0039] Among them, s f Indicates the size of a single file, S k represents the cache capacity of micro base station k, represents the cache indicator variable of content f at micro base station k at time t, Represents a collection of industrial equipment, represents a set of micro base stations, represents the file set, C represents the cache decision matrix of all micro base stations; The latency of the device actually acquiring the content is related to the associated base station and the data cache location, which can be obtained in S52. That is, the cache decision that maximizes the overall benefit is selected in each time slot;

[0040] S33: All micro base stations together form a multi-agent reinforcement learning model, in which each micro base station acts as an agent, and its state transition process can be modeled as a Markov decision process, which can obtain the local state value s k (t), global information g k (t) and the reward function R(t), which is expressed as follows:

[0041]

[0042]

[0043]

[0044] Among them, s k (t) includes the cache status of the last moment Request volume Local popularity And the learning rate of a single agent δ; global information includes the global popularity at the last moment and cache status The last actual observation value o k (t) is expressed as:

[0045] o k (t) = {s k (t),g k (t)}

[0046] In addition, for the action variable a k (t) only considers the cache strategy, that is,

[0047] S34: At a single micro base station, a DQN machine learning framework is used to train local cache decisions. Specifically, the action value function Q(o,a) is first defined as:

[0048]

[0049] Among them, v t is the discount factor; then for each update, the largest action is selected, i.e.

[0050] Then define the update mechanism of the network parameter ω as the least square mean square error, which is expressed as:

[0051]

[0052] Among them, y(t) represents its actual benefit, which will be obtained after the transmission strategy is formulated in S5;

[0053] S35: After each agent makes a decision, all decisions are uploaded to the cloud for evaluation. This process requires the final network benefit in S53 and uses the least squares mean square error to update the parameters of the critic network.

[0054] Further, step S4 specifically includes the following steps:

[0055] S41: The number of requests for all micro base stations to upload each file in the previous time slot and the short-term cache decision information of the previous time slot to the macro base station;

[0056] S42: The macro base station accumulates the regional request volume for global popularity calculation and calculates the request volume of the neighbors around each micro base station in It is used to transmit the short-term buffering decision to each micro base station in the next time slot.

[0057] Further, step S5 specifically includes the following steps:

[0058] S51: First, the cache status of the requested content is analyzed. If the micro base station has the requested content cached, the service is provided directly. Otherwise, the file is collaboratively transmitted to the associated micro base station through a neighboring base station or macro base station, and the neighboring base station collaboration cost needs to be deducted. If neither of the neighboring base stations has the file, the file is obtained from the cloud, and the cloud content acquisition cost is deducted.

[0059] S52: Based on the theoretical unit bandwidth rate in S11, the associated rate table can be obtained, and the delay of transmitting the unit file is expressed as

[0060]

[0061] in, This means that the content f in the k neighbor base station of the micro base station u If there is no cache, the value is 0, otherwise it is 1. Each industrial device selects the micro base station or macro base station with the minimum file acquisition latency within the access range based on the calculated results and its own latency requirements. Note that macro base stations have a larger coverage area in actual scenarios.

[0062] S53: Each micro base station uploads the unit rate obtained by the current time slot service to the macro base station for centralized processing as a reward and feeds it back to the micro base station group to adjust the model;

[0063] S54: If the model-related parameters need to be modified, the operation is stopped and ended. Otherwise, the cache strategy is updated again after the time slot number t=t+1 from S21.

[0064] The beneficial effects of the present invention are as follows: by dividing the file popularity into two different time slots to update the file popularity, the present invention can track the changes in short-term popular content as much as possible while ensuring the overall popular trend; and the collaborative caching scheme adopted can more efficiently utilize the cache space compared with the traditional caching strategy, while ensuring the cache hit rate and increasing the diversity of content as much as possible, effectively solving the problem that in the industrial Internet scenario, it is difficult to guarantee the delay requirement of popular content after industrial equipment obtains and processes.

[0065] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0067] Figure 1 This is a network architecture diagram of the data intelligent caching method based on collaborative perception of the present invention;

[0068] Figure 2 This is a logical diagram of the Actor-Critic machine learning architecture of the present invention;

[0069] Figure 3 This is a flow chart of the collaborative caching method of the present invention. DETAILED DESCRIPTION

[0070] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0071] See also Figures 1 to 3The present invention provides a data intelligent caching method based on collaborative perception. This method is aimed at the industrial Internet scenario where a large number of industrial devices collect information and process it on the cloud server. Each device obtains content from the cloud center, which will occupy a large amount of link resources and cause a large amount of delivery delay. The conventional LRU and LFU caching solutions cannot efficiently utilize the cache space. Designing an efficient caching strategy will help improve the performance of intelligent industrial applications. First, a transmission rate model of macro base stations and micro base stations is constructed; secondly, based on historical request information, the long-term and short-term file popularity are updated respectively; then, an Actor-Critic machine learning framework is used in combination with multi-agent reinforcement learning to collaboratively place cache files at the micro base station to minimize the delay in the device obtaining the file; finally, based on the delay in the device obtaining the data file, the micro base station or macro base station is adaptively associated.

[0072] The method specifically comprises the following steps:

[0073] Step 1: System cold start: Initialize and determine relevant file library information, industrial equipment location and channel conditions, cache system weight parameters, long-term cache update time slots, and cache file acquisition costs at different locations. Specific steps include:

[0074] Step 1.1: Get the relevant file library And according to the position relationship between the industrial equipment and the micro base station, the transmission equivalent rate per unit bandwidth is initialized, which is specifically expressed as:

[0075]

[0076] in, Indicates the standard bandwidth size allocated by the micro base station to the device.

[0077] Step 1.2: Set the weight parameters α∈[0,1] and β∈[0,1] in the cache system, which respectively represent the influence of the neighbor file popularity on the local popularity of the micro base station and the influence of the latest file request ratio in the previous time slot on the local popularity; then set the long-term cache update time slot T; in addition, set the delay cost parameter λ for obtaining files at different locations s and λ c , which represent the latency cost of neighboring micro base stations collaborating to obtain files and the latency cost of obtaining files from the cloud, respectively. The specific relationship is expressed as λ c >>λ s .

[0078] Step 2: Popularity update: Based on the information in step S1, update the global and local popularity The specific steps include:

[0079] Step 2.1: First, cold start the entire system. At this time, there is no historical file request, so the average initial popularity is set to

[0080] Step 2.2: Then, update the file popularity at different scales in each time slot t. When the current time slot t corresponds to n times the long-term cache time slot T, the long-term global popularity is updated to

[0081]

[0082] Where K represents the number of micro base stations. The corresponding local popularity will be updated in the long-term time slot to ensure that it does not deviate too much from the overall popularity trend. The specific calculation is as follows

[0083]

[0084] in, represents the number of micro base stations adjacent to micro base station k.

[0085] Step 2.3: In normal time slots, that is, when the time slot is not an integer multiple of the long-term cache time slot T, the long-term popularity is no longer updated but only the local popularity is updated, which is only related to the file hit situation. The specific update method is

[0086]

[0087] in, and They represent the request volume of file f at the previous moment and the request volume in the entire area respectively.

[0088] Step 3: File cache decision: Based on the popularity information obtained in step S2, first update the long-term cache matrix at the macro base station according to the global popularity in each long-term cache update time slot T. To ensure that the overall popular files can be served at least at the macro base station. Further use the multi-agent reinforcement learning of the Actor-Critic framework to improve the short-term cache matrix at the micro base station. Determine and obtain all cached decision models at the current moment. This includes the following steps:

[0089] Step 3.1: During the long-term cache time slot, cached files are first updated at the macro base station using the popularity information obtained in step S2. The long-term popularity sequence for all files is analyzed. Considering that long-term popularity corresponds to long-term preferences across the entire network, files are cached in descending order of popularity. Existing cached content remains unchanged, and only some less popular files are replaced with more popular ones to speed up replacement.

[0090] Step 3.2: To further reduce the latency of industrial equipment in obtaining the required files, in the short-term cache time slot, it is necessary to cache the content with more regional characteristics at the corresponding micro base station and ensure that the real-time change characteristics can be tracked. First, the optimization problem is modeled as follows

[0091]

[0092]

[0093]

[0094] Among them, s f Indicates the size of a single file, S k represents the cache capacity of micro base station k, represents the actual latency of the device to obtain the content, which is related to the associated base station and the data cache location, and can be obtained in step 5.2. That is, in each time slot, a cache decision that maximizes the overall benefit is selected.

[0095] Step 3.3: All micro base stations together form a multi-agent reinforcement learning model, in which each micro base station acts as an agent, and its state transition process can be modeled as a Markov decision process, which can obtain the local state value s k (t), global information g k (t) and the reward function, which is expressed as follows

[0096]

[0097]

[0098]

[0099] Among them, s k (t) includes the cache status, request volume, local popularity and single agent learning rate δ at the last moment; while global information includes the global popularity and cache status at the last moment. The actual observation value is expressed as

[0100] o k (t) = {s k (t),gk (t)}

[0101] In addition, for action variables, only the cache strategy is considered, i.e.

[0102] Step 3.4: At a single micro base station, a DQN machine learning framework is used to train the local cache decision. Specifically, the action value function is first defined as

[0103]

[0104] Among them, v t is the discount factor. Then for each update, the largest action is selected The update mechanism of the network parameter ω is further defined as the least square mean square error form as follows

[0105]

[0106] Among them, y(t) represents its actual benefit, which will be obtained after the transmission strategy is formulated in S5.

[0107] Step 3.5: After each agent makes a decision, upload all decisions to the cloud for evaluation. This process requires the final network benefit in S53 and uses the least squares mean square error to update the parameters of the critic network.

[0108] Step 4: Information upload and collection: Collect the number of requests for each region at time t And the cache decision information of the last moment The data is uploaded to the macro base station, processed by the macro base station, and fed back to the micro base station for caching decision in the next time slot. The specific steps include:

[0109] Step 4.1: The number of requests for all micro base stations to upload each file in the previous time slot And the short-term cache decision information of the previous moment to the macro base station;

[0110] Step 4.2: The macro base station accumulates the regional requests for global popularity calculation and calculates the requests of the neighbors around each micro base station. in It is used to be transmitted to each micro base station in the next time slot for its own short-term caching decision.

[0111] Step 5: Determine transmission resource allocation: Based on the industrial equipment information obtained in step S1 and the cache status obtained in step S3, the transmission rate requirements of different industrial equipment are combined and the time transmission benefit results are uploaded to the cloud (Critic). Then, the results are fed back to each micro base station (Actors) and the benefit feedback (Rewards) is obtained to update the local model. When the relevant model parameters need to be modified, the system can stop, otherwise it will continue to update the cache file. Specifically, it includes the following steps:

[0112] Step 5.1: First, analyze the cache status of the requested content. If the micro base station has the content cached, it will be served directly. Otherwise, the file will be collaboratively transferred to the associated micro base station through a neighboring base station or macro base station, and the neighboring base station collaboration cost will be deducted. If neither of them has the file, it will be obtained from the cloud, and the cloud content acquisition cost will be deducted.

[0113] Step 5.2: Further, the associated rate table can be obtained based on the theoretical unit bandwidth rate in S11, and the delay of transmitting the unit file is further expressed as

[0114]

[0115] in, This means that the content f in the k neighbor base station of the micro base station u If there is no cache, the value is 0, otherwise it is 1. Each industrial device selects a micro or macro base station with the minimum file acquisition latency within its access range based on the calculated results and its own latency requirements. Note that macro base stations have a larger coverage area in actual scenarios.

[0116] Step 5.3: Each micro base station uploads the unit rate obtained by the current time slot service to the macro base station for centralized processing as a reward and feeds it back to the micro base station group to adjust the model;

[0117] Step 5.4: If the model-related parameters need to be modified, stop the operation and end. Otherwise, continue to update the cache strategy from S21 after the time slot number t=t+1.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A data intelligent caching method based on collaborative perception, characterized in that: First, by analyzing historical request information and request hit status, the global and local popularity are updated on both long and short time scales. Then, the device request file transmission delay in the industrial Internet scenario is modeled. Finally, a multi-agent reinforcement learning model based on the actor-critic framework is used to update the cache status at the micro base station, and bandwidth resources are adaptively allocated to maximize the rate weighted sum objective function. The method specifically includes the following steps: S1: System cold start: Initialize and determine relevant file library information, industrial equipment location and channel conditions, cache system weight parameters, long-term cache update time slots, and cache file acquisition costs at different locations; S2: Popularity update: Based on the information in step S1, at each time t, the global popularity and local popularity are updated according to the set long-term cache update time slot and the cache hit status of the requested file; S3: File caching decision: Based on the popularity-related information obtained in step S2, the long-term cache matrix is ​​first updated at the macro base station based on the global popularity at each long-term cache update time slot T to ensure that all popular files can be served at least at the macro base station. Then, a multi-agent reinforcement learning model based on the actor-critic framework is used to update the short-term cache matrix at the micro base station to obtain the decision model for all caches at the current moment. S4: Information upload collection: collect the number of requests in each area at time t And the cache decision information of the last moment Uploaded to the macro base station, and after being processed by the macro base station, fed back to the micro base station for the next cache decision; S5: Determine transmission resource allocation: Based on the industrial equipment information obtained in step S1 and the cache status obtained in step S3, the transmission rate requirements of different industrial equipment are combined, and the time transmission benefit results are uploaded to the cloud. Then, the results are fed back to each micro base station and the benefit feedback is used to update the local model. If the relevant model parameters need to be modified, the system stops; otherwise, the cache file will continue to be updated. Step S1 specifically includes the following steps: S11: Get the relevant file library And according to the position relationship between the industrial equipment and the micro base station, the transmission equivalent rate per unit bandwidth is initialized, which is specifically expressed as: in, Indicates the equivalent transmission rate per unit bandwidth; Indicates the standard bandwidth size allocated by the micro base station to the device, P k represents the transmission power at micro base station k, represents the channel gain between micro base station k and device u at time t, and N0 represents the noise power spectral density; S12: Set the weight parameters α∈[0,1] and β∈[0,1] in the cache system, which respectively represent the influence of the neighbor file popularity on the local popularity of the micro base station and the influence of the latest file request ratio at the last moment on the local popularity; then set the update time slot T of the long-term cache; and set the delay cost parameter λ for obtaining files at different locations s and λ c , which represent the latency cost of neighboring micro base stations collaborating to obtain files and the latency cost of obtaining files from the cloud, respectively. The specific relationship is expressed as λ c >>λ s ; Step S2 specifically includes the following steps: S21: Cold start the entire system and set the average initial popularity based on the number of files in the file library. The expression is: in, They represent the initial popularity of content f at micro base station k and the overall network, respectively, and F represents the total number of contents in the content library; S22: Update the file popularity at different scales at each time t; when the current time t corresponds to n times the long-term cache update time slot T, update the long-term global popularity, which is expressed as: in, and They represent the real-time popularity of the content f at the micro base station k and the overall network at time t, respectively, and K represents the number of micro base stations. The corresponding local popularity will be updated in the long-term time slot, and the expression is: in, represents the number of micro base stations adjacent to micro base station k, Y k represents the set of neighboring micro base stations of micro base station k, α∈[0,1] represents the neighbor popularity influence factor; S23: In normal time slots, that is, when the time slot is not an integer multiple of the long-term cache update time slot T, the long-term popularity is no longer updated but only the local popularity is updated, and it is only related to the file hit situation; the specific update method is: in, and They represent the number of requests for file f at the previous moment and the number of requests in the entire region, and β represents the influence factor of popularity update weight; Step S3 specifically includes the following steps: S31: When it is time to update the long-term cache, the cached files are first updated at the macro base station using the popularity-related information obtained in step S2. The long-term popularity sequences corresponding to all files are first analyzed, and files are cached in descending order of popularity. The content already stored in the cache remains unchanged, and only some files with lower popularity are replaced with files with higher popularity to speed up the replacement process. S32: During short-term caching, content with more regional characteristics is cached at the corresponding micro base station, ensuring that real-time changes can be tracked. First, the optimization problem is modeled as follows: Among them, s f Indicates the size of a single file, S k represents the cache capacity of micro base station k, represents the cache indicator variable of content f at micro base station k at time t, Represents a collection of industrial equipment, represents a set of micro base stations, represents the file set, C represents the cache decision matrix of all micro base stations; Indicates the latency of the device actually obtaining the content; that is, choosing the caching decision that maximizes the overall benefit at each moment; S33: All micro base stations together form a multi-agent reinforcement learning model, in which each micro base station acts as an agent, and its state transition process is modeled as a Markov decision process, which can obtain the local state value s k (t), global information g k (t) and the reward function R(t), which is expressed as follows: Among them, s k (t) includes the cache status of the last moment Request volume Local popularity And the learning rate of a single agent δ; global information includes the global popularity at the last moment and cache status The last actual observation value o k (t) is expressed as: o k (t)={s k (t),g k (t)} In addition, for the action variable a k (t) only considers the cache strategy, that is, S34: At a single micro base station, a DQN machine learning framework is used to train local cache decisions. Specifically, the action value function Q(o,a) is first defined as: Among them, v t is the discount factor; then for each update, the largest action is selected, i.e. Then define the update mechanism of the network parameter ω as the least square mean square error, which is expressed as: Among them, y(t) represents the actual benefit; S35: After each agent makes a decision, all decisions are uploaded to the cloud for evaluation, and the parameters of the critic network are updated using the least square error. Step S4 specifically includes the following steps: S41: All micro base stations upload the request volume of each area at time t And the short-term cache decision information of the previous moment to the macro base station; S42: The macro base station accumulates the regional request volume for global popularity calculation and calculates the request volume of the neighbors around each micro base station in Used to transmit the short-term cache decision to each micro base station at the next moment; Step S5 specifically includes the following steps: S51: First, the cache status of the requested content is analyzed. If the micro base station has the requested content cached, the service is provided directly. Otherwise, the file is collaboratively transmitted to the associated micro base station through a neighboring base station or macro base station, and the neighboring base station collaboration cost needs to be deducted. If neither of the neighboring base stations has the file, the file is obtained from the cloud, and the cloud content acquisition cost is deducted. S52: Based on the associated rate table of the theoretical unit bandwidth rate in S11, the delay of transmitting the unit file is expressed as in, This means that the content f in the k neighbor base station of the micro base station u If there is no cache, the value is 0, otherwise it is 1; each industrial device selects the micro base station or macro base station with the minimum file acquisition delay within the access range according to the calculated results and the latency requirements of its own device to associate; S53: Each micro base station uploads the unit rate obtained by the current service to the macro base station for centralized processing as a reward and feeds it back to the micro base station group to adjust the model; S54: If the model-related parameters need to be modified, the operation is stopped and ended. Otherwise, the cache strategy is updated again from S21 after time t=t+1.

Images