Service timeliness optimization method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a service timeliness optimization method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring service feedback information in a historical time period; the service feedback information is an index when the service object feeds back a service request to the target object under the current caching strategy and the current routing strategy; solving in combination with the service feedback information and a preset timeliness optimization model to obtain a target caching strategy and a target routing strategy corresponding to a future time period; the target caching strategy and the target routing strategy are used for optimizing service timeliness. By adopting the method, the service response efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of Internet of Things, and in particular to a method, apparatus, computer equipment, computer-readable storage medium and computer program product for optimizing service timeliness. Background Art

[0002] With the rapid development of mobile communication technology, Internet of Vehicle (IoV), as an emerging application of Internet of Things (IoT), connects vehicles to the Internet, provides vehicles with rich and real-time information services, and promotes the development of intelligent transportation. The timeliness of information in IoV is crucial, especially for services such as navigation and real-time traffic conditions. The freshness of information directly affects the service quality.

[0003] However, due to the increasing amount of information and computing requirements in IoV (such as high-definition video processing), the vehicle's own computing and storage resources are limited, which may lead to higher processing delays and energy consumption. To address this problem, the cloud-edge collaborative architecture provides a more effective solution for IoV. By combining the powerful computing and storage capabilities of cloud computing with the advantages of edge computing being close to users and responding quickly, cloud-edge collaboration can effectively reduce the computing burden on the vehicle side and improve the information processing efficiency of the IoV system.

[0004] However, centrally storing information in the cloud or on edge servers may result in untimely information feedback, especially when the data volume is large or the requests are frequent. Remote transmission will cause additional delays and affect the timeliness of the service. Summary of the invention

[0005] Based on this, it is necessary to provide a service timeliness optimization method, device, computer equipment, computer-readable storage medium and computer program product that can improve service timeliness in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for optimizing service timeliness, the method comprising:

[0007] Obtaining service feedback information within a historical time period; the service feedback information is an indicator of when the service object feeds back a service request to the target object under the current cache strategy and the current routing strategy;

[0008] The service feedback information and the preset timeliness optimization model are combined to solve the corresponding target cache strategy and target routing strategy for the future time period; the target cache strategy and the target routing strategy are used to optimize the service timeliness.

[0009] In one embodiment, the process of constructing the timeliness optimization model includes:

[0010] According to the service object, determining a path for the service object to feedback a service request to the target object, and constructing timeliness corresponding to the path;

[0011] Based on the timeliness corresponding to the path and the response delay of the service object, the timeliness optimization target is obtained;

[0012] The timeliness optimization model is obtained according to the preset optimization constraints and the timeliness optimization target.

[0013] In one embodiment, the service object includes an edge server and a cloud server; the response delay of the service object includes an edge response delay and a cloud response delay; the path includes an edge service path and a cloud service path;

[0014] The edge service path is a transmission path from the edge server to the target object;

[0015] The cloud service path is a transmission path from the cloud server to the edge server and from the edge server to the target object.

[0016] In one embodiment, the constructing the timeliness corresponding to the path includes:

[0017] Constructing the timeliness corresponding to the edge service path through the data transmission mode corresponding to the edge server to the target object;

[0018] The timeliness of the cloud service path is established by using the link bandwidth corresponding to the cloud server to the edge server and the data transmission method corresponding to the edge server to the target object.

[0019] In one embodiment, obtaining the timeliness optimization target based on the timeliness corresponding to the path and the response delay of the service object includes:

[0020] Obtaining an edge timeliness optimization target according to the timeliness corresponding to the edge service path and the edge response delay;

[0021] Obtaining a cloud timeliness optimization target according to the timeliness corresponding to the cloud service path and the cloud response delay;

[0022] The timeliness optimization target is obtained according to the edge timeliness optimization target and the cloud timeliness optimization target.

[0023] In one embodiment, the service feedback information and the preset timeliness optimization model are combined to solve the corresponding target cache strategy and target routing strategy for the future time period, including:

[0024] Predict the initial cache strategy and initial routing strategy based on service feedback information;

[0025] Simulating the initial cache strategy and the initial routing strategy to obtain prediction feedback information;

[0026] According to the prediction feedback information, the initial cache strategy and the initial routing strategy are adjusted until the prediction feedback information corresponding to the initial cache strategy and the initial routing strategy meets the timeliness optimization model, thereby obtaining the target cache strategy and the target routing strategy.

[0027] In a second aspect, the present application provides a device for optimizing service timeliness, the device comprising:

[0028] An acquisition module is used to acquire service feedback information within a historical time period; the service feedback information is an indicator of when a service object feeds back a service request to a target object under a current cache strategy and a current routing strategy;

[0029] The solution module is used to combine the service feedback information and a preset timeliness optimization model to solve and obtain the corresponding target cache strategy and target routing strategy for the future time period; the target cache strategy and the target routing strategy are used to optimize service timeliness.

[0030] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0032] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0033] The above-mentioned service timeliness optimization method, device, computer equipment, computer-readable storage medium and computer program product can obtain the service performance under the current cache and current routing strategy by obtaining service feedback information of the historical time period. After combining the feedback information and solving the timeliness optimization model, a target cache strategy and routing strategy suitable for the future time period are generated, thereby optimizing the timeliness of the service and ensuring that the target object can quickly obtain the required data. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 An application environment diagram of a method for optimizing service timeliness in an embodiment;

[0036] Figure 2 A schematic diagram of a process flow of a method for optimizing service timeliness in one embodiment;

[0037] Figure 3 A schematic diagram of an IoV scenario in an embodiment;

[0038] Figure 4 A structural block diagram of a device for optimizing service timeliness in one embodiment;

[0039] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] The service timeliness optimization method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the target object 102 communicates with the edge server 104 through the network, and the edge server 104 communicates with the cloud server 106. The data storage system can store the data that the server 104 or the cloud server 106 needs to process. The data storage system can be integrated on the edge server 104 or the cloud server 106. The target object 102 sends an access request to the edge server 104; if the transmission content corresponding to the access request is cached on the edge server 104, the edge server 104 sends the transmission content to the target object 102; if the transmission content is cached on the cloud server 106, the cloud server 106 sends the transmission content to the edge server 104, and then the edge server 104 sends it to the target object 102. Since the cache location and routing of the transmission content will affect the timeliness of the service to the target object 102, a service timeliness optimization method is proposed in this application.

[0042] In an exemplary embodiment, Figure 2As shown, a service timeliness optimization method is provided, which is applied to Figure 1 The cloud server 106 in the example is used as an example to illustrate, including the following steps 202 to 204. Among them:

[0043] Step 202, obtaining service feedback information within a historical time period; the service feedback information is an indicator when the service object feeds back a service request to the target object under the current cache strategy and the current routing strategy.

[0044] Among them, the service object refers to the system component that provides content or services. For example, in the Internet of Vehicles scenario, the service object can be an edge server or a remote server, which is responsible for responding to user requests and providing the required information or data.

[0045] The target object refers to the entity receiving the service, that is, the user end device that sends the request and receives the data. In the IoV scenario, the target object is usually a vehicle or a user device on the vehicle, which receives information from the service object. Figure 3 As shown, Figure 3 FIG. 1 is a schematic diagram of an IoV scenario in an embodiment. Figure 3 The edge server is the road test unit (RSU), and the target object is the vehicle. The cloud server represents the core computing and storage node of the system. The cloud server communicates with multiple RSUs through wireless connections to provide computing and data support. RSUs are distributed in different areas, and each RSU is responsible for vehicle communication and data transmission in an area. The vehicle is located in the coverage area of ​​the RSU and exchanges data with the cloud server through the RSU. The communication between the vehicle and the RSU shows that the vehicle can receive services from the cloud through the RSU.

[0046] Optionally, in a cloud computing scenario, the service object can be a cloud server or a microservice node, which is responsible for processing requests and providing computing resources; in a content distribution network scenario, the service object can be an edge node or a cache server, which is responsible for providing cached content, and the target object can be an end user or a client browser, which requests and receives content, such as web pages and videos.

[0047] Among them, the current caching strategy determines the storage location of the content, and the current routing strategy determines the path of content transmission. Exemplarily, the caching strategy can determine whether to store the content on the edge server or on the cloud server. If the transmitted content is stored on the edge server, then when the target object sends a service request to the edge server, the transmitted content is sent from the edge server to the target object. If the transmitted content is stored on the cloud server, then the edge server will send a request to the cloud server, and the transmitted content will be sent from the cloud to the original edge server, and then from the edge server to the target object. Correspondingly, the purpose of the routing strategy is to select the optimal transmission path according to the network status to ensure that the transmitted content can reach the target object in a timely manner.

[0048] First, obtain the service feedback information within the historical time period. The service feedback information includes various performance indicators accumulated during the operation within the historical time period, that is, the performance of the service object under the current caching strategy and the current routing strategy.

[0049] Exemplarily, the service feedback information includes indicators when the service object feeds back the service request to the target object under the current cache strategy and the current routing strategy, such as response delay, information hit rate, transmission delay, etc. These indicators can reflect the impact of different strategy combinations on system timeliness in actual applications and are important inputs to the optimization model.

[0050] Step 204, combining the service feedback information and the preset timeliness optimization model to solve, obtain the corresponding target cache strategy and target routing strategy for the future time period; the target cache strategy and target routing strategy are used to optimize service timeliness.

[0051] Taking the IoV scenario as an example, the network load and target object requirements in IoV and edge computing are constantly changing, and the target object's request mode, data access frequency, network bandwidth, edge server load, etc. will change with the time. Therefore, predicting the cache strategy and routing strategy in the future time period in step 204 can adapt to the upcoming process and load changes.

[0052] Next, the service feedback information and the pre-set service timeliness optimization model are used. The service timeliness optimization model is a pre-defined optimization goal that aims to improve the timeliness of information transmission by analyzing and optimizing cache strategies and routing strategies. The service timeliness optimization model includes various performance constraints, objective functions, and input parameters. The goal is to minimize the average delay or age of information, that is, the timeliness of the service.

[0053] In this embodiment, the service feedback information is used as the input of the model, so that the model can be calculated and optimized based on actual data, thereby better reflecting the operating performance of the system in reality. After the service feedback information is input, the optimization model is solved according to the service feedback information. The solution process is the core of the model. By analyzing the feedback information in the input data, the timeliness optimization model will try to find a combination of cache strategies and routing strategies to maximize the timeliness of the service in the future time period. Finally, the target cache strategy and target routing strategy applicable to the future time period are obtained. The target cache strategy and target routing strategy will be directly used to guide the cache strategy and routing decisions in the future time period. Among them, the target cache strategy defines which content should be cached in the edge server so that in the future time period, it can quickly respond to user requests and reduce the delay of information transmission; the target routing strategy specifies the optimal data transmission path under different network load and delay conditions, thereby ensuring that the information can reach the user end in time.

[0054] Optionally, the solution process may involve optimization algorithms or reinforcement learning algorithms (such as Deep Q Network or DDQN), which dynamically adjust caching and routing strategies based on historical data and current input information to adapt to the actual needs of the system.

[0055] In the above embodiment, by obtaining service feedback information of historical time periods, the service performance under the current cache and current routing strategy can be obtained. After combining the feedback information and solving the timeliness optimization model, a target cache strategy and routing strategy suitable for future time periods are generated, thereby optimizing the timeliness of the service and ensuring that the target object can quickly obtain the required data.

[0056] In one embodiment, the construction process of the above-mentioned timeliness optimization model includes: determining the path when the service object feeds back the service request to the target object according to the service object, and constructing the timeliness corresponding to the path; obtaining the timeliness optimization target based on the timeliness corresponding to the path and the response delay of the service object; obtaining the timeliness optimization model according to the preset optimization constraints and the timeliness optimization target.

[0057] Optionally, before building the timeliness optimization model, a model will be built based on real application scenarios.

[0058] Optionally, when the target object attempts to obtain the transmission content, it will be generated and processed by the directly connected edge server and then transmitted to the target object. The process of the request being responded to by the edge server obeys the queuing theory model, so it is modeled as M / M / k b In the queuing system, each edge server has the same edge task arrival rate, number of edge servers, and edge service rate, denoted as λ b , k b and μ b.

[0059] Among them, the edge task arrival rate indicates the number of requests that arrive at the edge server per unit time. This indicator reflects the task load of the edge server. The higher the arrival rate, the more frequent the vehicle requests are, and the greater the resource demand for the edge server. Too high a task arrival rate may cause server overload, increase queuing delay, and thus affect service timeliness.

[0060] The number of edge servers refers to the total number of edge servers deployed in the system. More edge servers can share the task load, reducing the workload of each server, thereby reducing response latency and improving service timeliness.

[0061] The edge service rate indicates the number of tasks that each edge server can process per unit time. The higher the service rate, the faster the server can complete the processing of each task, reducing queuing and waiting time. The edge service rate is determined by the computing power of the edge server and is an important indicator for measuring the server's processing performance.

[0062] When a service request is not met on the edge server, the cloud server is needed to receive the service. Compared with the edge server, the cloud server has greater computing and storage capabilities and stores all the content in the network, so the service request will be met in the cloud. Similarly, the cloud is modeled as M / M / k c In the queuing system, each cloud server has the same cloud task arrival rate, number of cloud servers, and cloud service rate, denoted as λ c , k c and μ c .

[0063] The cloud task arrival rate refers to the number of requests that arrive at the cloud server per unit time. The cloud task arrival rate reflects the system's demand for cloud resources. Unlike the edge task arrival rate, the cloud task arrival rate may be higher because the cloud is usually used to process tasks with large computational load and high storage requirements. When the cloud task arrival rate is too high, it may increase the queuing time of the cloud server, thereby affecting the service response delay.

[0064] The number of cloud servers refers to the total number of cloud servers deployed in the system. The number of cloud servers affects the computing and storage capacity of the system. More cloud servers can share computing tasks, reduce the pressure on a single server, shorten queuing delays, and improve service timeliness. The number of cloud servers is also a key indicator of a cloud computing center, which directly affects the computing processing capabilities of the system.

[0065] The cloud service rate refers to the number of tasks that a cloud server can handle per unit time. Since the computing and storage capabilities of cloud servers are usually higher than those of edge servers, their service rates are relatively high. A high service rate means that cloud servers can complete tasks more efficiently, reducing queues and waiting time, and are particularly suitable for processing tasks with high computing loads. The cloud service rate has a direct impact on the processing performance and response speed of the entire system.

[0066] In addition, the total signal volume of edge services is also set during the modeling process. The total signal volume of edge services refers to the total amount of signals transmitted from the edge server to the vehicle during the edge service process. This indicator can reflect the communication strength and quality between the edge server and the vehicle. The larger the total signal volume, the wider the signal coverage and the better the signal quality, which helps to improve the transmission speed and data integrity and reduce the transmission delay.

[0067] It should be noted that any parameters or indicators used in the modeling process are set according to the real environment.

[0068] After that, the corresponding timeliness optimization model is constructed. Timeliness is essentially composed of the service object's response delay to the target object's service request and the communication delay of the transmission content on the path from the service object to the target object.

[0069] Furthermore, since the service object includes an edge server and a cloud server, the response delay of the service object includes an edge response delay and a cloud response delay. Among them, the edge response delay represents the response delay of the edge server in responding to the service request sent by the service object, which may include a queuing delay and a service delay; the cloud service response delay represents the response delay of the cloud server in responding to the service request sent by the service object. Similarly, the cloud service response delay may include a queuing delay and a service delay. Then, the edge service path represents the transmission path from the edge server to the target object, and the cloud service path is the transmission path from the cloud server to the edge server and from the edge server to the target object.

[0070] Therefore, we can first determine the path when the service object feeds back the service request to the target object, and build the timeliness corresponding to the path; then, based on the timeliness corresponding to the path and the response delay of the service object, that is, the edge response delay and the cloud response delay, we can get the timeliness optimization target of the transmission content from the service object to the target object receiving the transmission content, that is, the edge timeliness optimization target and the cloud timeliness optimization target. Finally, according to the preset optimization constraints and timeliness optimization targets, we can get the timeliness optimization model.

[0071] Optionally, an edge timeliness optimization target is obtained based on the timeliness corresponding to the edge service path and the edge response delay; a cloud timeliness optimization target is obtained based on the timeliness corresponding to the cloud service path and the cloud response delay.

[0072] The calculation process of edge response delay and cloud response delay is as follows:

[0073] Combined with the above established model, the utilization rate of edge server b can be expressed as:

[0074]

[0075] According to the equation as well as π n,b It represents the probability that there are n access requests in the queuing system of edge server b, expressed as:

[0076]

[0077] According to the Erlang waiting formula, the average queue length of edge server b can be expressed as:

[0078]

[0079] Therefore, the queuing delay of the request at edge server b can be expressed as:

[0080]

[0081] The edge response delay can be expressed as formula (5):

[0082]

[0083] Similarly, consistent with the edge server, the cloud response delay is shown in formula (6):

[0084]

[0085] Furthermore, in one embodiment, the above-mentioned construction of timeliness corresponding to the path includes: constructing timeliness corresponding to the edge service path through the data transmission method corresponding to the edge server to the target object; constructing timeliness corresponding to the cloud service path through the link bandwidth corresponding to the cloud server to the edge server and the data transmission method corresponding to the edge server to the target object.

[0086] From the above content, it can be seen that if the service object is an edge server, the delay corresponding to the edge service path is the communication delay from the edge server to the target object; if the service object is a cloud server, the communication delay corresponding to the cloud service path is the communication delay from the cloud server to the edge server and from the edge server to the target object. It should be noted that when the service object is a cloud server, the communication delay from the edge server to the target object in the communication delay on its path is consistent with the communication delay when the service object is an edge server.

[0087] Furthermore, in this embodiment, the timeliness of the edge service path is established through the data transmission method from the edge server to the target object.

[0088] Optionally, the data transmission mode can be NOMA (non-orthogonal multiple access), which can provide services to multiple users or devices on the same time and frequency resources, thereby improving the utilization efficiency of the spectrum and reducing transmission delays. This method is particularly suitable for wireless communication environments, where multiple devices need to transmit data at the same time, and can achieve coordination and scheduling between multiple devices through power control and other means.

[0089] In the NOMA downlink scenario, the edge server superimposes the signals of all its directly connected users, and the total edge service signal volume x corresponding to edge service b is b It can be expressed as:

[0090]

[0091] Among them, U b For all vehicles under the edge server, P b is the total transmission power of edge server b, β bu is the ratio of the total power of the edge server allocated to the vehicle user u connected to the edge server b, S bu is the signal of user u under edge server b, h bu is the channel coefficient of user u under edge server b. Therefore, the signal-to-noise ratio of the user under edge server b can be expressed as:

[0092]

[0093] Where σ is the Gaussian white noise variance on the vehicle terminal side.

[0094] According to the Shannon formula, the NOMA next-generation transmission rate of edge server b transmitting content to user u can be expressed as:

[0095] C b,bu =B b,bu log2(1+γ bu ) Formula (8)

[0096] Among them, B b,bu is the transmission bandwidth between edge server b and the user. Therefore, after using NOMA technology, the delay when edge server b transmits content k to user u, that is, the timeliness corresponding to the edge service path is:

[0097]

[0098] It can be seen that the edge timeliness optimization goal is:

[0099]

[0100] Formula (10) means that the edge timeliness optimization target is the time it takes for the transmission content to reach the target formation minus the time it takes for the transmission content to be generated at the edge server u k , which is the edge response delay plus the corresponding timeliness of the edge service path.

[0101] Among them, the timeliness of the cloud service path is constructed by the link bandwidth corresponding to the cloud server to the edge server and the data transmission method corresponding to the edge server to the target object. According to the above content, the timeliness corresponding to the edge service path has been obtained, so the timeliness corresponding to the cloud server to the edge server will be constructed next to obtain the timeliness corresponding to the cloud service path.

[0102] Among them, the link bandwidth from the remote server to the edge server is l c,b , then the corresponding delay from the cloud server to the edge server is Where S k is the transmission content. It can be seen that the timeliness of the cloud service path is

[0103] Therefore, the cloud timeliness optimization goal is:

[0104]

[0105] Finally, the timeliness optimization target is obtained based on the edge timeliness optimization target and the cloud timeliness optimization target.

[0106] use Indicates whether the edge server stores the transmitted content k, is a Boolean variable. Correspondingly, the probability that the cloud server stores content k is

[0107] In summary, the timeliness optimization objective is as shown in formula (12):

[0108]

[0109] Finally, according to the preset optimization constraints and timeliness optimization goals, the timeliness optimization model can be obtained.

[0110] The preset optimization constraints may include cache capacity limitations, network bandwidth limitations, server processing capabilities and other constraints to ensure that the system operates within the limitations of physical resources.

[0111] Exemplarily, the preset optimization constraints are as follows:

[0112]

[0113] Among them, C b Indicates the maximum cache capacity of the edge server. This is because the storage space of the edge server is limited. Exceeding the capacity may cause cache overflow or data loss. By limiting the cache usage, you can ensure the stable operation of the edge server and make more efficient use of the available cache space.

[0114] The second constraint indicates that the cache decision can only be 0 or 1, and the third constraint limits the remote server usage rate to no more than 1. This binary setting simplifies the decision-making process and ensures that the cache status of each content is clear, thereby reducing system complexity and computational overhead, and making it easier to solve the optimal cache configuration through the optimization model.

[0115] The third constraint is to limit the utilization rate of the remote server to no more than 1. This constraint is to prevent server overload, because overload will lead to longer response time, reduced information timeliness, and even possible system crash. Keeping the server utilization rate below 1 helps to ensure the service quality of the system, improve the response efficiency of user requests, and ensure the continuity and stability of the service.

[0116] In the above embodiments, the corresponding paths are constructed according to the service objects, and the delays on the paths are calculated, and the edge timeliness optimization targets and the cloud timeliness optimization targets are obtained respectively. Finally, the timeliness optimization model is accurately constructed according to the constraints.

[0117] In one embodiment, the service feedback information and the pre-set timeliness optimization model are combined to solve and obtain the corresponding target cache strategy and target routing strategy for the future time period, including: predicting the initial cache strategy and the initial routing strategy according to the service feedback information; simulating the initial cache strategy and the initial routing strategy to obtain the predicted feedback information; adjusting the initial cache strategy and the initial routing strategy according to the predicted feedback information until the predicted feedback information corresponding to the initial cache strategy and the initial routing strategy meets the timeliness optimization model, and obtaining the target cache strategy and the target routing strategy.

[0118] First, the initial cache strategy and initial routing strategy are predicted based on the service feedback information. This step is to use the existing data to infer the initial decision under the current state. After the initial strategy is generated, it is simulated to predict the possible effect of the strategy in the future. During the simulation, the prediction feedback information is generated, including the expected cache hit rate, transmission delay, etc. These prediction feedback information reflects the performance of the initial cache strategy and initial routing strategy in the future, providing a reference for subsequent optimization. The prediction feedback information is predicted to determine whether the current initial cache strategy and initial routing strategy meet the timeliness optimization goal. If the requirements are not met, the initial strategy is adjusted. This step of adjustment is to optimize the allocation of cache resources and routing paths to improve the timeliness of the system's services. Through multiple cycles of "simulation-feedback-adjustment", the strategy combination is gradually optimized until the prediction feedback information meets the requirements of the timeliness optimization model. When the prediction results of the model meet the timeliness optimization model, the final cache strategy and routing strategy are the target cache strategy and target routing strategy, which are used for actual application in the future time period to ensure that the system provides timely and efficient services.

[0119] Furthermore, the above steps can be implemented through a pre-trained prediction model, wherein the training process of the prediction model includes: based on historical service feedback information and the current system status, continuously iteratively optimizing the cache strategy and routing strategy, and gradually learning and generating the optimal strategy combination that adapts to different network conditions. Through multiple simulation feedback and strategy adjustments, the model is continuously optimized during training until it has the ability to accurately predict future strategies. The initial model can be based on a deep neural network model of reinforcement learning (such as DDQN), which adopts random or heuristic action selection in the early stage and is updated and optimized through preliminary feedback data.

[0120] Exemplarily, in one embodiment, the timeliness optimization model is optimized and solved based on the DDQN strategy, and RSU cache and content distribution routing decisions can be made according to the historical request information in the IoV and the currently available network resources. The reward function is designed based on the system delay to gradually optimize the parameters of the neural network to improve the decision accuracy, thereby effectively improving the freshness of the system information.

[0121] Furthermore, the design of the three elements of action, state and reward in deep reinforcement learning includes designing state, action and reward functions.

[0122] State: In this embodiment, the state at the current time t is defined as S t , X b,t represents the storage state of RSU at time t,

[0123] Action: The actions of this model include RSU caching decisions and content distribution routing decisions. K∈F,X b,t+1 represents the cache decision of RSU at the next moment, n b,t+1 It indicates the routing decision of RSU for content distribution at the next moment.

[0124] Reward: In this embodiment, the reward value is a function inversely proportional to timeliness, aiming to minimize the information age of the system through training. The reward value obtained after executing the action can be expressed as T e is the number of iterative training cycles of the DDQN model, and γ is the reward discount factor of the DDQN model, which can indicate the influence of the rewards of the historical training cycles on the reward value of the current cycle.

[0125] In this embodiment, there are two neural network architectures, namely the current Q network Q and the target Q network Q', which have the same structure. t , the current Q network output Q value Q(S t ,A t ; ω). Based on the obtained Q value, the cloud-edge collaborative IoV selects an action according to the greedy strategy. Specifically, the system randomly selects an action with probability ε∈(0, 1) or selects an action with probability 1-ε according to the expression Select the action with the largest Q value. Execute the selected action A t After that, you can get the reward value R t and the next state S t+1 If the current request can be satisfied by the cache content in the next hop node, the training process is terminated. t ,A t ,R t ,S t+1 ,is_end) is stored in the experience revisit unit to update the current Q network Q. Next, let S t =S t+1 , randomly extract some five-tuples in small batches from the experience replay unit and calculate the Q value y of the target Q network Q' t , its expression can be written as:

[0126]

[0127] Subsequently, the parameters ω of the current Q network can be updated using the mean square error loss function and the gradient back propagation of the neural network. The loss function formula is L(ω) = E[(y t -Q(S t ,A t ;ω)) 2], and after a certain number of steps, the target Q network parameter ω' is updated to make ω'=ω and ω' is updated only once after a certain number of steps. Eventually, through continuous iterative training, the parameters of the neural network will become more and more accurate, and ideal caching and routing decisions can be made to reduce the information age of the entire IoV and improve the freshness of information.

[0128] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0129] Based on the same inventive concept, the embodiment of the present application also provides a service timeliness optimization device for implementing the service timeliness optimization method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more service timeliness optimization device embodiments provided below can refer to the limitations of the service timeliness optimization method above, and will not be repeated here.

[0130] In an exemplary embodiment, Figure 4 As shown, a device for optimizing service timeliness is provided, including: an acquisition module 410 and a solution module 420, wherein:

[0131] The acquisition module 410 is used to acquire service feedback information within a historical time period; the service feedback information is an indicator when the service object feeds back a service request to the target object under the current cache strategy and the current routing strategy.

[0132] The solution module 420 is used to combine the service feedback information and the preset timeliness optimization model to solve and obtain the corresponding target cache strategy and target routing strategy for the future time period; the target cache strategy and target routing strategy are used to optimize the service timeliness.

[0133] In one embodiment, the above device further includes an optimization model construction module, and the optimization model construction module includes:

[0134] The path timeliness optimization unit is used to determine the path when the service object feeds back the service request to the target object according to the service object, and to construct the timeliness corresponding to the path.

[0135] The optimization target unit is used to obtain a timeliness optimization target based on the timeliness of the path correspondence and the response delay of the service object.

[0136] The optimization model unit is used to obtain a timeliness optimization model according to preset optimization constraints and timeliness optimization goals.

[0137] In one embodiment, the service objects in the acquisition module 410 include edge servers and cloud servers; the paths in the path timeliness optimization unit include edge service paths and cloud service paths; the response delays of the service objects in the optimization target unit include edge response delays and cloud response delays; wherein, the edge service path is the transmission path from the edge server to the target object; wherein, the cloud service path is the transmission path from the cloud server to the edge server and from the edge server to the target object.

[0138] In one embodiment, the path timeliness optimization unit includes:

[0139] The edge path subunit is used to build the timeliness corresponding to the edge service path through the data transmission mode corresponding to the edge server to the target object.

[0140] The cloud path subunit is used to build the timeliness of the cloud service path through the link bandwidth corresponding to the cloud server to the edge server and the data transmission method corresponding to the edge server to the target object.

[0141] In one embodiment, the above-mentioned optimization target unit includes:

[0142] The edge optimization subunit is used to obtain the edge timeliness optimization target according to the timeliness corresponding to the edge service path and the edge response delay.

[0143] The cloud optimization subunit is used to obtain the cloud timeliness optimization target according to the timeliness corresponding to the cloud service path and the cloud response delay.

[0144] The target establishment subunit is used to obtain the timeliness optimization target according to the edge timeliness optimization target and the cloud timeliness optimization target.

[0145] In one embodiment, the solution module 420 includes:

[0146] The prediction unit is used to predict the initial cache strategy and the initial routing strategy according to the service feedback information.

[0147] The simulation unit is used to simulate the initial cache strategy and the initial routing strategy to obtain prediction feedback information.

[0148] The adjustment unit is used to adjust the initial cache strategy and the initial routing strategy according to the prediction feedback information until the prediction feedback information corresponding to the initial cache strategy and the initial routing strategy meets the timeliness optimization model, thereby obtaining the target cache strategy and the target routing strategy.

[0149] Each module in the above service timeliness optimization device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0150] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store service feedback information. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a service timeliness optimization method is implemented.

[0151] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0152] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the method in any one of the above embodiments when executing the computer program.

[0153] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in any one of the above embodiments are implemented.

[0154] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the method in any one of the above embodiments when executed by a processor.

[0155] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0156] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0157] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for optimizing service timeliness, characterized in that: The method comprises: Obtaining service feedback information within a historical time period; the service feedback information is an indicator of when the service object feeds back a service request to the target object under the current cache strategy and the current routing strategy; The service feedback information and the preset timeliness optimization model are combined to solve the corresponding target cache strategy and target routing strategy for the future time period; the target cache strategy and the target routing strategy are used to optimize the service timeliness.

2. The method according to claim 1, characterized in that The construction process of the timeliness optimization model includes: According to the service object, determining a path for the service object to feedback a service request to the target object, and constructing timeliness corresponding to the path; Based on the timeliness corresponding to the path and the response delay of the service object, a timeliness optimization target is obtained; The timeliness optimization model is obtained according to the preset optimization constraints and the timeliness optimization target.

3. The method according to claim 2, characterized in that The service object includes an edge server and a cloud server; the response delay of the service object includes an edge response delay and a cloud response delay; the path includes an edge service path and a cloud service path; The edge service path is a transmission path from the edge server to the target object; The cloud service path is a transmission path from the cloud server to the edge server and from the edge server to the target object.

4. The method according to claim 3, characterized in that The timeliness of the construction corresponding to the path includes: Constructing the timeliness corresponding to the edge service path through the data transmission mode corresponding to the edge server to the target object; The timeliness of the cloud service path is established by using the link bandwidth corresponding to the cloud server to the edge server and the data transmission method corresponding to the edge server to the target object.

5. The method according to claim 4, characterized in that The obtaining of a timeliness optimization target based on the timeliness corresponding to the path and the response delay of the service object includes: Obtaining an edge timeliness optimization target according to the timeliness corresponding to the edge service path and the edge response delay; Obtaining a cloud timeliness optimization target according to the timeliness corresponding to the cloud service path and the cloud response delay; The timeliness optimization target is obtained according to the edge timeliness optimization target and the cloud timeliness optimization target.

6. The method according to claim 1, characterized in that The solution is performed by combining the service feedback information and the preset timeliness optimization model to obtain the corresponding target cache strategy and target routing strategy for the future time period, including: Predict the initial cache strategy and initial routing strategy based on service feedback information; Simulating the initial cache strategy and the initial routing strategy to obtain prediction feedback information; According to the prediction feedback information, the initial cache strategy and the initial routing strategy are adjusted until the prediction feedback information corresponding to the initial cache strategy and the initial routing strategy meets the timeliness optimization model, thereby obtaining the target cache strategy and the target routing strategy.

7. A device for optimizing service timeliness, characterized in that: The device comprises: An acquisition module is used to acquire service feedback information within a historical time period; the service feedback information is an indicator of when a service object feeds back a service request to a target object under a current cache strategy and a current routing strategy; The solution module is used to combine the service feedback information and a preset timeliness optimization model to solve and obtain the corresponding target cache strategy and target routing strategy for the future time period; the target cache strategy and the target routing strategy are used to optimize service timeliness.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images