A method and system for collaborative optimization of task scheduling and data caching

Abstract

This invention discloses a method and system for collaborative optimization of task scheduling and data caching, belonging to the fields of edge computing and mobile network technology. The method includes: constructing an edge computing environment through multiple edge servers, cloud servers, and a data cache; performing service chain-aware matching on arriving tasks within the edge computing environment to reuse computation results; scheduling tasks using a particle swarm optimization algorithm; obtaining a task scheduling scheme based on a comprehensive fitness function and task scheduling; executing the task scheduling scheme to obtain intermediate result data items; evaluating the value of the intermediate result data items using a multi-dimensional evaluation model to obtain a value evaluation result; updating the content of the data cache based on a genetic algorithm and the value evaluation result; and achieving collaborative optimization between repeated task scheduling and data caching. This invention achieves effective processing of tasks combining multiple services through a service chain-aware matching mechanism and establishes a bidirectional feedback collaborative optimization mechanism for task scheduling and data caching.

Description

Technical Field

[0001] This invention relates to the fields of edge computing and mobile network technology, and in particular to a method and system for collaborative optimization of task scheduling and data caching. Background Technology

[0002] The rapid development of Internet of Things (IoT) technology and mobile internet has spurred the emergence of massive numbers of smart terminal devices and innovative mobile service applications. These applications place higher demands on service response time and computing power. Mobile Edge Computing (MEC) significantly reduces service response latency and alleviates the load on the core network by deploying computing resources at the network edge.

[0003] With the widespread adoption of smart applications, modern applications are increasingly composed of complex tasks consisting of multiple sequential processing steps. This task processing paradigm, which executes multiple functional modules in a predefined order, is defined as a Service Function Chain (SFC). Taking intelligent video surveillance as an example, a complete video analysis task requires multiple processing steps in sequence, such as video data decompression, object detection, feature extraction, and behavior recognition.

[0004] In a typical edge computing environment, multiple edge servers collaborate with cloud servers to process computing tasks. Edge servers typically have limited computing power and storage space, which constrains task processing. Existing research on handling service chaining tasks mainly faces the following problems: First, existing technologies mainly target task scheduling for single services and do not fully consider complex task forms composed of multiple service combinations, such as service chains, service workflows, and parallel service combinations, which makes it impossible to effectively utilize the dependencies and intermediate results between services.

[0005] Second, traditional task scheduling methods lack effective management and reuse mechanisms for intermediate result data. During service chain processing, each service generates new data items as input for subsequent services. These intermediate result data items have significant reuse value, but existing methods cannot effectively identify and utilize these computational results.

[0006] Third, the existing task scheduling strategy and data caching strategy are independent of each other and lack a collaborative optimization mechanism. The task scheduling decision does not consider the impact of cached content on scheduling efficiency, and the caching strategy is not dynamically adjusted according to the task scheduling results, resulting in the overall system performance failing to reach its optimal level.

[0007] Fourth, in resource-constrained edge computing environments, balancing task processing performance, server load, and cloud server usage costs to achieve system-level performance optimization remains a challenging problem. Summary of the Invention

[0008] This invention provides a method and system for collaborative optimization of task scheduling and data caching, which addresses the problems in the prior art such as insufficient consideration of complex task forms composed of multiple service combinations, lack of effective management and reuse mechanisms for intermediate result data, lack of collaborative optimization mechanisms, and inability to balance the relationship between task processing performance, server load, and cloud server usage costs.

[0009] On one hand, embodiments of the present invention provide a method for collaborative optimization of task scheduling and data caching, including: Step 1: Build an edge computing environment using multiple edge servers, cloud servers, and data caches; Step 2: In the edge computing environment, perform service chain-aware matching on the arriving tasks to achieve reuse of computing results; Step 3: Perform task scheduling using the particle swarm optimization algorithm; Step 4: Obtain the task scheduling scheme based on the comprehensive fitness function and the task scheduling; Step 5: Execute the task scheduling scheme to obtain intermediate result data items; Step 6: Use a multi-dimensional evaluation model to evaluate the value of the intermediate result data items and obtain the value evaluation results; Step 7: Update the content of the data cache according to the genetic algorithm and the value assessment results; Repeat steps two through seven to achieve collaborative optimization of task scheduling and data caching.

[0010] In one possible implementation, the service chain-aware matching includes: Match the result data items in the data cache with the data items required by the task; Match the service chain attribute of the result data item with the service chain of the task; Update the data items and service chains required for the task based on the matching results of the service chain attributes.

[0011] In one possible implementation, the comprehensive fitness function includes task completion time, server overlap penalty, and cloud server usage penalty, expressed by the following formula:

[0012]

[0013] in The task completion time is OP, the server overlap penalty is CP, the cloud server usage penalty is M, and the edge server set is M={m1,m2,...,m x}, m k m is an element in set M k The m k Represents the k-th edge server. It is the first The task of time , For the task The completion time.

[0014] In one possible implementation, the multi-dimensional evaluation model processes the value of the intermediate result data item based on its size, access frequency, and service chain.

[0015] In one possible implementation, the multi-dimensional evaluation model is further provided with a dynamic weight adjustment mechanism, which adaptively adjusts the weight parameters in the multi-dimensional evaluation model according to the operating state of the entire system.

[0016] In one possible implementation, updating the data cache based on the genetic algorithm and the value assessment result includes: The genetic algorithm is used to determine the normal, consistent cache content of the data cache. The cached content for the data cache in the case of insufficient space is determined based on the value assessment results.

[0017] On the other hand, embodiments of the present invention provide a task scheduling and data caching collaborative optimization system, including: an environment building module, used to build an edge computing environment through multiple edge servers, cloud servers and data caches; The service chain-aware matching module is used to perform service chain-aware matching on the arriving tasks in the edge computing environment to achieve the reuse of computing results; The task scheduling module is used to schedule the tasks using the particle swarm optimization algorithm and to obtain a task scheduling scheme based on the comprehensive fitness function and the task scheduling. The data evaluation module is used to execute the task scheduling scheme to obtain intermediate result data items, and to evaluate the value of the intermediate result data items through a multi-dimensional evaluation model to obtain the value evaluation result. The cache management module is used to update the content of the data cache based on the genetic algorithm and the value evaluation results; The collaborative optimization control module is used to control the service chain perception and matching module, task scheduling module, data evaluation module and cache management module to achieve collaborative optimization of task scheduling and data caching.

[0018] The task scheduling and data caching collaborative optimization method and system of the present invention have the following advantages: (1) Through the service chain awareness matching mechanism, the system can effectively process multiple service combination tasks. The system can identify and utilize the dependencies in the service chain, dynamically optimize the task execution path by matching the intermediate result data in the cache, avoid repeated calculations, and significantly improve processing efficiency.

[0019] (2) A two-way feedback collaborative optimization mechanism for task scheduling and data caching was established. The results of task scheduling provide a basis for evaluating the value of data for caching strategies, while the optimized cached content provides more reuse opportunities for subsequent task scheduling, forming a positive feedback loop and continuously improving system performance.

[0020] (3) A multi-dimensional data value assessment model and dynamic weight adjustment mechanism were designed. The model comprehensively considers multiple dimensions such as data item size, access frequency, and service chain processing value, and can adaptively adjust weight parameters according to the system operating status to ensure that the caching strategy can adapt to different workload modes.

[0021] (4) Intelligent optimization algorithms are used to solve complex scheduling and caching optimization problems. Particle swarm optimization algorithm is used for task scheduling, taking into account task completion time, server load balancing and cloud server usage costs; genetic algorithm is used for cache content optimization to maximize data value under limited storage space conditions. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a task scheduling and data caching collaborative optimization method provided in this application embodiment; Figure 2 A structural diagram of a task scheduling and data caching collaborative optimization system provided in this application embodiment; Figure 3 The figure shows the variation of average processing latency of each round of tasks in a task scheduling simulation based on service chain-aware matching combined with data caching, which is provided in the embodiments of this application. Figure 4A trend chart showing the changing task optimization rate in each round of task scheduling simulation based on service chain-aware matching combined with data caching, provided for an embodiment of this application; Figure 5 A schematic diagram illustrating the impact of cache capacity on system performance in a task scheduling and data caching collaborative optimization system provided in this application, showing the average completion latency of each round of tasks under different MEC storage capacities; Figure 6 A graph showing the change in task optimization rate of a task scheduling and data caching collaborative optimization system provided in this application embodiment, illustrating the impact of cache capacity on system performance; Figure 7 A comparison chart of the average completion latency under different numbers of tasks in a task scheduling and data caching collaborative optimization system provided in this application embodiment; Figure 8 A comparison chart of the corresponding task optimization rates of a task scheduling and data caching collaborative optimization system provided in this application embodiment, showing the impact of the number of tasks per round on system performance; Figure 9 A comparison chart of the average completion latency under different initial data item numbers for a task scheduling and data caching collaborative optimization system provided in this application embodiment; Figure 10 A comparison chart showing the impact of the number of initial data item types on system performance in a task scheduling and data caching collaborative optimization system provided in this application embodiment; Figure 11 A comparison chart of the average task completion latency under different service types for the impact of the number of service types on system performance in a task scheduling and data caching collaborative optimization system provided in this application embodiment; Figure 12 A comparison chart of the corresponding task optimization rates of the impact of the number of service types on system performance in a task scheduling and data caching collaborative optimization system provided in this application embodiment; Figure 13 A comparison chart of the average completion latency in each round of experiments for a task scheduling and data caching collaborative optimization system provided in this application embodiment, showing the effect of the data caching module of the system. The system uses caching and does not use caching for task scheduling schemes. Figure 14 A comparison of runtime in multiple experiments of two schemes for the data caching module effect of a task scheduling and data caching collaborative optimization system provided in this application embodiment; Figure 15A comparison of convergence curves of four algorithms (SCCTS, SCHTS, SCGTS, and SCPTS) for different optimization algorithms in a task scheduling and data caching collaborative optimization system provided in this application embodiment. Figure 16 This is a comparison chart of the average convergence time of different optimization algorithms for a task scheduling and data caching collaborative optimization system provided in this application embodiment. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Figure 1 This is a flowchart illustrating a task scheduling and data caching collaborative optimization method provided by an embodiment of the present invention; the embodiment of the present invention provides a task scheduling and data caching collaborative optimization method, including: Step 1: Build an edge computing environment using multiple edge servers, cloud servers, and data caches; Step 2: In the edge computing environment, perform service chain-aware matching on the arriving tasks to achieve reuse of computing results; Step 3: Perform task scheduling using the particle swarm optimization algorithm; Step 4: Obtain the task scheduling scheme based on the comprehensive fitness function and the task scheduling; Step 5: Execute the task scheduling scheme to obtain intermediate result data items; Step 6: Use a multi-dimensional evaluation model to evaluate the value of the intermediate result data items and obtain the value evaluation results; Step 7: Update the content of the data cache according to the genetic algorithm and the value assessment results; Repeat steps two through seven to achieve collaborative optimization of task scheduling and data caching.

[0026] The service chain-aware matching includes: Match the result data items in the data cache with the data items required by the task; Match the service chain attribute of the result data item with the service chain of the task; Update the data items and service chains required for the task based on the matching results of the service chain attributes.

[0027] The comprehensive fitness function includes task completion time, server overlap penalty, and cloud server usage penalty, expressed by the following formula:

[0028]

[0029] in The task completion time is OP, the server overlap penalty is CP, the cloud server usage penalty is M, and the edge server set is M={m1,m2,...,m x}, m k m is an element in set M k The m k Represents the k-th edge server. It is the first The task of time , For the task The completion time.

[0030] The multi-dimensional evaluation model processes the value of the intermediate result data items based on their size, access frequency, and service chain.

[0031] The multi-dimensional evaluation model also includes a dynamic weight adjustment mechanism, which adaptively adjusts the weight parameters in the multi-dimensional evaluation model based on the overall system operating status.

[0032] The content of the data cache is updated based on the genetic algorithm and the value assessment results, including: The genetic algorithm is used to determine the normal, consistent cache content of the data cache. The cached content for the data cache in the case of insufficient space is determined based on the value assessment results.

[0033] For example, a general edge computing environment consists of a set of X edge servers, M = { , ,..., } and a cloud server It consists of. Each edge server M has the following attributes: total computing power Total storage space It has an independent data item cache. In data item cache There is an initial data item D={ , ,..., } and the resulting data item R= Initial data items These are attributes of the task, used to describe the data items required for the task's calculation, and the resulting data items. It is the result generated after the task uses the initial data items through a single service or service chain.

[0034] Meanwhile on the edge server and There is bandwidth between them Edge server There is bandwidth between the cloud and the cloud. , The transmission speed between each MEC was calculated using the Shannon formula. And the transmission speed between each MEC and the cloud To form a transmission speed matrix Where G is the channel gain, N is the channel noise density, and P is the device antenna transmit power. Calculate each... Form a matrix.

[0035]

[0036]

[0037] The set of service types consisting of L types of services is S = { , ,..., }. Simultaneously using a matrix To represent the service deployment status of all edge servers, each element in the matrix... Edge servers are all represented by binary variables. Does the service exist? S.

[0038] =

[0039] At any moment There are W tasks U={ , ,..., Reaching the edge server. Each task The following attributes will be present during generation: Required initial data items Required computing power ; Task size Required service chain .

[0040] Table 1 summarizes the key symbols used in edge server environments: Table 1. Symbol table used in edge computing environments

[0041] Based on traditional edge environment data storage, the following data storage model was designed considering the characteristics of service chains and the concept of serving task execution.

[0042] First, create a data item record table to record the attribute information of data items. When a data item... or On a server When a data item appears in the table, its attribute information needs to be updated in the data item record table. This applies to initial data items. The attribute information is the size of the data item. and frequency of occurrence As for the result data items This will add the following to the initial data item attributes: the initial data item used by the resulting data item. The service chain it passes through .

[0043] The result data items are not only the results generated after the task has gone through all the services in the service chain, but also the intermediate results generated when the task passes through certain services in the service chain. For example, the task... Initial data items are needed and complete the service chain. When using initial data items After the first service The intermediate result data generated afterwards This is referred to as the result data item, and its frequency of occurrence is... Initial data items used Initial data items used Record the data and calculate its size. .

[0044] =

[0045] in It is a service The influencing factor is derived from the role of the service, indicating the direction and magnitude of changes in the size of data items, such as the service. If it's an unzipped file, then... >0 and The value is relatively large. However, in this example, three result data items will be recorded in the data item record table.

[0046] This record table is independent for each server because each edge server covers a different area. In edge computing environments, tasks within a specific geographical area often exhibit strong spatial locality. This means that within the same edge server's coverage area, task data and service requirements may be highly similar. Secondly, the homogeneity of tasks is caused by specific user behavior patterns, application types, environments, and factors within a region. Common user needs and region-specific application scenarios within a certain range may lead to similar data access patterns. Therefore, records and subsequent updates are performed separately for each edge server.

[0047] For data item caching on each MEC The total size of all data items must be less than the data item cache. ,in Indicates the size of the data item.

[0048]

[0049] First, we need to be aware of the time. The service chain of incoming tasks is matched with the data storage. Task service chain matching aims to pair result data items in the data item cache with the task's attributes, thereby updating the task's service chain and the required data items. Matched tasks are categorized into three types: unoptimized, partially optimized, and fully optimized. First, using... This indicates whether the task has been optimized.

[0050] =

[0051] After the task service chain is matched, it can be accessed. Calculate the time Optimization rate of all incoming tasks .

[0052]

[0053] Where W is time. The total number of tasks at that time. Unoptimized tasks are those that do not have a usable result data item matched in the data item cache. Therefore, the task inherits the required data items and service chains that need to be completed before the task was optimized. =

[0054] =

[0055] Both the partially optimized and fully optimized tasks have already been matched with corresponding result data items in the data item cache, so their required data items have changed to the found result data items. : =

[0056] Furthermore, they need to complete the service chain transformation to the initial state of the service chain and remove the service chain containing the found result data items: = -

[0057] For fully optimized tasks, no further execution is needed; only the result data items need to be retrieved. Therefore, the service chain for this task is empty. =

[0058] After optimization, the tasks arriving at this point enter the task scheduling section, where the goal of task scheduling is to optimize each task. Each service in the service chain Find an optimal edge server To achieve this, R minimizes the average completion latency of all tasks. (Task completion latency) It is mainly divided into three parts: the transmission latency of the task and the required data items. And the computational latency of the task .

[0059] =

[0060] During the scheduling process + <

[0061] in It is a binary variable representing a data item. Is it cached on the server? It is server m k The remaining cache capacity, It is a task Size.

[0062] =

[0063] A transfer matrix was established when describing a general edge computing environment. For unoptimized and partially optimized tasks, the number of transmissions is... -1 represents the service chain length reduced by one, which accounts for the delay in each transmission. This includes the data item transmission delay. and task transmission latency .

[0064] =

[0065] =

[0066] =

[0067] in, It is a data item The transmission latency to server k It is a task The transmission latency to server k, when two adjacent services in a service chain are executed on the same server. and All are 0. For a fully optimized task, the completion latency is... Only for data item transmission delay In other words, the transmission latency of sending the retrieved result data items to the server where the task is located. = and = 0.

[0068] To reasonably calculate the computation latency of tasks running in parallel on the edge server, an overlapping interval method is used. If multiple tasks execute simultaneously within the same time interval, their computation is recalculated. The computational power required for the latency-sensitive task is... The server's total computing power is The number of tasks currently being processed on the server is The calculation delay formula for the task is as follows: = (1+log( )) /

[0069] in It is server m k Looking back at the total computing power, each task has a total latency. Then, the task scheduling decision can be formulated as an optimization problem, the objective of which is to minimize the time step at time 1. The average processing latency of incoming tasks, of which For a moment The optimization rate of the incoming task set.

[0070] Min

[0071] st =

[0072] + <

[0073] =

[0074] =

[0075] =

[0076] = (1+log( )) /

[0077]

[0078]

[0079] To address the various factors that need to be considered in the task scheduling problem, a two-stage optimization method is proposed. Two optimization methods are performed alternately, and through multiple iterations, the continuously updated cache provides continuous optimization for task processing, finally obtaining the optimal solution for the task scheduling scheme.

[0080] The first stage is task processing, at any given moment. A certain number of tasks arrive at the edge server. At this stage, a service chain-aware matching and task scheduling decision-making scheme is generated for this random number of tasks.

[0081] First, perform service chain-aware matching, using the attributes of the task: required data items. Match all cached result data items on the server. The properties of each result data item contain its initial data item. .

[0082]

[0083] After the first matching, a set was obtained. Next, we'll use another attribute of the task: the service chain to be completed. , go to the collection The matching process takes place within the task's service chain. Go and Collection The attributes of the result data items and the service chains they traversed. Perform a match. =

[0084] After each set has been searched If none of the result data items are found, then attempt to link the task's service chain. The last service is removed, and matching continues until only one service remains without a match, indicating that no result data item can optimize the task. If a result data item is matched during the above process... ,but This is the optimized task. Required data items The service chain of this task It's the original service chain. Subtract the service chain of the result data item The rest = -

[0085] After the tasks are matched through the service chain, the required data items for all tasks will be determined by... Updated to ; and the service chain for the task is Updated to Use new task attributes to schedule it.

[0086] The system consists of an edge server set M and a cloud server. The system consists of edge servers with limited service capabilities and computing resources, while cloud servers, although possessing all service capabilities, have higher usage costs. To optimize this complex scheduling scenario, a service chain-aware task scheduling algorithm based on PSO is designed. In this algorithm, each particle represents a complete scheduling scheme, containing the execution time arrangements of all tasks on different servers. A matrix is ​​used to represent the complete scheduling scheme represented by the particle. The first dimension of this matrix is ​​the task number, and the second dimension is the entire scheduling process of a task, including the position of each service in the service chain executing the task and the start and end times at that position. Below is an example of a particle.

[0087]

[0088] As shown in the matrix, for Clearly, the service chain has two services, so there are two servers to complete the service, namely... and First of all The processing time ranges from 3.1 to 7.5 seconds, indicating that the task and the data items required for that task are transferred to the server. The time is 3.1, the computation time of the first service of the task is 7.5-3.1, and then the task... and The transmission time between services is the start time of the second service minus the end time of the first service, i.e., 9 - 7.5, for a total time of 15. If the two services in the task's service chain are executed on the same server, the transmission time is 0. For example... The first two services; special cases such as Task 3, which is a fully optimized task, are empty. Therefore, the completion time for all tasks is... The calculation formula is as follows: +=

[0089] in For the task The completion time.

[0090] The algorithm's fitness function comprehensively considers three key factors: total completion time, server overlap penalty, and cloud server usage penalty. The overlap penalty analyzes the time overlap of tasks on edge servers and uses a logarithmic function to calculate the performance loss caused by parallel execution, thereby avoiding overloading a single server. Furthermore, by introducing a server load factor, the differences in processing capabilities among different servers are further refined. The overlap penalty (OP) formula is as follows:

[0091] Where I represents the overlapping interval I, and OI represents the set of all overlapping intervals. Represents the set of tasks within interval I. Indicates the duration of interval I. Represents edge server Loading factor, Indicates task The completion time without considering overlap. Indicates task The overlapping time within interval I. To avoid excessive use of cloud servers, a cloud server usage penalty is set. The penalty formula for cloud servers (CP) is as follows:

[0092] in Indicates the base penalty coefficient. Indicates the utilization rate of cloud servers. The threshold representing cloud server utilization. This represents the total penalty for all tasks assigned to the cloud server (proportional to the number of tasks). The fitness function value is the sum of the total task completion time and the penalty.

[0093]

[0094] The algorithm continuously updates the particle's velocity and position during the iteration process. The velocity update considers three aspects: inertia weight, individual cognition, and social cognition. Among them, the inertia weight is dynamically adjusted with the number of iterations to balance the global search and local search capabilities. The particle velocity update formula is as follows.

[0095]

[0096] in Represents particles The speed at the pt-th iteration; Represents particles The optimal position of the individual at the pt-th iteration; Represents particles The position at the pt-th iteration; Indicates the globally optimal position; It is the inertial weight, used to control the tendency of a particle to maintain its current velocity; and These represent cognitive learning factors and social learning factors, respectively, which are used to control the tendency of particles to move toward the individual optimum and the global optimum. and Represents a random number between [0,1]; After the location is updated, the algorithm will repair the solution to ensure that the scheduling scheme meets the server capacity constraints and task dependencies. The location update formula is as follows.

[0097]

[0098] To ensure convergence and solution quality, the algorithm employs an elitist retention strategy, maintaining the optimal solution throughout the iteration process. Simultaneously, by setting appropriate population size and iteration counts, a balance is struck between computational overhead and solution quality. Experiments demonstrate that the algorithm effectively balances task completion time, server load, and cloud server usage costs. Compared to traditional scheduling algorithms, this algorithm exhibits superior performance when handling complex tasks with service chains and data dependencies. Particularly under resource-constrained edge server conditions, through a reasonable overlap penalty mechanism and cloud server usage control, it achieves efficient resource utilization while maintaining low system overhead.

[0099] Finally, the task scheduling decision with the best fitness value is used as the final solution. The entire process of this solution is also the detailed process of generating the result data items. This solution is used to update the data caching system.

[0100] The algorithm's output includes complete scheduling information and details about the result data items generated by each task on a specific server during the service chain process. After obtaining this information, cache updates begin. The generated result data items record their generation location. Some data items reach cache thresholds, which are determined by the number of cached items. and frequency threshold , >

[0101] >

[0102] The formula for calculating frequency is as follows: =

[0103] in This indicates the time when the result data item last appeared. Similarly, Indicates the time of the first occurrence. This represents the number of times the resulting data item appears. If a threshold is met, caching can be considered. If the remaining cache space is greater than the size of the data item, it is stored; otherwise, a cache update is performed. A genetic algorithm is used to generate an individual population using all data items that meet the requirements for storage on the current server. After iteration, the individual with the optimal fitness value is obtained, and the cache is updated according to the content of that individual.

[0104] A comprehensive scoring function was designed for calculating the fitness value in genetic algorithms. This function consists of the size, frequency, and utility of the resulting data items, respectively corresponding to... , The comprehensive scoring function is as follows: ( / )= + +

[0105] As for weight , To consider various factors related to data items, a weight update method was designed. This method primarily involves dynamically adjusting three key weight parameters of the caching strategy by analyzing various system metrics: (SIZE weight) (FREQUENCY weight) (UTILITY weights). First, calculate four key metrics: cache utilization, average data size, optimization rate trend, cache utilization trend, and data size trend. When the optimization rate shows a downward trend, make corresponding adjustments based on the cache utilization: if the cache utilization exceeds the weight adjustment threshold... Then increase and reduce ,Keep Unchanged; if the data size shows an increasing trend, then increase. and reduce At the same time, maintain Unchanged. Conversely, when the optimization rate is trending upward and cache utilization is also improving, slightly increase. At the same time, maintain and The settings remain unchanged. This adjustment mechanism ensures that the caching strategy can adaptively optimize based on the system's operating status. Finally, all weights undergo boundary checks and normalization to ensure that the weight values ​​always remain within a reasonable range and that the sum of the three weights is 1. This dynamic adjustment mechanism allows the caching system to better adapt to different workloads and data access patterns. At the end of each optimization round, weight checks and updates ensure that the system can continue to perform its work stably.

[0106] The final results can be summarized in the following aspects: As the number of rounds of the experiment increases (e.g. Figure 3 , Figure 4 (x-axis) average task completion time ( Figure 3 The vertical axis is stable within a low range, indicating the task's optimization rate ( Figure 4 The vertical axis continuously increases and then tends to stabilize at a certain value.

[0107] like Figures 5 to 12 As shown, we modified the variables of edge server storage capacity (in TB), number of tasks per round, number of initial data items, and service type to observe the impact on average task completion latency and average task optimization rate.

[0108] Figure 13 and Figure 14 The impact of having or not having data caching on the average processing latency of the entire task scheduling and the difference in the total runtime of the algorithm are respectively.

[0109] Figure 15 and Figure 16 The comparison shows the optimal solution obtained by different algorithms in one round of experiments and the time taken to obtain the optimal solution, illustrating the reasons for choosing the algorithm.

[0110] This paper introduces a two-stage optimization method: task scheduling based on service chain-aware matching and a data caching method. Task processing is divided into service chain-aware matching and task scheduling. Service chain-aware matching reduces processing steps by matching the data cached data with the required data items and service chains, thus simplifying the task. Task scheduling utilizes particle swarm optimization to obtain an optimal solution that avoids tasks running concurrently and minimizes the use of cloud servers, achieving optimal latency. The result data items from task processing update the data cache. During the update process, if the cache space is insufficient for direct storage, a genetic algorithm is used on all data items to be cached to obtain the optimal storage scheme for updating the cache, providing continuous assistance for the next task's service chain-aware matching. In summary, these two optimization methods alternate and exchange information to achieve the best overall system performance and optimize task processing latency.

[0111] This invention also provides a task scheduling and data caching collaborative optimization system, comprising: The environment building module is used to build an edge computing environment using multiple edge servers, cloud servers, and data caches. The service chain-aware matching module is used to perform service chain-aware matching on the arriving tasks in the edge computing environment to achieve the reuse of computing results; The task scheduling module is used to schedule the tasks using the particle swarm optimization algorithm and to obtain a task scheduling scheme based on the comprehensive fitness function and the task scheduling. The data evaluation module is used to execute the task scheduling scheme to obtain intermediate result data items, and to evaluate the value of the intermediate result data items through a multi-dimensional evaluation model to obtain the value evaluation result. The cache management module is used to update the content of the data cache based on the genetic algorithm and the value evaluation results; The collaborative optimization control module is used to control the service chain perception and matching module, task scheduling module, data evaluation module and cache management module to achieve collaborative optimization of task scheduling and data caching.

[0112] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0113] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for task scheduling and data cache co-optimization, characterized in that, The application relates to a method for realizing collaborative optimization of task scheduling and data caching. The method comprises the following steps: step 1, constructing an edge computing environment through multiple edge servers, cloud servers and data caches; step 2, performing service chain perception matching on an arrived task in the edge computing environment to realize result multiplexing; step 3, performing task scheduling on the task through a particle swarm optimization algorithm; step 4, obtaining a task scheduling scheme according to a comprehensive fitness function and the task scheduling; step 5, obtaining intermediate result data items by executing the task scheduling scheme; step 6, performing value evaluation on the intermediate result data items through a multi-dimensional evaluation model to obtain a value evaluation result; and step 7, updating the content of the data cache according to a genetic algorithm and the value evaluation result. The method further comprises the following steps: repeating the steps 2 to 7 to realize collaborative optimization of task scheduling and data caching. The service chain perception matching comprises the following steps: matching result data items in the data cache according to data items required by the task; matching service chain attributes of the result data items according to service chains of the task; and updating the data items and the service chains required by the task according to a matching result of the service chain attributes. The comprehensive fitness function comprises task completion time, server overlap penalty and cloud server use penalty, and is expressed by the following formula: f (x) = w1 * T (x) + w2 * P (x) + w3 * C (x), wherein T (x) represents the task completion time, P (x) represents the server overlap penalty, C (x) represents the cloud server use penalty, and w1, w2 and w3 represent weight parameters. The multi-dimensional evaluation model evaluates the value of the intermediate result data items according to the size, access frequency and service chain of the intermediate result data items. The multi-dimensional evaluation model is further provided with a dynamic weight adjustment mechanism which adjusts the weight parameters in the multi-dimensional evaluation model according to the running state of the whole system. The method further comprises the following steps: determining the regular normal cache content of the data cache through the genetic algorithm; and determining the cache content of the insufficient space state of the data cache according to the value evaluation result. The application further provides a method for realizing collaborative optimization of task scheduling and data caching.

2. The method of claim 1, wherein, The method comprises the following steps: constructing an edge computing environment through multiple edge servers, cloud servers and data caches by an environment construction module; performing service chain perception matching on an arrived task in the edge computing environment to realize result multiplexing by a service chain perception matching module; performing task scheduling on the task through a particle swarm optimization algorithm by a task scheduling module; obtaining a task scheduling scheme according to a comprehensive fitness function and the task scheduling by the task scheduling module; executing the task scheduling scheme to obtain intermediate result data items by a data evaluation module; performing value evaluation on the intermediate result data items through a multi-dimensional evaluation model to obtain a value evaluation result by the data evaluation module; updating the content of the data cache according to a genetic algorithm and the value evaluation result by a cache management module; and controlling the service chain perception matching module, the task scheduling module, the data evaluation module and the cache management module to realize collaborative optimization of task scheduling and data caching by a collaborative optimization control module. ​ ​ ​ 3. The method of claim 1, wherein, ​ wherein is the task completion time, OP is the server overlap penalty, CP is the cloud server usage penalty, M is a set of edge servers, said M = {m1, m2,..., m x}, m k is an element m k from the set M, said m k represents the kth edge server, is the task at time t, , is the completion time of the task .

4. The method of claim 1, wherein, ​ 5. The method of claim 4, wherein, ​ 6. The method of claim 1, wherein, ​ ​ ​ 7.A system for task scheduling and data cache co-optimization, characterized in that, ​ ​ ​ ​ ​ ​ ​

Images