Fine-grained optical network service scheduling method and device

Abstract

The embodiment of the invention provides a fine-grained optical network service scheduling method and device, and the method comprises the steps: calculating a load parameter of an optical service unit according to the calculation power demand of a service borne by the optical service unit; calculating a time delay parameter of the optical service unit according to the time delay sensitivity parameter of the service carried by the optical service unit; determining a replication dynamic equation of the optical service unit in the current network state based on the load parameter and the time delay parameter; solving the replication dynamic equation based on a Q learning method, and determining a time delay weight parameter for scheduling the service according to a time delay optimization strategy and a load weight parameter for scheduling the service according to a load balancing strategy; and scheduling services according to the time delay optimization strategy and the corresponding time delay weight parameters, and according to the load balancing strategy and the corresponding load weight parameters. Optimized and reasonable scheduling of services with different granularities can be realized, the requirements of different services are met, and the resource utilization rate and the system performance are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of optical communication technology, and in particular to a fine-grained optical network service scheduling method and device. Background Art

[0002] Optical transport networks (OTN) are deployed on a large scale in backbone networks and metropolitan area networks with large bandwidth and long-distance transmission capabilities. However, when faced with various types of small-granularity services with low bandwidth requirements and low latency, there is a waste of resources and the problem of being unable to meet business needs. Optical Service Units (OSU) and Fine-Grained Optical Transport Networks (fgOTN) are designed to address the shortcomings of OTN in small-granularity services. They provide smaller bandwidth granularity in hardware and can realize the transmission of multi-granularity services. However, there is no solution for how to optimize the scheduling according to the type and demand of multi-granularity services. Summary of the invention

[0003] In view of this, the purpose of the embodiments of the present application is to propose a fine-grained optical network data scheduling method and device to solve the problem of service scheduling.

[0004] Based on the above purpose, the embodiment of the present application provides a fine-grained optical network service scheduling method, including:

[0005] Calculate the load parameters of the optical service unit according to the computing power requirements of the services carried by the optical service unit;

[0006] Calculating the delay parameter of the optical service unit according to the delay sensitivity parameter of the service carried by the optical service unit;

[0007] Based on the load parameter and the delay parameter, determining a replication dynamic equation of the optical service unit under the current network state;

[0008] Solving the replication dynamic equation based on the Q learning method, determining the delay weight parameter for scheduling services according to the delay optimization strategy, and the load weight parameter for scheduling services according to the load balancing strategy;

[0009] Services are scheduled according to the delay optimization strategy and the corresponding delay weight parameters, and according to the load balancing strategy and the corresponding load weight parameters.

[0010] Optionally, determining the replication dynamic equation of the optical service unit in the current network state based on the load parameter and the delay parameter includes:

[0011] Determine the load balancing benefit according to the load parameter and a preset load benefit function;

[0012] Determining the delay benefit according to the delay parameter and a preset delay benefit function;

[0013] The replication dynamic equation is determined according to the load balancing benefit and the delay benefit.

[0014] Optionally, the load benefit function is:

[0015] U l (s i )=-Mean(L(s i )) / S (3)

[0016] Among them, L(s i ) is the optical service unit s i The load parameter, S is the normalization factor, and Mean() is the mean function;

[0017] The delay benefit function is:

[0018] U d (s i )=-D(s i ) (4)

[0019] Among them, D(s e ) is the optical service unit s i Delay parameters;

[0020] The replication dynamic equation is:

[0021]

[0022] Among them, the stable point corresponding to the replicated dynamic equation is d , χ l are the delay weight parameter and the load weight parameter respectively, is the average return, calculated as:

[0023]

[0024] Among them, U d (s d ) is the benefit obtained by executing the delay optimization strategy, U l (s l ) is the benefit of executing the load balancing strategy.

[0025] Optionally, solving the replication dynamic equation based on a Q-learning method includes:

[0026] Taking the current network state as the current state, selecting the delay optimization strategy or the load balancing strategy as an action, and calculating the reward value according to a preset reward function after executing the selected strategy;

[0027] According to the reward value, the Q value is updated according to the preset Q value update method until the maximum Q value is obtained, and the χ value corresponding to the maximum Q value is d , χ l As a stable point for replicating the dynamic equations.

[0028] Optionally, the reward function is:

[0029] R = x d ×U d (s i )+χ l ×U l (s i ) (13)

[0030] The Q value updating method is:

[0031]

[0032] Where η is the learning rate, γ is the discount factor, R is the reward value obtained by executing action a in state s, Q(s,a) is the Q value obtained by executing action a in state s, s′ is the new state after executing action a, and a′ is the new action executed in the new state. is the maximum Q value obtained by executing action a′ in the new state s′.

[0033] The embodiment of the present application also provides a fine-grained optical network service scheduling device, including:

[0034] A load calculation module, used to calculate the load parameters of the optical service unit according to the computing power requirements of the services carried by the optical service unit;

[0035] A delay calculation module, used to calculate the delay parameter of the optical service unit according to the delay sensitivity parameter of the service carried by the optical service unit;

[0036] An equation determination module, used to determine the replication dynamic equation of the optical service unit under the current network state based on the load parameter and the delay parameter;

[0037] A solution module, used to solve the replication dynamic equation based on a Q learning method, determine a delay weight parameter for scheduling services according to a delay optimization strategy, and a load weight parameter for scheduling services according to a load balancing strategy;

[0038] The scheduling module is used to schedule services according to the delay optimization strategy and the corresponding delay weight parameters, and according to the load balancing strategy and the corresponding load weight parameters.

[0039] Optionally, the equation determination module is used to determine the load balancing benefit according to the load parameters and a preset load benefit function; determine the delay benefit according to the delay parameters and a preset delay benefit function; and determine the replication dynamic equation according to the load balancing benefit and the delay benefit.

[0040] Optionally, the load benefit function is:

[0041] U l (s i )=-Mean(L(s i )) / S (3)

[0042] Among them, L(s i ) is the optical service unit s i The load parameter, S is the normalization factor, and Mean() is the mean function;

[0043] The delay benefit function is:

[0044] U d (s i )=-D(s i ) (4)

[0045] Among them, D(s i ) is the optical service unit s i Delay parameters;

[0046] The replication dynamic equation is:

[0047]

[0048] Among them, the stable point corresponding to the replicated dynamic equation is d , χ l are the delay weight parameter and the load weight parameter respectively, is the average return, calculated as:

[0049]

[0050] Among them, U d (s d ) is the benefit obtained by executing the delay optimization strategy, U l (s l ) is the benefit of executing the load balancing strategy.

[0051] Optionally, the solution module is used to take the current network state as the current state, select the delay optimization strategy or the load balancing strategy as an action, and after executing the selected strategy, calculate the reward value according to a preset reward function; update the Q value according to the reward value according to a preset Q value update method until the maximum Q value is obtained, and replace the x value corresponding to the maximum Q value with the x value corresponding to the maximum Q value. d , χ l As a stable point for replicating the dynamic equations.

[0052] Optionally, the reward function is:

[0053] R = x d ×U d (s i )+χ l ×U l (s i ) (13)

[0054] The Q value updating method is:

[0055]

[0056] Where η is the learning rate, γ is the discount factor, R is the reward value obtained by executing action a in state s, Q(s,a) is the Q value obtained by executing action a in state s, s′ is the new state after executing action a, and a′ is the action performed in the new state. is the maximum Q value obtained by executing action a′ in the new state s′.

[0057] From the above, it can be seen that the fine-grained optical network data scheduling method and device provided in the embodiment of the present application calculates the load parameters of the optical service unit according to the computing power requirements of the service carried by the optical service unit, calculates the delay parameters of the optical service unit according to the delay sensitivity parameters of the service carried by the optical service unit, determines the replication dynamic equation of the optical service unit in the current network state based on the load parameters and the delay parameters, solves the replication dynamic equation based on the Q learning method, determines the delay weight parameters for scheduling the service according to the delay optimization strategy, and the load weight parameters for scheduling the service according to the load balancing strategy; schedules the service according to the delay optimization strategy and the corresponding delay weight parameters, and according to the load balancing strategy and the corresponding load weight parameters. The present application can realize the optimized and reasonable scheduling of services of different granularities, meet the needs of different services, and improve resource utilization and system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 A schematic diagram of a method flow of an embodiment of the present application;

[0060] Figure 2 A schematic diagram of the optical network topology structure of an embodiment of the present application;

[0061] Figure 3 This is a block diagram of the device structure of an embodiment of the present application;

[0062] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0064] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0065] In the related technologies, the optical service unit (OSU) is a technical improvement made to address the shortcomings of OTN technology. It adopts a payload block division method and can carry services of different granularities of 2M-100Gbps. In specific applications, the services of different granularities carried by the optical service unit include various types, and different types of services have different requirements for resources and quality. For example, for services with high real-time requirements such as video conferencing, online transactions, and autonomous driving, it is necessary to provide delay optimization scheduling strategies from the perspectives of time and path planning; for services with high concurrent access such as e-commerce websites and social platforms, services with high resource consumption such as big data analysis and machine learning training, and distributed system services such as content distribution networks and microservice architectures, it is necessary to provide optimized load balancing scheduling strategies from the perspective of computing resources. However, the current optical service unit only supports the transmission of services of different granularities in terms of hardware, and does not provide solutions for the types and requirements of services of different granularities to meet business needs through reasonable scheduling.

[0066] In view of this, an embodiment of the present application provides a fine-grained optical network service scheduling method, which determines the weight parameters of the optical service unit's execution of the delay optimization strategy and the load balancing strategy based on the evolutionary game model and the reinforcement learning method according to the current network status such as the load and delay of the optical service unit. For the carried services, scheduling is performed according to the delay optimization strategy and the corresponding weight parameters, and according to the load balancing strategy and the corresponding weight parameters. Through fine-grained service scheduling, the needs of different service types can be met and the overall performance of the system can be improved.

[0067] like Figure 1 As shown, the embodiment of the present application provides a fine-grained optical network service scheduling method, including:

[0068] S101: Calculate a load parameter of the optical service unit according to a computing power requirement of a service carried by the optical service unit;

[0069] In this embodiment, the optical network receives various types of services, and the various types of services are mapped to optical service units according to the arrival time. The optical service units can carry services of various bandwidth granularities. Among them, according to the service type, the carried services can be divided into real-time services with high real-time requirements and load services with high resource occupation requirements. Different types of services have different computing power requirements for optical service units.

[0070] In some methods, the load parameter of the optical service unit is determined according to the proportion of the computing power demand of the service in the optical service unit, and the calculation method is:

[0071]

[0072] Among them, L(s i ) is the optical service unit s i Load parameter, Fsij For business j in optical business unit s i The computing power required on the network is M, M is the number of services, and N is the number of optical service units.

[0073] In some methods, the computing power demand of the service on the optical service unit is the computing resources required to execute the service in the optical service unit, and the computing resources include CPU resources, storage resources, bandwidth resources, power resources, delay requirements, etc.

[0074] S102: Calculate a delay parameter of the optical service unit according to a delay sensitivity parameter of a service carried by the optical service unit;

[0075] In this embodiment, considering that different services have different requirements for real-time performance, and because the network environment in different areas may be different, the delay reflected by the service in different areas is different, the delay parameter of the optical service unit is obtained by weighted summation of the delay sensitivity parameter of the service and the delay of processing the corresponding service, and the calculation method is:

[0076]

[0077] Among them, D(s i ) is the optical service unit s i The delay parameter, S j is the delay sensitivity parameter of service j. The delay sensitivity parameter is a weight determined according to the relative importance of service j and can be preset. For example, the key service has a larger weight, D ij Optical Business Units i The delay in processing or transmitting service j.

[0078] S103: Determine a replication dynamic equation of the optical service unit under the current network state based on the load parameter and the delay parameter;

[0079] In this embodiment, after the load parameters and delay parameters of the optical service unit are calculated, the load state and delay state of the optical service unit under the current network state are determined. Based on the evolutionary game model and reinforcement learning method, under the current network state, for the contradiction between overall load balancing and local minimum delay, a stable optimization strategy is sought by continuously adjusting the strategy and feedback of the network state.

[0080] Wherein, according to the load parameter and the delay parameter, the replication dynamic equation under the current network state is determined, including:

[0081] According to the load parameters, the load balancing benefit is determined according to the preset load benefit function;

[0082] Determine the delay benefit according to the delay parameter and the preset delay benefit function;

[0083] According to the load balancing benefit and the delay benefit, the replication dynamic equation of the optical service unit under the current network status is determined.

[0084] In this embodiment, the two goals of delay minimization and load balancing are mainly considered. To achieve the load balancing goal, a load benefit function is constructed as follows:

[0085] U l (s i )=-Mean(L(s i )) / S (3)

[0086] Among them, L(s i ) is the optical service unit s i load parameter; S is a normalization factor used to standardize the calculation of the load balancing benefit of the optical service unit so that the benefits of different nodes are comparable; Mean() represents a mean function used to calculate the average value.

[0087] In order to achieve the goal of minimizing delay, the delay benefit function is constructed as follows:

[0088] U d (s i )=-D(s i ) (4)

[0089] According to the constructed load-benefit function and delay-benefit function, the more balanced the load, the greater the load benefit, and the smaller the delay, the greater the delay benefit.

[0090] Among them, the optical service unit takes delay minimization as its goal, and the probability P of selecting the delay optimization strategy is d for:

[0091]

[0092] The optical service unit takes load balancing optimization as its goal and selects the probability P of a load balancing strategy. l for:

[0093]

[0094] Among them, α and β are adjustment parameters used to control the impact of delay and load on the selection probability.

[0095] Moreover, the probability of the optical service unit selecting the delay optimization strategy and the probability of selecting the load balancing strategy satisfy:

[0096] P d +P l =1 (7)

[0097] In the evolutionary game model, the replication dynamic equation is used to describe the evolution of strategies in the optical business unit group. Let χ drepresents the delay weight parameter of the optical service unit selecting the delay optimization strategy to schedule the service, χ l The load weight parameter representing the load balancing strategy used by the optical service unit to schedule services is constructed as follows:

[0098]

[0099] in, is the average return, calculated as:

[0100]

[0101] Among them, U d (s d ) is the benefit obtained by executing the delay optimization strategy, U l (s l ) is the benefit of executing the load balancing strategy.

[0102] To ensure the stability of the system, calculate the Jacobian matrix of the system:

[0103]

[0104] The calculation results are:

[0105]

[0106] The stable points obtained by solving the replica dynamic equations correspond to Should meet and At the stable point, the real parts of the eigenvalues ​​of the Jacobian matrix are all negative, indicating that the system is stable.

[0107] By solving the replicated dynamic equation, we can obtain the stable point of the equation. That is, the delay weight parameter of the optical service unit scheduling the service according to the delay optimization strategy is obtained and load weight parameters for scheduling services according to load balancing strategies Therefore, by performing service scheduling according to the two strategies and the corresponding delay weight parameters and load weight parameters, the needs of delay-related services and load-related services can be met, system stability can be ensured, and resource utilization can be improved.

[0108] S104: Solving the replication dynamic equation based on the Q learning method, determining a delay weight parameter for scheduling services according to the delay optimization strategy, and a load weight parameter for scheduling services according to the load balancing strategy;

[0109] In this embodiment, in the process of solving the replication dynamic equation, the Q learning method is used to determine the optimal strategy for scheduling services of the optical service unit through continuous interaction with the environment, that is, to select the optimal delay weight parameter of the delay optimization strategy and the optimal load weight parameter of the load balancing strategy.

[0110] In some implementations, solving the replication dynamic equation based on a Q-learning method includes:

[0111] Taking the current network state of the optical service unit as the current state, selecting a delay optimization strategy or a load balancing strategy as an action, and calculating a reward value according to a preset reward function after executing the selected strategy;

[0112] According to the reward value, the Q value is updated according to the preset Q value update method until the maximum Q value is obtained, and the χ value corresponding to the maximum Q value is d , χ l As a stable point for replicating the dynamic equations.

[0113] In this embodiment, the Q learning method is used to solve the stable point of the replication dynamic equation, that is, to determine the optimal weight ratio of the delay optimization strategy and the load balancing strategy selected by the optical service unit. In the Q learning process, the current load state and delay state of the optical service unit are taken as the current state, and the delay optimization strategy or the load balancing strategy is selected as the action. After the action is performed in the current state, the reward value is calculated according to the state after the action is performed, and the Q value is updated according to the reward value. The above process is executed until the maximum Q value is obtained. The weight parameter corresponding to the maximum Q value is the stable point at which the system reaches a stable state. Thus, the optimal weight ratio of the optical service unit execution strategy is determined through the Q learning method.

[0114] Among them, the reward function is constructed according to the load benefit and delay benefit of the optical service unit, which is expressed as:

[0115] R = x d ×U d (s i )+χ l ×U l (s i ) (13)

[0116] In the Q learning process, the optimal strategy is learned by updating the Q value. The Q value update method is:

[0117]

[0118] When the maximum Q value is calculated, the corresponding reward function R d , χ l Stable point It is expressed as:

[0119]

[0120] Where η is the learning rate, γ is the discount factor, R is the reward value obtained by executing action a in state s, Q(s,a) is the Q value obtained by executing action a in state s, s′ is the new state after executing action a, and a′ is the new action executed in the new state. It is the maximum Q value obtained from all possible new actions a′ in the new state s′, that is, the expected benefit of selecting the best action in the new state.

[0121] S105: Scheduling services according to the delay optimization strategy and the corresponding delay weight parameters, and according to the load balancing strategy and the corresponding load weight parameters.

[0122] In this embodiment, after determining the delay weight parameter of the delay optimization strategy and the load weight parameter of the load balancing strategy executed by the optical service unit based on the evolutionary game model and the Q learning method, the optical service unit schedules the various services it carries according to the delay optimization strategy and the corresponding delay weight parameter, according to the load balancing strategy and the corresponding load weight parameter. In some methods, for various services carried by the optical service unit, the optical service unit executes the delay optimization strategy and the load balancing strategy in combination with the service type, the delay weight parameter, and the load weight parameter. For example, the delay weight parameter is determined to be 0.4 and the load weight parameter is 0.6. For real-time services, the delay optimization strategy is executed according to the weight of 0.4, that is, 40% of the decision weight is allocated to the delay optimization strategy. For load-type services, the load balancing strategy is executed according to the weight of 0.6, that is, 60% of the decision weight is allocated to the load balancing strategy. It should be noted that the specific scheduling method of the optical service unit for various services needs to be comprehensively determined based on multiple aspects such as the service type, service importance, service demand, resource conditions, and determined weights, which will not be described in detail in this embodiment.

[0123] In some ways, the delay optimization strategy is to reduce the time delay of data in the transmission process through various methods such as path planning and caching technology, ensure that data is transmitted as quickly as possible in the network, and reduce the waiting time of the service; the load balancing strategy is to balance the load of each node by reasonably distributing the workload among multiple servers or resource nodes to avoid overloading a single node. The execution of the strategy includes but is not limited to path planning, computing resource allocation, etc. This embodiment does not explain and list the specific delay optimization strategy and load balancing strategy in detail.

[0124] In some methods, considering that the network status is constantly changing, a policy update cycle is set. In each policy update cycle, the delay weight parameters of the delay optimization strategy and the load weight parameters of the load balancing strategy executed by the optical service unit are determined according to the method of the present application, and the service scheduling is performed according to the corresponding strategy and the corresponding weight parameters, so as to maintain the optimal resource allocation and system performance through dynamic adjustment to adapt to the network status. In other methods, when a resource bottleneck occurs in the optical service unit, the delay weight parameters of the delay optimization strategy and the load weight parameters of the load balancing strategy executed by the optical service unit are re-determined according to the method of the present application, and the service scheduling is performed according to the corresponding strategy and the updated weight parameters.

[0125] like Figure 2 As shown, in some application scenarios, various resource nodes in the optical network are deployed in different areas, for example, nodes 3, 4, 5, and 8 are deployed in metropolitan area A, and nodes 9, 10, and 11 are deployed in metropolitan area C, etc. According to actual application requirements, service scheduling can be performed by region, or service scheduling can be performed for the entire optical network system. When scheduling according to the region, the method of the present application is used to determine the optimal weight parameters for each optical service unit in the target area to execute the delay optimization strategy and load balancing strategy under the current network state, and perform service scheduling according to the corresponding strategy and the corresponding optimal weight parameters. By performing fine-grained service scheduling in the target area, the overall load balance and the optimal local delay in the target area are achieved. The optimized network state. When scheduling for the entire system, the optimal weight parameters for all optical service units in the system to execute the delay optimization strategy and load balancing strategy under the current network state are determined, and service scheduling is performed according to the corresponding strategy and the corresponding optimal weight parameters. By performing fine-grained service scheduling in the entire system, the overall load balance and the optimal local delay in the entire system are achieved. The optimized network state.

[0126] The fine-grained optical network service scheduling method provided in the embodiment of the present application determines the delay weight parameters of the delay optimization strategy and the load weight parameters of the load balancing strategy for the optical service unit to execute, for the services of different granularities carried by the optical service unit, in combination with the service requirements for resources and quality, based on the evolutionary game model and the reinforcement learning method, and performs service scheduling according to the delay optimization strategy and the corresponding delay weight parameters, the load balancing strategy and the corresponding load weight parameters, so as to realize fine-grained service scheduling, meet the requirements of different types of services, optimize system resource configuration, and improve system performance.

[0127] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0128] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0129] like Figure 3 As shown, the embodiment of the present application provides a fine-grained optical network service scheduling device, including:

[0130] A load calculation module, used to calculate the load parameters of the optical service unit according to the computing power requirements of the services carried by the optical service unit;

[0131] A delay calculation module, used to calculate the delay parameter of the optical service unit according to the delay sensitivity parameter of the service carried by the optical service unit;

[0132] An equation determination module, used to determine the replication dynamic equation of the optical service unit under the current network state based on the load parameter and the delay parameter;

[0133] A solution module, used to solve the replication dynamic equation based on a Q learning method, determine a delay weight parameter for scheduling services according to a delay optimization strategy, and a load weight parameter for scheduling services according to a load balancing strategy;

[0134] The scheduling module is used to schedule services according to the delay optimization strategy and the corresponding delay weight parameters, and according to the load balancing strategy and the corresponding load weight parameters.

[0135] For the convenience of description, the above devices are described in terms of functions divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0136] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0137] Figure 4A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0138] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0139] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0140] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0141] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0142] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0143] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0144] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0145] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0146] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0147] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0148] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0149] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present disclosure.

Claims

1. A fine-grained optical network service scheduling method, characterized in that: include: Calculate the load parameters of the optical service unit according to the computing power requirements of the services carried by the optical service unit; Calculating the delay parameter of the optical service unit according to the delay sensitivity parameter of the service carried by the optical service unit; Based on the load parameter and the delay parameter, determining a replication dynamic equation of the optical service unit under the current network state; Solving the replication dynamic equation based on the Q learning method, determining the delay weight parameter for scheduling services according to the delay optimization strategy, and the load weight parameter for scheduling services according to the load balancing strategy; Services are scheduled according to the delay optimization strategy and the corresponding delay weight parameters, and according to the load balancing strategy and the corresponding load weight parameters.

2. The method according to claim 1, characterized in that Based on the load parameter and the delay parameter, determining the replication dynamic equation of the optical service unit in the current network state includes: Determine the load balancing benefit according to the load parameter and a preset load benefit function; Determining the delay benefit according to the delay parameter and a preset delay benefit function; The replication dynamic equation is determined according to the load balancing benefit and the delay benefit.

3. The method according to claim 2, characterized in that The load benefit function is: U l (s i )=-Mean(L(s i )) / S (3) Among them, L(s i ) is the optical service unit s i The load parameter, S is the normalization factor, and Mean() is the mean function; The delay benefit function is: U d (s i )=-D(s i ) (4) Among them, D(s i ) is the optical service unit s i Delay parameters; The replication dynamic equation is: Among them, χd and χl corresponding to the stable point of the replicated dynamic equation are the delay weight parameter and the load weight parameter respectively, is the average return, calculated as: Among them, U d (s d ) is the benefit obtained by executing the delay optimization strategy, U l (s l ) is the benefit of executing the load balancing strategy.

4. The method according to claim 3, characterized in that Solving the replication dynamic equation based on the Q-learning method includes: Taking the current network state as the current state, selecting the delay optimization strategy or the load balancing strategy as an action, and calculating the reward value according to a preset reward function after executing the selected strategy; According to the reward value, the Q value is updated according to the preset Q value update method until the maximum Q value is obtained, and the χd and χl corresponding to the maximum Q value are used as the stable points of the replication dynamic equation.

5. The method according to claim 4, characterized in that The reward function is: R=χd×U d (s i )+χl×U l (s i )(13) The Q value updating method is: Where η is the learning rate, γ is the discount factor, R is the reward value obtained by executing action a in state s, Q(s, a) is the Q value obtained by executing action a in state s, s′ is the new state after executing action a, and a′′ is the new action executed in the new state. is the maximum Q value obtained by executing action a′ in the new state s′.

6. A fine-grained optical network service scheduling device, characterized in that: include: A load calculation module, used to calculate the load parameters of the optical service unit according to the computing power requirements of the services carried by the optical service unit; A delay calculation module, used to calculate the delay parameter of the optical service unit according to the delay sensitivity parameter of the service carried by the optical service unit; An equation determination module, used to determine the replication dynamic equation of the optical service unit under the current network state based on the load parameter and the delay parameter; A solution module, used to solve the replication dynamic equation based on a Q learning method, determine a delay weight parameter for scheduling services according to a delay optimization strategy, and a load weight parameter for scheduling services according to a load balancing strategy; The scheduling module is used to schedule services according to the delay optimization strategy and the corresponding delay weight parameters, and according to the load balancing strategy and the corresponding load weight parameters.

7. The device according to claim 6, characterized in that The equation determination module is used to determine the load balancing benefit according to the load parameters and the preset load benefit function; determine the delay benefit according to the delay parameters and the preset delay benefit function; and determine the replication dynamic equation according to the load balancing benefit and the delay benefit.

8. The device according to claim 7, characterized in that The load benefit function is: U l (s i )=-Mean(L(s i )) / S (3) Among them, L(s i ) is the optical service unit s i The load parameter, S is the normalization factor, and Mean() is the mean function: The delay benefit function is: U d (s i )=-D(s i ) (4) Among them, D(s i ) is the optical service unit s i Delay parameters; The replication dynamic equation is: Among them, χd and χl corresponding to the stable point of the replicated dynamic equation are the delay weight parameter and the load weight parameter respectively, is the average return, calculated as: Among them, U d (s d ) is the benefit obtained by executing the delay optimization strategy, U l (s l ) is the benefit of executing the load balancing strategy.

9. The device according to claim 8, characterized in that The solution module is used to take the current network state as the current state, select the delay optimization strategy or the load balancing strategy as an action, and calculate the reward value according to a preset reward function after executing the selected strategy; According to the reward value, the Q value is updated according to the preset Q value update method until the maximum Q value is obtained, and the χd and χl corresponding to the maximum Q value are used as the stable points of the replication dynamic equation.

10. The device according to claim 9, characterized in that The reward function is: R=χd×U d (s i )+χl×U l (s i )(13) The Q value updating method is: Where η is the learning rate, γ is the discount factor, R is the reward value obtained by executing action a in state s, Q(s, a) is the Q value obtained by executing action a in state s, s′ is the new state after executing action a, and a′ is the action performed in the new state. is the maximum Q value obtained by executing action a′ in the new state s′.

Images