DWDM (Discrete Wavelength Division Multiplexing) multi-target dynamic optimization method and system based on reinforcement learning and hybrid model

Abstract

The invention relates to the field of dense wavelength division multiplexing networks, and discloses a DWDM multi-target dynamic optimization method and system based on reinforcement learning and a hybrid model, and the method comprises the steps: carrying out the distributed collection of the real-time operation state of a DWDM link, and generating a dynamic request set with a hierarchical identifier according to the time delay sensitivity and the bandwidth demand degree; performing time sequence dislocation analysis on the dynamic request set, performing parallel sliding observation under different time scales, and generating a composite feature set describing link dynamics; based on the composite feature set, using a hybrid reinforcement learning model to perform iterative deduction on the business evolution process; defining a wavelength interval corresponding to the blocking risk fragment and the associated optical node as a to-be-optimized domain, and constructing a wavelength conflict relation graph with an associated weight; and based on the wavelength conflict relation graph, carrying out adaptive adjustment on a reward function structure and a decision step length of the hybrid reinforcement learning model. The method has the advantage of improving the running efficiency and service quality of the DWDM network.

Description

Technical Field

[0001] This invention relates to the field of dense wavelength division multiplexing (DWDM) networks, specifically to a multi-objective dynamic optimization method and system for DWDM based on reinforcement learning and a hybrid model. Background Technology

[0002] Dense Wavelength Division Multiplexing (DWDM) networks typically allocate wavelength resources using static or semi-static optimization methods to improve bandwidth utilization and ensure service quality. However, in actual operation, especially in scenarios with bursty and highly dynamic traffic such as data center interconnects or 5G bearer networks, traditional optimization methods are prone to wavelength resource fragmentation. Specifically, when multiple short-duration bursty services arrive at the network simultaneously, although the overall available bandwidth remains sufficient, discontinuous wavelength allocation prevents some services from establishing optical channels in a timely manner, leading to decreased resource utilization and increased service latency. Existing technologies lack optimization strategies capable of responding to dynamic bursty services at the millisecond level while simultaneously considering multiple objectives, making it difficult to fully guarantee network performance under high-frequency bursty loads. Therefore, it is essential to design a DWDM multi-objective dynamic optimization method and system based on reinforcement learning and hybrid models to improve the operational efficiency and service quality of DWDM networks. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a multi-objective dynamic optimization method and system for DWDM based on reinforcement learning and hybrid models, which has the advantages of improving the operating efficiency and service quality of DWDM networks and solves the problems mentioned in the background technology.

[0004] To achieve the aforementioned goals of improving the operational efficiency and service quality of DWDM networks, this invention provides the following technical solution: a multi-objective dynamic optimization method for DWDM based on reinforcement learning and hybrid models, comprising the following steps:

[0005] The real-time operating status of DWDM links is collected in a distributed manner, and the service arrival sequence is segmented in multiple dimensions by means of a burst traffic identification network. Dynamic request sets with hierarchical identifiers are generated according to latency sensitivity and bandwidth requirements.

[0006] Perform timing misalignment analysis on the dynamic request set, conduct parallel sliding observations at different time scales, and combine optical power offset trajectory, link attenuation mode and wavelength utilization mapping to generate a composite feature set describing the link dynamics.

[0007] Based on a composite feature set, a hybrid reinforcement learning model is used to iteratively extrapolate the business evolution process. By capturing abrupt changes in allocation density and resource utilization entropy, potential blocking risk segments are screened out.

[0008] The wavelength range and associated optical nodes corresponding to the blocking risk segment are defined as the domain to be optimized, and the optical amplification unit, modulation control unit and monitoring unit in the domain to be optimized are integrated to construct a wavelength conflict relationship diagram with correlation weights.

[0009] Based on the wavelength conflict relationship diagram, combined with the energy efficiency fluctuation trend of the entire network and the link robustness index, the reward function structure and decision step size of the hybrid reinforcement learning model are adaptively adjusted, and dynamic scheduling instructions for DWDM resource allocation are output.

[0010] Preferably, the process of generating a dynamic request set with hierarchical identifiers based on latency sensitivity and bandwidth requirements is as follows:

[0011] Spatiotemporal sampling is performed on the service arrival sequence obtained from real-time link monitoring to extract the arrival rate, duration, and required bandwidth of the requests;

[0012] Map the latency tolerance threshold of the request to the target bandwidth requirement to the hierarchical rule table to generate different priority labels;

[0013] Request sets under the same priority label are hierarchically identified to form dynamic request sets with timestamps and priority codes.

[0014] Preferably, the process of performing timing misalignment analysis on the dynamic request set is as follows:

[0015] The dynamic request set is divided into high-sensitivity requests and low-sensitivity requests according to priority tags;

[0016] Short-duration sliding windows are assigned to highly sensitive requests, while long-duration sliding windows are assigned to low-sensitivity requests, forming a multi-scale sliding window set;

[0017] Synchronously read dynamic request sets within a multi-scale sliding window, and eliminate cross-domain observation errors through timestamp alignment and link phase correction methods;

[0018] Correlation matching is performed on the observation results of adjacent sliding windows to obtain the observation vector set after temporal misalignment.

[0019] Preferably, the process of generating a composite feature set describing the dynamics of the link is as follows:

[0020] Power offset curves were constructed using optical power sampling data, and short-term oscillation and long-term drift characteristics were extracted.

[0021] Exponential smoothing is performed on the link loss data to obtain a decay mode vector that reflects link stability;

[0022] A distribution matrix is ​​constructed from wavelength utilization data to quantify the sparsity and continuity of resource usage;

[0023] The power offset feature, attenuation mode vector, and wavelength utilization matrix are uniformly mapped to a composite feature set through a feature embedding method.

[0024] Preferably, the process of iteratively extrapolating the business evolution using a hybrid reinforcement learning model is as follows:

[0025] The composite feature set is input into a hybrid model that combines deep Q-learning and policy gradient;

[0026] By using serialized state transition modeling, the evolution trajectory of the business under different resource allocations is iteratively simulated, and the exploration rate and learning rate are dynamically adjusted during the simulation process.

[0027] Preferably, the process of screening out segments with potential blocking risks is as follows:

[0028] Calculate the allocation density curve in the business evolution trajectory and identify local abrupt changes;

[0029] The changes in the entropy of statistical resource utilization are measured, and the locations of abrupt change points are detected.

[0030] If the density jump point and the entropy mutation point overlap in time, the corresponding segment is marked as a blocking risk segment, and the blocking risk segment with delay label and resource label is output.

[0031] Preferably, the process of defining the wavelength range and associated optical nodes corresponding to the blocking risk segment as the domain to be optimized is as follows:

[0032] Range calibration is performed on the wavelength range involved in the blocking risk segment;

[0033] Search for upstream and downstream optical nodes connected within the wavelength range to form a set of nodes to be optimized;

[0034] The wavelength range and the set of nodes to be optimized are jointly defined as the optimization domain.

[0035] Preferably, the process of constructing a wavelength conflict relationship graph with correlation weights is as follows:

[0036] Within the domain to be optimized, optical amplification units, modulation and control units, and monitoring units are extracted and mapped to a set of nodes.

[0037] Establish edge connections based on wavelength competition between nodes, power regulation effects, and historical link status;

[0038] Each edge is assigned a conflict intensity weight using an edge weight calculation function;

[0039] The set of nodes and weighted edges are combined to form a wavelength conflict relationship graph.

[0040] Preferably, the process of outputting dynamic scheduling instructions for DWDM resource allocation is as follows:

[0041] By integrating the wavelength conflict relationship diagram with the overall network energy efficiency fluctuation trend, key conflict paths and risk accumulation values ​​are extracted.

[0042] By combining link robustness metrics, the reward function of the reinforcement learning model is weighted and the decision step size is reset, and dynamic scheduling instructions are generated based on the dynamically optimized model.

[0043] A multi-objective dynamic optimization system based on reinforcement learning and hybrid models for DWDM includes:

[0044] Request collection module: Distributed collection of real-time operating status of DWDM links, and generation of dynamic request sets with hierarchical identifiers by combining burst traffic identification network;

[0045] Misalignment Analysis Module: Performs temporal misalignment analysis on dynamic request sets and generates a composite feature set describing link dynamics through multi-scale sliding observations;

[0046] Inference and screening module: Based on the composite feature set, it uses a hybrid reinforcement learning model to infer the business evolution process and screen potential blocking risk segments;

[0047] Domain graph construction module: Maps blocking risk segments to wavelength ranges and optical nodes, and constructs a wavelength conflict relationship graph with associated weights;

[0048] Adaptive scheduling module: Based on wavelength conflict relationship graph and network-wide robustness index, it adaptively adjusts reinforcement learning model parameters and outputs dynamic scheduling instructions.

[0049] Compared with existing technologies, this invention provides a multi-objective dynamic optimization method and system for DWDM based on reinforcement learning and hybrid models, which has the following beneficial effects:

[0050] This invention combines reinforcement learning with a hybrid model to achieve dynamic optimization management of multiple objectives and constraints in DWDM networks, offering significant advantages over existing technologies. By distributing and collecting real-time link operating status and combining it with a burst traffic identification mechanism, this method can accurately capture the dynamic changing characteristics of service requests, prioritizing services with high latency sensitivity and high bandwidth requirements to form a hierarchical dynamic request set, thereby avoiding resource waste and link congestion problems caused by traditional static allocation. Through temporal misalignment analysis and multi-scale sliding window parallel observation, it can fully reflect the dynamic characteristics of links at different time scales, integrating multi-dimensional information such as optical power shift, link attenuation mode, and wavelength utilization into a composite feature set, providing comprehensive data support for subsequent reinforcement learning decisions and effectively improving the accuracy of service evolution trajectory prediction. Using a hybrid reinforcement learning model for iterative deduction, it can capture allocation density jump points and resource utilization entropy mutation points, achieving accurate identification of potential congestion risk segments and providing a scientific basis for subsequent optimization. By jointly defining the congestion risk segment with relevant wavelength ranges and optical nodes as the optimization domain, and constructing a wavelength conflict relationship graph with associated weights, the conflict intensity and coupling relationship between nodes can be clearly defined, providing a quantitative reference for resource scheduling. Based on the overall network energy efficiency fluctuation trend and link robustness indicators, the reward function and decision step size of the reinforcement learning model are adaptively adjusted to generate optimized dynamic scheduling instructions, achieving efficient allocation of wavelength resources, robust link operation, and improved overall network energy efficiency. This approach can significantly reduce link congestion risk, improve resource utilization, enhance network adaptive regulation capabilities, and provide a systematic solution for service assurance and network performance optimization under burst traffic conditions, possessing broad engineering application value. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the method of the present invention;

[0052] Figure 2 This is a schematic diagram of the structure of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example 1: Please refer to Figure 1 As shown in the embodiment of the present invention, the DWDM multi-objective dynamic optimization method based on reinforcement learning and hybrid models includes the following steps:

[0055] S1: Distributed collection of real-time operating status of DWDM links, and multi-dimensional segmentation of service arrival sequences using burst traffic identification network, generating dynamic request sets with hierarchical identifiers according to latency sensitivity and bandwidth requirements.

[0056] The process in S1 of generating a dynamic request set with hierarchical identifiers according to latency sensitivity and bandwidth requirement is as follows:

[0057] Spatiotemporal sampling is performed on the service arrival sequence obtained from real-time link monitoring to extract the arrival rate, duration, and required bandwidth of the requests; real-time transmission data of each fiber link in the DWDM network is periodically sampled to capture the arrival time, duration of link occupation, and required transmission bandwidth of each service request, forming a raw sampled data set that can describe the dynamic characteristics of the service; during the sampling process, the link topology and utilization of different wavelength channels are considered, and the sampling results are arranged in chronological order.

[0058] The latency tolerance threshold of the request is mapped to the target bandwidth requirement and then to the hierarchical rule table to generate different priority labels. Based on the service type, latency sensitivity and bandwidth requirement of each service request, it is compared and mapped with the pre-designed hierarchical rule table. The hierarchical rule table sets the latency tolerance range and bandwidth priority for different levels of services. The level that the request meets is calculated based on the actual indicators of the request and the corresponding priority label is generated, so that the request can be clearly marked as a high, medium and low level of processing priority.

[0059] Request sets under the same priority label are hierarchically identified to form a dynamic request set with timestamps and priority codes. Under the same priority label, all business requests are sorted according to arrival time, and timestamp information is attached to each request to preserve the temporal characteristics of the requests. The request set is internally hierarchically identified, and internal sequence numbers or hierarchical codes are assigned to requests based on link resource occupancy and potential conflict risks, thereby forming a dynamic request set that contains both priority information and preserves time and hierarchical order.

[0060] S2: Perform timing misalignment analysis on the dynamic request set, conduct parallel sliding observations at different time scales, and combine optical power offset trajectory, link attenuation mode and wavelength utilization mapping to generate a composite feature set describing the dynamics of the link.

[0061] The process of performing timing misalignment analysis on the dynamic request set in S2 is as follows:

[0062] The dynamic request set is divided into high-sensitivity requests and low-sensitivity requests according to priority tags. Each request in the dynamic request set is traversed, and its attached priority tag information is read. Requests that are highly sensitive to latency or have high bandwidth requirements are divided into high-sensitivity request groups, and requests that have a large tolerance for latency or low bandwidth usage are divided into low-sensitivity request groups. In the process of division, the current load status of the link and the historical business fluctuation characteristics are also considered to ensure that the request division can accurately reflect its actual needs for resource allocation urgency.

[0063] Short-term sliding windows are assigned to high-sensitivity requests, and long-term sliding windows are assigned to low-sensitivity requests, forming a multi-scale sliding window set. Based on the priority of the divided requests, short-term observation sliding windows are designed for high-sensitivity requests to capture details of sudden traffic changes, while relatively long sliding windows are configured for low-sensitivity requests to smooth out overall business fluctuations. The sliding windows are arranged continuously on the time axis, and the start and end times of each sliding window are recorded to form a multi-scale sliding window set, so that network state changes can be observed simultaneously at different time resolutions.

[0064] The dynamic request set is synchronously read within a multi-scale sliding window, and cross-domain observation errors are eliminated by timestamp alignment and link phase correction. Within each sliding window time range, data from high-sensitivity and low-sensitivity requests are extracted synchronously, and the timestamps of each request are uniformly aligned to eliminate time differences caused by acquisition delays of different links or node clock deviations. The observation data is corrected by combining link phase information to correct timing deviations caused by optical signal transmission delays or wavelength drift, so as to ensure that the observation data of all requests within the sliding window can form a consistent and comparable time series.

[0065] Correlation matching is performed on the observation results of adjacent sliding windows to obtain the observation vector set after time-series misalignment; the synchronously corrected request data in the sliding window is matched with the data of the previous sliding window one by one, and the continuity and abrupt change points of the time series are identified by analyzing the arrival order, duration and bandwidth usage trend of the requests; the final observation vector set is generated, in which each vector contains the misaligned request characteristics, timestamp and priority information, which can truly reflect the evolution of link load and business dynamics at multiple time scales.

[0066] The process of generating the composite feature set describing the dynamics of the link in S2 is as follows:

[0067] Power offset curves are constructed using optical power sampling data, and short-term oscillation and long-term drift characteristics are extracted. Optical power sampling data are obtained from real-time monitoring of each DWDM link, and the power changes of each wavelength channel are recorded in time series. By analyzing the increase and decrease amplitude and fluctuation period between consecutive sampling points, short-term oscillation characteristics are extracted to reflect the impact of sudden traffic or transient fluctuations on link power. At the same time, the power change trend over a longer period is smoothed to extract long-term drift characteristics to reflect the overall stability of the link and the slow power attenuation trend, forming a power offset curve feature set that can describe the dynamics of the link.

[0068] Exponential smoothing is performed on the link loss data to obtain an attenuation mode vector reflecting link stability; loss measurement data of each link in different wavelength channels are collected, and the time series data is subjected to exponential weighted smoothing to reduce the impact of instantaneous fluctuations on the overall trend; the link attenuation mode is extracted from the smoothed loss data to reflect the stability of the link under continuous service load, including attenuation rate, periodic fluctuations and abnormal jitter, and it is organized into a vector form.

[0069] A distribution matrix is ​​constructed from wavelength utilization data to quantify the sparsity and continuity of resource occupancy. The resource usage of each wavelength channel on the link is sampled, and the occupancy and idle status of each wavelength are statistically analyzed within a specified time period. The occupancy status of each wavelength is arranged in chronological order to construct a two-dimensional distribution matrix, where rows represent time sampling points and columns represent wavelength channels. The sparsity analysis of the matrix quantifies the concentration of resource distribution, and the continuity analysis identifies the smoothness and abrupt changes in resource utilization, thereby reflecting the dynamic resource status of the network under multiple wavelengths and multiple services.

[0070] Power offset features, attenuation mode vectors, and wavelength utilization matrices are uniformly mapped to a composite feature set using a feature embedding method. The three independent feature sets are then processed in a unified manner, and the power offset curve, attenuation mode vector, and wavelength utilization matrix are mapped to the same multidimensional feature space using a feature embedding method. During the embedding process, the relative relationships and time dependencies between the features are preserved, enabling data from different sources to be interactively analyzed and compared in a unified space. Finally, a composite feature set containing power dynamics, link stability, and resource utilization is generated.

[0071] S3: Based on a composite feature set, a hybrid reinforcement learning model is used to iteratively extrapolate the business evolution process. By capturing abrupt changes in allocation density and resource utilization entropy, potential blocking risk segments are screened out.

[0072] The process of iteratively extrapolating the business evolution process using a hybrid reinforcement learning model in S3 is as follows:

[0073] The composite feature set is input into a hybrid model combining deep Q-learning and policy gradient; the generated composite feature set is serialized according to time order and service priority, and used as state input to the hybrid reinforcement learning model; the model is jointly composed of a deep Q-learning module and a policy gradient module. The deep Q-learning module is used to evaluate the immediate reward value of different resource allocation schemes, and the policy gradient module is used to optimize long-term policies, so as to comprehensively evaluate and predict the resource allocation strategy of DWDM services considering the current network state and future evolution trends;

[0074] By using serialized state transition modeling, the evolution trajectory of the service under different resource allocations is iteratively simulated, and the exploration rate and learning rate are dynamically adjusted during the simulation. During the model operation, the state at each time step, the resource allocation actions taken, and the observed system response are formed into a state transition sequence. These sequences are used for iterative simulation and deduction to predict the evolution trajectory of the service under different resource scheduling strategies. During the iteration process, the exploration rate is dynamically adjusted based on the model's feedback on prediction errors and reward fluctuations to balance the exploration of new strategies with the utilization of historical strategies. The learning rate is also adaptively adjusted based on the learning progress and convergence to ensure that the model can stably and efficiently learn the optimal scheduling strategy under different service loads and link states.

[0075] The process of filtering out potential blocking risk segments in S3 is as follows:

[0076] Calculate the allocation density curve in the service evolution trajectory and identify local abrupt changes; for each service evolution trajectory derived from the hybrid reinforcement learning model, generate the corresponding allocation density curve based on the amount of optical wavelength resources and link bandwidth occupied at different time steps; analyze the continuous changes of the curve on the time axis, identify abrupt changes and steep increase sections. These local abrupt changes indicate that the link resources are concentrated or overloaded in a short period of time, which can reflect potential resource bottlenecks.

[0077] The system calculates the magnitude of changes in entropy in resource utilization and detects the location of abrupt change points. For the link resource utilization status within each sliding window or time period, it calculates the entropy change in resource allocation to quantify the balance and uncertainty of resource occupancy. By comparing the magnitude of entropy changes at consecutive time points, it identifies the location of abrupt change points in entropy. These abrupt change points correspond to moments when the resource utilization pattern in the link suddenly changes or fluctuations intensify, and can indicate potential load shocks or congestion trends.

[0078] If the allocation density jump point and the entropy change point overlap in time sequence, the corresponding segment is marked as a blocking risk segment, and the blocking risk segment with latency label and resource tag is output. The local jump points identified in the allocation density curve and the entropy change change points are compared on the time axis to find overlapping segments that occur simultaneously or are highly similar. The service request segments corresponding to these overlapping segments are marked as blocking risk segments, and their start and end times, related link resource occupancy information and service latency sensitivity labels are attached to each segment. Finally, a set of blocking risk segments with complete latency label and resource tag is output.

[0079] S4: Define the wavelength range and associated optical nodes corresponding to the blocking risk segment as the domain to be optimized, and integrate the optical amplification unit, modulation control unit and monitoring unit in the domain to be optimized to construct a wavelength conflict relationship diagram with associated weights.

[0080] The process of defining the wavelength range and associated optical nodes corresponding to the blocking risk segment in S4 as the domain to be optimized is as follows:

[0081] The wavelength range involved in the blocking risk segment is calibrated; for each blocking risk segment identified, the wavelength channel it occupies is analyzed, and the starting and ending wavelength ranges involved in the segment are determined by reading the link wavelength usage and service resource occupancy information; during the calibration process, the physical proximity between wavelengths and the possibility of cross-wavelength interference are considered, and the relevant wavelengths are expanded or fine-tuned to ensure that the calibrated wavelength range can completely cover the area where the potential blocking risk occurs.

[0082] The upstream and downstream optical nodes connected by the wavelength range are retrieved to form a set of nodes to be optimized. According to the calibrated wavelength range, all upstream and downstream optical nodes traversed by the wavelength channel are retrieved along the fiber optic link topology map, including optical amplification units, wavelength cross nodes, optical splitting and multiplexing nodes, and access terminal nodes. At the same time, the current load status, available resources, and historical congestion records of these nodes are recorded to form a set of nodes to be optimized.

[0083] The wavelength range and the set of nodes to be optimized are jointly defined as the domain to be optimized. During the definition process, each domain to be optimized contains not only specific wavelength range information, but also the topological location and status attributes of the relevant nodes. This allows for overall optimization and coordinated control of the entire domain during subsequent wavelength scheduling, resource reallocation, or congestion resolution, ensuring that the operation can cover all critical resources involved in potential risk segments.

[0084] The process of constructing the wavelength conflict relationship graph with correlation weights in S4 is as follows:

[0085] Within the domain to be optimized, optical amplification units, modulation control units, and monitoring units are extracted and mapped to a set of nodes. In the defined domain to be optimized, the optical nodes traversed by the wavelength ranges involved are scanned in detail, with a focus on extracting optical amplification units, modulation control units, and various monitoring units. These physical devices are mapped to a set of logical nodes according to their functional attributes and locations. Each node contains device type, link location, current power status, and historical load information, so that the allocation of wavelength resources and potential conflict relationships between nodes can be fully represented in the subsequent graph structure.

[0086] Based on the wavelength competition relationship between nodes, the impact of power adjustment, and historical link status, edge connection relationships are established; the wavelength sharing situation among nodes in the domain to be optimized is analyzed to identify the resource competition that may occur between different nodes on the same wavelength channel; at the same time, the impact of optical power adjustment on downstream links is considered, including factors such as gain changes, signal attenuation, and interference between adjacent channels, and the possibility of conflict between nodes is assessed in combination with historical link status data; based on the analysis results, edge connection relationships are established between nodes, with each edge reflecting the possible conflict or coupling effect between two nodes at a specific wavelength;

[0087] Each edge is assigned a conflict intensity weight using an edge weight calculation function. For each established edge, the edge weight calculation function comprehensively quantifies the wavelength competition between nodes, the impact of power adjustment, and the frequency of historical link anomalies to calculate the conflict intensity weight. The edge weight not only considers the potential conflicts in the current state but also combines historical link performance and load fluctuations to conduct a forward-looking assessment of the resulting resource congestion and wavelength reuse interference, so that each edge can accurately reflect its conflict contribution in the entire optimization domain.

[0088] The set of nodes and weighted edges are combined to form a wavelength conflict relationship graph. The set of nodes and weighted edges are combined in a graph structure manner. In the graph, nodes represent physical devices or functional units, edges represent wavelength conflict relationships between nodes, and edge weights reflect the conflict intensity. The resulting wavelength conflict relationship graph can fully depict the potential resource competition and coupling relationships between nodes in the optimization domain, while taking into account the effects of physical topology, wavelength distribution, and power adjustment.

[0089] S5: Based on the wavelength conflict relationship diagram, combined with the energy efficiency fluctuation trend of the whole network and the link robustness index, the reward function structure and decision step size of the hybrid reinforcement learning model are adaptively adjusted, and dynamic scheduling instructions for DWDM resource allocation are output.

[0090] The process of outputting dynamic scheduling instructions for DWDM resource allocation in S5 is as follows:

[0091] By integrating the wavelength conflict relationship diagram with the overall network energy efficiency fluctuation trend, key conflict paths and risk accumulation values ​​are extracted. The generated wavelength conflict relationship diagram is then multidimensionally integrated with the energy efficiency fluctuation data of each link in the network, and the node and edge information in the diagram is matched with indicators such as link power consumption, signal gain changes, and network load fluctuations. By analyzing the conflict weights and energy efficiency fluctuation amplitudes between nodes in the diagram, the key paths most prone to congestion or resource conflicts under the current link state are identified. At the same time, the conflict intensity and potential congestion risk of these paths are cumulatively calculated to form the risk accumulation value of each path, which is used to quantify the potential impact of each conflict path on the overall network resource allocation efficiency.

[0092] By combining link robustness metrics, the reward function of the reinforcement learning model is weighted and the decision step size is reset. Dynamic scheduling instructions are generated based on the dynamically optimized model. After obtaining the critical conflict paths and risk accumulation values, the reward function of the hybrid reinforcement learning model is dynamically weighted with reference to link robustness metrics such as available bandwidth fluctuation, optical signal bit error rate, and link redundancy. This allows the model to balance network-wide energy efficiency optimization and link stability in resource allocation decisions. Simultaneously, based on the model's prediction of risk accumulation values ​​and the contribution of each path to overall network performance, the decision step size and action exploration range are reset to ensure that the scheduling strategy can both quickly respond to burst traffic and smoothly adjust wavelength resource allocation. Finally, based on the adjusted model, detailed dynamic scheduling instructions are output, including the allocation scheme of each wavelength channel, the operation sequence of relevant optical nodes, and the power adjustment parameters of the control unit, achieving efficient, robust, and dynamic management of DWDM network resources.

[0093] Example 2: As Figure 2 As shown, the DWDM multi-objective dynamic optimization system based on reinforcement learning and a hybrid model includes:

[0094] Request collection module: Distributed collection of real-time operating status of DWDM links, and generation of dynamic request sets with hierarchical identifiers by combining burst traffic identification network;

[0095] Misalignment Analysis Module: Performs temporal misalignment analysis on dynamic request sets and generates a composite feature set describing link dynamics through multi-scale sliding observations;

[0096] Inference and screening module: Based on the composite feature set, it uses a hybrid reinforcement learning model to infer the business evolution process and screen potential blocking risk segments;

[0097] Domain graph construction module: Maps blocking risk segments to wavelength ranges and optical nodes, and constructs a wavelength conflict relationship graph with associated weights;

[0098] Adaptive scheduling module: Based on wavelength conflict relationship graph and network-wide robustness index, it adaptively adjusts reinforcement learning model parameters and outputs dynamic scheduling instructions.

[0099] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0100] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-objective dynamic optimization method for DWDM based on reinforcement learning and hybrid models, characterized in that, Includes the following steps: The real-time operating status of DWDM links is collected in a distributed manner, and the service arrival sequence is segmented in multiple dimensions by means of a burst traffic identification network. Dynamic request sets with hierarchical identifiers are generated according to latency sensitivity and bandwidth requirements. Perform timing misalignment analysis on the dynamic request set, conduct parallel sliding observations at different time scales, and combine optical power offset trajectory, link attenuation mode and wavelength utilization mapping to generate a composite feature set describing the link dynamics. Based on a composite feature set, a hybrid reinforcement learning model is used to iteratively extrapolate the business evolution process. By capturing abrupt changes in allocation density and resource utilization entropy, potential blocking risk segments are screened out. The wavelength range and associated optical nodes corresponding to the blocking risk segment are defined as the domain to be optimized, and the optical amplification unit, modulation control unit and monitoring unit in the domain to be optimized are integrated to construct a wavelength conflict relationship diagram with correlation weights. Based on the wavelength conflict relationship diagram, combined with the energy efficiency fluctuation trend of the entire network and the link robustness index, the reward function structure and decision step size of the hybrid reinforcement learning model are adaptively adjusted, and dynamic scheduling instructions for DWDM resource allocation are output.

2. The DWDM multi-objective dynamic optimization method based on reinforcement learning and hybrid models according to claim 1, characterized in that, The process of generating a dynamic request set with hierarchical identifiers based on latency sensitivity and bandwidth requirements is as follows: Spatiotemporal sampling is performed on the service arrival sequence obtained from real-time link monitoring to extract the arrival rate, duration, and required bandwidth of the requests; Map the latency tolerance threshold of the request to the target bandwidth requirement to the hierarchical rule table to generate different priority labels; Request sets under the same priority label are hierarchically identified to form dynamic request sets with timestamps and priority codes.

3. The DWDM multi-objective dynamic optimization method based on reinforcement learning and hybrid models according to claim 2, characterized in that, The process of performing timing misalignment analysis on a dynamic request set is as follows: The dynamic request set is divided into high-sensitivity requests and low-sensitivity requests according to priority tags; Short-duration sliding windows are assigned to highly sensitive requests, while long-duration sliding windows are assigned to low-sensitivity requests, forming a multi-scale sliding window set; Synchronously read dynamic request sets within a multi-scale sliding window, and eliminate cross-domain observation errors through timestamp alignment and link phase correction methods; Correlation matching is performed on the observation results of adjacent sliding windows to obtain the observation vector set after temporal misalignment.

4. The DWDM multi-objective dynamic optimization method based on reinforcement learning and hybrid models according to claim 3, characterized in that, The process of generating a composite feature set describing the dynamics of the link is as follows: Power offset curves were constructed using optical power sampling data, and short-term oscillation and long-term drift characteristics were extracted. Exponential smoothing is performed on the link loss data to obtain a decay mode vector that reflects link stability; A distribution matrix is ​​constructed from wavelength utilization data to quantify the sparsity and continuity of resource usage; The power offset feature, attenuation mode vector, and wavelength utilization matrix are uniformly mapped to a composite feature set through a feature embedding method.

5. The DWDM multi-objective dynamic optimization method based on reinforcement learning and hybrid models according to claim 4, characterized in that, The process of iteratively extrapolating the business evolution using a hybrid reinforcement learning model is as follows: The composite feature set is input into a hybrid model that combines deep Q-learning and policy gradient; By using serialized state transition modeling, the evolution trajectory of the business under different resource allocations is iteratively simulated, and the exploration rate and learning rate are dynamically adjusted during the simulation process.

6. The DWDM multi-objective dynamic optimization method based on reinforcement learning and hybrid models according to claim 5, characterized in that, The process of filtering out potential blocking risk segments is as follows: Calculate the allocation density curve in the business evolution trajectory and identify local abrupt changes; The changes in the entropy of statistical resource utilization are measured, and the locations of abrupt change points are detected. If the density jump point and the entropy mutation point overlap in time, the corresponding segment is marked as a blocking risk segment, and the blocking risk segment with delay label and resource label is output.

7. The DWDM multi-objective dynamic optimization method based on reinforcement learning and hybrid models according to claim 6, characterized in that, The process of defining the wavelength range and associated optical nodes corresponding to the blocking risk segment as the domain to be optimized is as follows: Range calibration is performed on the wavelength range involved in the blocking risk segment; Search for upstream and downstream optical nodes connected within the wavelength range to form a set of nodes to be optimized; The wavelength range and the set of nodes to be optimized are jointly defined as the optimization domain.

8. The DWDM multi-objective dynamic optimization method based on reinforcement learning and hybrid models according to claim 7, characterized in that, The process of constructing a wavelength conflict relationship graph with correlation weights is as follows: Within the domain to be optimized, optical amplification units, modulation and control units, and monitoring units are extracted and mapped to a set of nodes. Establish edge connections based on wavelength competition between nodes, power regulation effects, and historical link status; Each edge is assigned a conflict intensity weight using an edge weight calculation function; The set of nodes and weighted edges are combined to form a wavelength conflict relationship graph.

9. The DWDM multi-objective dynamic optimization method based on reinforcement learning and hybrid models according to claim 8, characterized in that, The process of outputting dynamic scheduling instructions for DWDM resource allocation is as follows: By integrating the wavelength conflict relationship diagram with the overall network energy efficiency fluctuation trend, key conflict paths and risk accumulation values ​​are extracted. By combining link robustness metrics, the reward function of the reinforcement learning model is weighted and the decision step size is reset, and dynamic scheduling instructions are generated based on the dynamically optimized model.

10. A DWDM multi-objective dynamic optimization system based on reinforcement learning and a hybrid model, applied to the method described in any one of claims 1-9, characterized in that, include: Request collection module: Distributed collection of real-time operating status of DWDM links, and generation of dynamic request sets with hierarchical identifiers by combining burst traffic identification network; Misalignment Analysis Module: Performs temporal misalignment analysis on dynamic request sets and generates a composite feature set describing link dynamics through multi-scale sliding observations; Inference and screening module: Based on the composite feature set, it uses a hybrid reinforcement learning model to infer the business evolution process and screen potential blocking risk segments; Domain graph construction module: Maps blocking risk segments to wavelength ranges and optical nodes, and constructs a wavelength conflict relationship graph with associated weights; Adaptive scheduling module: Based on wavelength conflict relationship graph and network-wide robustness index, it adaptively adjusts reinforcement learning model parameters and outputs dynamic scheduling instructions.

Images