Laser frequency dynamic compensation and optimization system based on reinforcement learning

Abstract

The invention relates to the technical field of laser system control, and discloses a laser frequency dynamic compensation and optimization system based on reinforcement learning. The system comprises a data sensing module used for collecting real-time data of operation of a laser system; the feature analysis module is used for generating a comprehensive feature vector by applying a wavelet packet decomposition technology; the strategy generation module is used for generating a regulation and control strategy based on a strategy model trained by the deep Q network; the optimization model building module is used for building a multi-objective optimization model by using a particle swarm optimization algorithm to adjust operation parameters; and the hierarchical control implementation module realizes hierarchical execution of the regulation and control strategy through a hierarchical architecture of a decision layer, an adjustment layer and an execution layer. The frequency stability and the power utilization rate of a laser system can be effectively improved, the problems that a traditional method is low in regulation and control precision, poor in optimization effect and the like in a complex environment are solved, and the method has good application prospects in the fields of optical communication, laser processing and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of laser system control, and specifically to a laser frequency dynamic compensation and optimization system based on reinforcement learning. Background Art

[0002] At present, with the wide application of laser technology, the performance optimization of laser systems has become a key research direction. The stability of laser frequency and the power utilization rate have crucial impacts on many fields relying on laser technology. For example, in the field of optical communication, tiny fluctuations in laser frequency may lead to a decline in signal transmission quality, an increase in bit error rate, seriously affecting the accuracy and stability of data transmission, and further restricting the improvement of the transmission distance and rate of optical communication systems; in the field of laser processing, unstable frequency will cause a decrease in processing accuracy and affect product quality, while low power utilization efficiency will increase production costs and reduce production efficiency.

[0003] Traditional laser frequency control methods often show limitations when facing complex and changeable working environments and fluctuations in system internal parameters. The early open-loop control method only adjusts according to preset parameters and cannot perceive the actual operating state of the system in real time. When environmental factors such as temperature and humidity change or components inside the laser system age, it is difficult to effectively adjust the laser frequency, resulting in a large frequency deviation and unable to meet the high-precision application requirements.

[0004] The later-developed closed-loop control methods, although able to adjust the laser frequency to a certain extent according to feedback signals, mostly adopt fixed control strategies and parameters and lack adaptability. Under different working conditions, these fixed parameters cannot achieve the optimal control effect. For example, in an environment with drastic temperature changes, the frequency response characteristics of the laser system will change, and traditional closed-loop control systems cannot adjust the control parameters in time to adapt to this change, affecting both the accuracy and speed of frequency adjustment.

[0005] In terms of data processing and feature extraction, existing technical means also have deficiencies. For the large amount of complex data generated during the operation of the laser system, such as frequency fluctuation data, power data, temperature data, and environmental parameter data, traditional methods often simply analyze and process them, unable to fully exploit the deep information contained in the data. This leads to a lack of comprehensive and accurate basis when formulating control strategies, making it difficult to achieve precise control of laser frequency and power and unable to meet the optimization requirements of both frequency stability and power utilization rate simultaneously.

[0006] In addition, in terms of optimization algorithms and control architectures, existing laser systems also need to be improved. Traditional optimization algorithms are prone to falling into local optimal solutions when dealing with multi-objective optimization problems and cannot find the globally optimal combination of operating parameters. At the same time, simple control architectures are difficult to achieve refined hierarchical control of laser systems, lacking effective coordination at the decision-making, adjustment, and execution levels, resulting in the inability to fully improve the overall performance of the system. In summary, it is of great practical significance to develop a system that can adapt to complex environments, precisely control laser frequencies, and improve power utilization efficiency. Summary of the Invention

[0007] The purpose of the present invention is to provide a laser frequency dynamic compensation and optimization system based on reinforcement learning to solve the problems presented in the above background technology.

[0008] To achieve the above purpose, the present invention provides the following technical solution: A laser frequency dynamic compensation and optimization system based on reinforcement learning, the system includes:

[0009] Data perception module: used to collect real-time data during the operation of the laser system through a variety of sensors, including frequency fluctuation data, power data, temperature data, and environmental parameter data;

[0010] Feature analysis module: uses wavelet packet decomposition technology to perform multi-scale feature analysis on the collected real-time data to generate a comprehensive feature vector;

[0011] Policy generation module: inputs the comprehensive feature vector into a policy model trained based on a deep Q network to generate a laser frequency regulation policy;

[0012] Optimization model construction module: constructs a multi-objective optimization model according to the regulation policy, with frequency stability and power utilization efficiency as optimization objectives, and uses the particle swarm optimization algorithm to adjust the operating parameters of the laser system;

[0013] Hierarchical control implementation module: based on the optimal regulation plan output by the multi-objective optimization model, realizes the hierarchical execution of the regulation policy through a hierarchical control architecture, which includes a decision-making layer, an adjustment layer, and an execution layer. Among them, the decision-making layer generates a global regulation plan according to the regulation policy, the adjustment layer uses an adaptive filtering algorithm to dynamically optimize local regulation parameters, and the execution layer realizes precise regulation of laser frequency and power based on the adaptive sliding mode control algorithm.

[0014] Preferably, the collection of real-time data during the operation of the laser system through a variety of sensors includes:

[0015] The variety of sensors includes a frequency sensor, a power sensor, a temperature sensor, and an environmental monitoring sensor;

[0016] Synchronize the frequency sensor data and the environmental monitoring sensor data in time to construct a frequency-environment correlation graph; perform empirical mode decomposition on the power sensor data and the temperature sensor data to generate a device performance feature sequence;

[0017] Construct a multi-channel feature fusion network. The first channel uses a convolutional neural network to extract the spatial features of the frequency-environment correlation graph, and the second channel uses a gated recurrent unit network to extract the temporal features of the device performance feature sequence;

[0018] Fuse the spatial features and the temporal features through an attention mechanism to generate a fused feature matrix; perform dynamic feature capture on the fused feature matrix based on a long short-term memory network, and output a comprehensive feature vector including the frequency fluctuation state, the device performance state, and the environmental impact factors.

[0019] Preferably, the policy model trained based on the deep Q network adopts a hierarchical neural network structure and generates a regulation policy based on a dynamic reward allocation mechanism; the hierarchical neural network structure includes:

[0020] Construct a laser system-environment interaction graph. The nodes in the graph include a laser source node, an environment node, a power node, and a frequency node, and the node attributes include frequency values, power values, and environmental parameter values;

[0021] Adopt a two-level attention mechanism. The first level calculates the interaction weights between the laser source node and adjacent nodes through a spatial neural network layer, and the second level ranks the importance of the historical operating states through a temporal neural network layer;

[0022] Update and iterate the node features based on a multi-head attention module. Each attention head fuses the node attributes and external environmental parameters; stabilize the training process through skip connections and normalization layers, and finally output a regulation policy including frequency stability requirements and efficient power utilization.

[0023] Preferably, the particle swarm optimization algorithm integrates dynamic inertia weight adjustment and a local search strategy, including:

[0024] Model the parameter optimization problem as a multi-objective optimization problem, and the decision variables include continuous frequency adjustment variables and power adjustment variables;

[0025] Initialize the particle swarm and calculate the initial fitness value, and adopt a dynamic inertia weight adjustment mechanism to update the inertia weight coefficient according to the optimization process;

[0026] In the global search stage, guide the particle search based on the global optimal solution of the particle swarm; in the local search stage, introduce a local search operator to enhance the local search ability;

[0027] Use parallel computing to accelerate particle update, update the particle position each time and synchronize the global optimal solution through information sharing.

[0028] Preferably, the dynamic optimization of the local regulation parameters by the adaptive filtering algorithm includes:

[0029] Construct an adaptive prediction model, transform the laser system dynamics equation into an adaptive state space model, which includes frequency prediction error, power tracking error and differential terms of adjustment parameters;

[0030] Design an adaptive filtering optimization objective function, which includes a frequency fluctuation suppression term, a power stability improvement term and an adjustment accuracy enhancement term.

[0031] Preferably, the precise adjustment of the laser frequency and power by the execution layer based on the adaptive sliding mode control algorithm includes:

[0032] Design a double-loop sliding mode surface, map the frequency error and power error into a sliding mode surface switching function; construct an adaptive reaching law adjustment mechanism to dynamically adjust the sliding mode surface reaching law parameters according to the error change rate.

[0033] Preferably, the convolutional neural network uses a dilated convolutional pyramid structure to accelerate feature extraction, including:

[0034] Divide the frequency-environment correlation graph into subgraphs with different resolutions, and each subgraph stores the correlation features of frequency and environmental parameters;

[0035] Use dilated convolution in the convolutional layer to expand the receptive field, use adaptive pooling in the pooling layer to retain key features, and use transposed convolution in the transposed convolutional layer to restore spatial details;

[0036] Introduce a channel attention mechanism to optimize the weights of the feature map.

[0037] Preferably, the spatial neural network layer uses a relative position encoding mechanism, including:

[0038] Define the relative position vectors of the laser source node and adjacent nodes, including frequency difference, power transmission difference and environmental impact difference;

[0039] Map the relative position vectors into the bias parameters of the neural network through a non-linear mapping layer;

[0040] Superimpose the bias parameters on the standard neural network operations.

[0041] Preferably, the dynamic inertia weight adjustment mechanism is implemented based on an adaptive learning strategy, including:

[0042] Collect the fitness value and search accuracy during the historical optimization process as training samples;

[0043] Construct a mapping relationship between the neural network fitting inertia weight coefficient and the optimization state;

[0044] Update network parameters online through stochastic gradient descent method and adjust inertia weight distribution in real time;

[0045] When optimization stagnation is detected, an inertia weight reinitialization operation is triggered.

[0046] Preferably, the adaptive prediction model is implemented by Kalman filtering, including:

[0047] Frequency uncertainty is modeled as a noise term in the state-space model, and the noise covariance is determined by the historical prediction error. The Kalman filter algorithm is used to estimate and update the state variables to separate the deterministic and uncertain components.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The system has obvious advantages in data processing and feature analysis. The data perception module uses a variety of sensors to comprehensively collect laser system operating data, covering various information such as frequency fluctuations, power, temperature and environmental parameters. Subsequently, the feature analysis module uses wavelet packet decomposition technology to perform multi-scale feature analysis on these data. This technology can deeply explore the characteristics of data at different frequency scales, and can obtain richer and more valuable information compared to traditional simple data processing methods. The multi-channel feature fusion network combines convolutional neural networks, gated recurrent unit networks, and attention mechanisms to efficiently fuse different types of data features. This not only accurately captures the frequency fluctuation status, equipment performance status, and environmental influencing factors, but the generated comprehensive feature vector also provides a comprehensive and accurate basis for the subsequent formulation of control strategies, greatly improving the system's ability to recognize the operating status of the laser system.

[0050] The strategy generation module utilizes a strategy model based on deep Q-network training, combined with a hierarchical neural network structure and a dynamic reward allocation mechanism, producing promising results. By constructing a laser system-environment interaction graph, combined with a dual-level attention mechanism and a multi-head attention module, the hierarchical neural network fully accounts for the complex interactions between the laser system and its environment. This generated control strategy balances frequency stability requirements with efficient power utilization. Compared to traditional fixed strategies, it can be dynamically adjusted based on the system's real-time status, significantly improving the scientific nature and effectiveness of the control strategy and enabling more efficient operation of the laser system under varying operating conditions.

[0051] The optimization model building module uses a particle swarm optimization algorithm that integrates dynamic inertia weight adjustment and local search strategy to optimize the operating parameters of the laser system. The parameter optimization problem is modeled as a multi-objective optimization problem, which can consider two key objectives, frequency stability and power utilization rate, simultaneously. The dynamic inertia weight adjustment mechanism is based on an adaptive learning strategy, which adjusts the inertia weight coefficient in real time according to the optimization process. It is beneficial for global search in the initial stage of optimization and enhances local search ability in the later stage, avoiding falling into local optimal solutions. The adoption of parallel computing accelerates particle update, improves the optimization efficiency, enables the system to quickly find a better combination of operating parameters, and thus effectively improves the overall performance of the laser system.

[0052] The hierarchical control implementation module's hierarchical control architecture further improves the control effect of the system. The decision-making layer generates a global control plan according to the control strategy, grasps the control direction of the laser system from a macroscopic level, and ensures the integrity and coordination of the control. The adjustment layer uses an adaptive filtering algorithm to construct an adaptive prediction model and an optimization objective function, and performs dynamic optimization according to the laser system's dynamic equation to accurately adjust the local control parameters, improving the refinement degree of the control. The execution layer is based on an adaptive sliding mode control algorithm, designs a double-loop sliding mode surface and an adaptive reaching law adjustment mechanism, and can achieve accurate adjustment of the laser frequency and power. This hierarchical and collaborative control method makes the laser system more stable and accurate during the control process compared to the traditional simple control architecture, effectively improving the frequency stability and power utilization rate, and meeting the requirements of various high-precision application scenarios. Brief Description of the Drawings

[0053] Figure 1 is the working principle diagram of the laser frequency dynamic compensation and optimization system based on the present invention;

[0054] Figure 2 is the flowchart of data acquisition and comprehensive feature vector generation;

[0055] Figure 3 is the flowchart of the policy model based on deep Q-network training;

[0056] Figure 4 is the step diagram of the particle swarm optimization algorithm. Detailed Embodiment

[0057] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0058] Please refer to Figures 1-4, the present invention provides a laser frequency dynamic compensation and optimization system based on reinforcement learning, and its specific implementation scheme is elaborated in detail below.

[0059] The system mainly consists of a data perception module, a feature analysis module, a policy generation module, an optimization model construction module, and a hierarchical control implementation module.

[0060] The data perception module is responsible for collecting real-time data during the operation of the laser system through a variety of sensors. These sensors include frequency sensors, power sensors, temperature sensors, and environmental monitoring sensors. The frequency sensor monitors the laser frequency fluctuations in real time, the power sensor accurately measures the laser power data, the temperature sensor obtains the temperature information of the laser system, and the environmental monitoring sensor collects environmental parameter data such as air pressure and humidity.

[0061] The feature analysis module uses wavelet packet decomposition technology to perform multi-scale feature analysis on the collected real-time data. Wavelet packet decomposition technology can decompose complex signals at different frequency scales to obtain richer feature information. Through this technology, the frequency fluctuation data, power data, temperature data, and environmental parameter data are processed to generate a comprehensive feature vector, which contains key features in many aspects such as the operating state of the laser system and environmental factors.

[0062] The policy generation module inputs the comprehensive feature vector into a policy model trained based on the deep Q-network. The deep Q-network is a neural network model based on reinforcement learning. After a large number of trainings, it can generate corresponding action policies according to the input state information (i.e., the comprehensive feature vector). In this system, the policy model generates a laser frequency regulation policy to achieve reasonable regulation of the laser frequency.

[0063] The optimization model construction module constructs a multi-objective optimization model based on the regulation policy generated by the policy generation module. This model takes frequency stability and power utilization rate as optimization objectives and uses the particle swarm optimization algorithm to adjust the operating parameters of the laser system. The particle swarm optimization algorithm simulates the foraging behavior of bird flocks and searches for the optimal solution through the mutual cooperation and information sharing of individuals in the group. In this system, this algorithm optimizes the operating parameters such as the frequency adjustment variable and power adjustment variable of the laser system to improve the performance of the laser system.

[0064] Based on the optimal regulation scheme output by the multi-objective optimization model, the hierarchical control implementation module realizes the hierarchical execution of the regulation strategy through the hierarchical control architecture. The hierarchical control architecture includes a decision-making layer, an adjustment layer, and an execution layer. The decision-making layer generates a global regulation plan according to the regulation strategy, determining the regulation direction of the laser system from a macroscopic level; the adjustment layer uses an adaptive filtering algorithm to dynamically optimize the local regulation parameters, adjusts the local parameters according to the real-time operating state of the laser system, and improves the accuracy of regulation; the execution layer realizes the precise adjustment of the laser frequency and power based on the adaptive sliding mode control algorithm, ensuring the stable operation of the laser system according to the expected regulation scheme.

[0065] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0066] Embodiment 1:

[0067] In the data acquisition link, a frequency sensor is used to monitor the change of the laser frequency in real time and output the collected frequency data in the form of a time series. The environmental monitoring sensor synchronously collects environmental parameter data, such as temperature, humidity, air pressure, etc. Using time synchronization technology, the frequency sensor data and the environmental monitoring sensor data are time-aligned to construct a frequency-environment correlation diagram. In this correlation diagram, the abscissa represents time, and the ordinate shows different environmental parameters and the corresponding frequency data respectively, so as to clearly present the correlation relationship between the frequency and the environmental parameters.

[0068] The data collected by the power sensor and the temperature sensor are processed by the empirical mode decomposition (EMD) method. Empirical mode decomposition is an adaptive signal decomposition method that can decompose complex signals into multiple intrinsic mode functions (IMFs). After the empirical mode decomposition of the power sensor data and the temperature sensor data, a device performance feature sequence is generated. Each IMF component represents the characteristics of the signal at different time scales, and these feature sequences reflect the performance change laws of the device in terms of power and temperature.

[0069] A multi-channel feature fusion network is constructed, which includes two channels. The first channel uses a convolutional neural network (CNN) to extract the spatial features of the frequency-environment correlation diagram. The convolutional neural network can automatically learn the features in the image through components such as convolutional layers and pooling layers. In this embodiment, the convolutional neural network performs a convolutional operation on the frequency-environment correlation diagram to extract the spatial features therein, such as the local correlation patterns between the frequency and the environmental parameters. The second channel uses a gated recurrent unit network (GRU) to extract the temporal features of the device performance feature sequence. The gated recurrent unit network is a special recurrent neural network that can effectively process time series data and capture the long-term dependencies in the data. The GRU network processes the device performance feature sequence and extracts the feature information that changes over time.

[0070] Fuse spatial features and temporal features through an attention mechanism. The attention mechanism can perform weighted fusion on different features according to the importance of the features. In this embodiment, calculate the attention weights of the spatial features and the temporal features, and assign a larger fusion weight to the features with higher weights, thereby generating a fused feature matrix. Finally, perform dynamic feature capture on the fused feature matrix based on the Long Short-Term Memory network (LSTM). The Long Short-Term Memory network has the ability to remember long-term information and process sequence dependencies, and can capture comprehensive features such as the frequency fluctuation state, device performance state, and environmental influence factors of the laser system from the fused feature matrix, and output a comprehensive feature vector containing this information.

[0071] Embodiment 2:

[0072] The policy model trained based on the Deep Q-Network adopts a hierarchical neural network structure and generates a regulation policy based on a dynamic reward allocation mechanism. First, construct a laser system-environment interaction graph. The nodes in the graph include a laser source node, an environment node, a power node, and a frequency node. The laser source node records the relevant attributes of the laser source, such as the initial frequency setting value, emission power, etc.; the environment node stores the environmental parameter values, such as temperature, humidity, air pressure, etc.; the power node contains the power value information of the laser system; the frequency node records the real-time frequency value of the laser. These node attributes are updated in real time as the laser system operates.

[0073] Adopt a two-level attention mechanism. The first level calculates the interaction weights between the laser source node and adjacent nodes through a spatial neural network layer. The spatial neural network layer adopts a relative position encoding mechanism to define the relative position vector between the laser source node and adjacent nodes. This vector contains frequency differences, power transmission differences, and environmental influence differences. For example, the frequency difference represents the difference between the current frequency of the laser source node and the frequency of the adjacent frequency node; the power transmission difference reflects the change in power transmission from the laser source node to the adjacent power node; the environmental influence difference reflects the difference in environmental parameters between the environment where the laser source node is located and the adjacent environment node. Map the relative position vector to the bias parameter of the neural network through a non-linear mapping layer, and then superimpose the bias parameter on the standard neural network operation to calculate the interaction weights between the laser source node and adjacent nodes, highlighting the features of adjacent nodes that have a greater impact on the operation of the laser source.

[0074] The second level ranks the importance of historical operating states through a temporal neural network layer. The temporal neural network layer can learn the time series features in the historical operating data of the laser system, rank the historical operating states according to the influence degree of the operating states at different times on the current regulation decision, and assign a higher weight to the historical states with more reference value.

[0075] The node features are updated iteratively based on the multi-head attention module. Each attention head fuses the node attributes and external environment parameters to capture node information from different perspectives. The training process is stabilized through skip connections and normalization layers. Skip connections can avoid the vanishing gradient problem, enabling the network to better learn deep features; the normalization layer normalizes the data in the network to accelerate model convergence. After the above operations, the final output contains a regulation strategy for frequency stability requirements and power efficient utilization, providing a reasonable decision basis for laser frequency regulation.

[0076] Embodiment 3:

[0077] When using the particle swarm optimization algorithm to adjust the operating parameters of the laser system, first, the parameter optimization problem is modeled as a multi-objective optimization problem. The decision variables include the continuous frequency adjustment variable f adj and the power adjustment variable p adj , where the frequency adjustment variable f adj is used to adjust the frequency of the laser, and the power adjustment variable p adj is used to adjust the power of the laser.

[0078] Initialize the particle swarm and calculate the initial fitness value. The particle swarm consists of multiple particles, and each particle represents a set of possible combinations of frequency adjustment variables and power adjustment variables. In the initialization stage, random initial positions and velocities are assigned to each particle. According to the frequency stability objective function F s (f adj ,p adj ) and the power utilization objective function P u (f adj ,p adj ), calculate the initial fitness value of each particle. For example, the frequency stability objective function F s (f adj ,p adj ) can be defined as the reciprocal of the sum of the squares of the deviations between the current frequency and the target frequency. The smaller the deviation, the higher the frequency stability, and the larger the value of F s (f adj ,p adj ); the power utilization objective function P u (f adj ,p adj ) can be expressed as the ratio of the effective output power to the total input power. The higher the ratio, the higher the power utilization, and the larger the value of P u (f adj ,p adj ). Considering these two objective functions comprehensively, the initial fitness value fitness init of the particle is obtained.

[0079] The dynamic inertia weight adjustment mechanism is adopted to update the inertia weight coefficient according to the optimization process. The dynamic inertia weight adjustment mechanism is implemented based on an adaptive learning strategy. The fitness values in the historical optimization process are collected hist and the search accuracy accuracy hist are used as training samples. A neural network is constructed to fit the mapping relationship between the inertia weight coefficient w and the optimization state (represented by the fitness value and the search accuracy), that is, w = f(fitness hist , accuracy hist ). The network parameters are updated online by the stochastic gradient descent method to adjust the inertia weight allocation in real time. At the initial stage of optimization, a larger inertia weight is beneficial for the particles to perform global search and explore a wider solution space; as the optimization process progresses, the inertia weight is gradually reduced to enhance the local search ability of the particles. When it is detected that the optimization stagnates, that is, the fitness value does not improve significantly after consecutive iterations, the inertia weight re-initialization operation is triggered, enabling the particles to jump out of the local optimal solution and continue to search for a better parameter combination.

[0080] In the global search stage, the global optimal solution of the particle swarm is used to guide the particle search. Each particle updates its velocity and position according to its own historical best solution pBest and the global best solution gBest. The velocity update formula of the particle is:

[0081] v i (t + 1) = w × v i (t) + c1r1(t)[pBest i (t) - x i (t)] + c2r2(t)[gBest(t) - x i (t)]

[0082] where, v i (t) represents the velocity of the i-th particle at time t; x i (t) represents the position of the i-th particle at time t; w is the inertia weight coefficient; c1 and c2 are learning factors, usually set as constants, which are used to control the step size of the particle flying towards its own historical best solution and the global best solution; r1(t) and r2(t) are random numbers between [0, 1]. Through this formula, the particles continuously adjust their positions under the guidance of the global best solution and approach a better solution.

[0083] In the local search stage, a local search operator is introduced to enhance the local search ability. For example, the local random perturbation method can be adopted to randomly generate a new solution near the current particle position and calculate the fitness value of the new solution. If the fitness value of the new solution is better than the current solution, the particle position is updated, enabling the particle to search for the optimal solution more finely in the local area.

[0084] Parallel computing is used to accelerate particle updates. Each time the particle position is updated, the global optimal solution is synchronized through information sharing. By utilizing a multi-core processor or a parallel computing platform, the velocity and position updates of multiple particles are calculated simultaneously to improve the optimization efficiency. After each update, the information of each particle is aggregated through shared memory or network communication to synchronize the global optimal solution, ensuring that each particle can obtain the current optimal information for better subsequent search.

[0085] Example 4:

[0086] The adjustment layer uses an adaptive filtering algorithm to dynamically optimize the local regulation parameters. An adaptive prediction model is constructed, and the laser system dynamic equation is transformed into an adaptive state space model. The laser system dynamic equation describes the dynamic relationship between various physical quantities during the operation of the laser system. Through reasonable mathematical transformations, it is transformed into an adaptive state space model. This model includes the frequency prediction error e f , the power tracking error e p , and the differential term of the adjustment parameter The frequency prediction error e f represents the difference between the predicted frequency and the actual frequency. The power tracking error e p represents the deviation between the actual power and the target power. The differential term of the adjustment parameter reflects the change rate of the regulation parameter.

[0087] Design an adaptive filtering optimization objective function, which includes a frequency fluctuation suppression term J f , a power stability improvement term J p , and an adjustment precision enhancement term J a . The frequency fluctuation suppression term J f can be defined as the sum of the squares of the frequency prediction errors, that is where N is the number of sampling points, and e f (k) is the frequency prediction error at the k-th sampling point. By minimizing J f , the fluctuation of the laser frequency can be effectively suppressed. The power stability improvement term J p can be expressed as the sum of the squares of the power tracking errors, that is e p (k) is the power tracking error at the k-th sampling point. Minimizing J p helps to improve the stability of the laser power. The adjustment precision enhancement term J a can be defined according to the magnitude of the differential term of the adjustment parameter , for example By minimizing J a , the adjustment precision of the regulation parameter can be enhanced, making the regulation more accurate. Combining these three terms, the adaptive filtering optimization objective function J = J f + Jp +J a 。

[0088] The adaptive prediction model is implemented through Kalman filtering. The frequency uncertainty is modeled as a noise term in the state space model, and the noise covariance is determined by the historical prediction errors. Assume the historical prediction error is e hist , by performing statistical analysis on e hist , the noise covariance matrix Q is calculated. The Kalman filtering algorithm is used to estimate and update the state variables, separating the deterministic and uncertain components. The Kalman filtering algorithm is an optimal linear recursive estimation method. It continuously corrects the estimated value of the state variable through two steps: prediction and update. In the prediction step, the state value at the current moment is predicted based on the state estimate value at the previous moment and the system model; in the update step, the predicted value is corrected using the measurement value at the current moment to obtain a more accurate state estimate value. Through Kalman filtering, the operating state of the laser system can be predicted more accurately, providing reliable data support for adaptive filtering optimization.

[0089] Example 5:

[0090] The execution layer realizes precise adjustment of the laser frequency and power based on the adaptive sliding mode control algorithm. A double-loop sliding mode surface is designed, and the frequency error e f and the power error e p are mapped into the sliding mode surface switching function. Define the frequency sliding mode surface where c f is the parameter of the frequency sliding mode surface, used to adjust the dynamic characteristics of the frequency sliding mode surface; the power sliding mode surface c p is the parameter of the power sliding mode surface. The sliding mode surface switching function is defined according to the values of the frequency sliding mode surface and the power sliding mode surface. For example, S = S f +S p . When the system state moves on the sliding mode surface, the stability and robustness of the system can be guaranteed.

[0091] Construct an adaptive reaching law adjustment mechanism to dynamically adjust the parameters of the sliding mode surface reaching law according to the error change rate. The reaching law is the law that controls the sliding mode surface to reach the sliding mode surface from the initial state and maintain motion on the sliding mode surface. Common reaching laws include the constant velocity reaching law, the exponential reaching law, etc. In this example, the adaptive exponential reaching law is adopted, and its expression is:

[0092]

[0093] Among them, is the derivative of the sliding mode surface switching function; ε is the reaching law parameter used to control the reaching speed; k is the gain coefficient used to enhance the robustness of the system; sgn(S) is the sign function, where sgn(S)=1 when S>0; sgn(S)=0 when S = 0; sgn(S)=-1 when S<0. According to the rate of change of the frequency error and the rate of change of the power error dynamically adjust the reaching law parameters ε and k. For example, when the rate of change of the frequency error is large, appropriately increase the value of ε to make the system approach the sliding mode surface faster; when the rate of change of the power error is small, reduce the value of k to reduce the chattering of the system. Through this adaptive reaching law adjustment mechanism, the control parameters can be adjusted according to the real-time operating state of the system, realizing the precise adjustment of the laser frequency and power, and improving the performance and stability of the laser system.

[0094] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0095] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A laser frequency dynamic compensation and optimization system based on reinforcement learning, characterized in that, Including: Data perception module: used to collect real-time data during the operation of the laser system through multiple sensors, including frequency fluctuation data, power data, temperature data, and environmental parameter data; Feature analysis module: uses wavelet packet decomposition technology to perform multi-scale feature analysis on the collected real-time data and generate a comprehensive feature vector; Strategy generation module: inputs the comprehensive feature vector into a strategy model trained based on a deep Q network to generate a laser frequency regulation strategy; Optimization model construction module: constructs a multi-objective optimization model according to the regulation strategy. This model takes frequency stability and power utilization rate as optimization objectives and uses the particle swarm optimization algorithm to adjust the operating parameters of the laser system; Hierarchical control implementation module: based on the optimal regulation plan output by the multi-objective optimization model, realizes the hierarchical execution of the regulation strategy through a hierarchical control architecture. The hierarchical control architecture includes a decision-making layer, an adjustment layer, and an execution layer. Among them, the decision-making layer generates a global regulation plan according to the regulation strategy, the adjustment layer uses an adaptive filtering algorithm to dynamically optimize local regulation parameters, and the execution layer realizes precise regulation of laser frequency and power based on the adaptive sliding mode control algorithm.

2. The laser frequency dynamic compensation and optimization system based on reinforcement learning according to claim 1, wherein, The collection of real-time data during the operation of the laser system through multiple sensors includes: The multiple sensors include a frequency sensor, a power sensor, a temperature sensor, and an environmental monitoring sensor; Perform time synchronization on the frequency sensor data and the environmental monitoring sensor data to construct a frequency-environment correlation graph; perform empirical mode decomposition on the power sensor data and the temperature sensor data to generate a device performance feature sequence; Construct a multi-channel feature fusion network. The first channel uses a convolutional neural network to extract the spatial features of the frequency-environment correlation graph, and the second channel uses a gated recurrent unit network to extract the temporal features of the device performance feature sequence; Fuse the spatial features and the temporal features through an attention mechanism to generate a fused feature matrix; perform dynamic feature capture on the fused feature matrix based on a long short-term memory network, and output a comprehensive feature vector including frequency fluctuation state, device performance state, and environmental impact factors.

3. The laser frequency dynamic compensation and optimization system based on reinforcement learning according to claim 1, characterized in that, The strategy model trained based on the deep Q network adopts a hierarchical neural network structure and generates a regulation strategy based on a dynamic reward allocation mechanism; The hierarchical neural network structure includes: Construct a laser system-environment interaction graph. The nodes in the graph include a laser source node, an environment node, a power node, and a frequency node. The node attributes include frequency values, power values, and environmental parameter values; Adopt a two-level attention mechanism. The first level calculates the interaction weights between the laser source node and adjacent nodes through a spatial neural network layer, and the second level ranks the importance of historical operating states through a temporal neural network layer; Update and iterate the node features based on a multi-head attention module. Each attention head fuses the node attributes and external environmental parameters; stabilize the training process through skip connections and normalization layers, and finally output a regulation strategy including frequency stability requirements and efficient power utilization.

4. The laser frequency dynamic compensation and optimization system based on reinforcement learning according to claim 1, characterized in that The particle swarm optimization algorithm integrates dynamic inertia weight adjustment and local search strategies, including: Model the parameter optimization problem as a multi-objective optimization problem. The decision variables include continuous frequency adjustment variables and power adjustment variables; Initialize the particle swarm and calculate the initial fitness value. Adopt a dynamic inertia weight adjustment mechanism to update the inertia weight coefficient according to the optimization process. In the global search stage, guide the particle search based on the global optimal solution of the particle swarm. In the local search stage, introduce a local search operator to enhance the local search ability. Use parallel computing to accelerate particle update. Update the particle position each time and synchronize the global optimal solution through information sharing.

5. The laser frequency dynamic compensation and optimization system based on reinforcement learning according to claim 1, characterized in that, The adaptive filtering algorithm dynamically optimizes the local regulation parameters, including: Construct an adaptive prediction model, transform the laser system dynamic equation into an adaptive state space model, which includes frequency prediction error, power tracking error, and the differential term of the adjustment parameter. Design an adaptive filtering optimization objective function, including a frequency fluctuation suppression term, a power stability improvement term, and an adjustment accuracy enhancement term.

6. The laser frequency dynamic compensation and optimization system based on reinforcement learning according to claim 1, characterized in that, The execution layer realizes the precise adjustment of the laser frequency and power based on the adaptive sliding mode control algorithm, including: Design a double-loop sliding mode surface, map the frequency error and power error to the sliding mode surface switching function. Construct an adaptive reaching law adjustment mechanism to dynamically adjust the parameters of the sliding mode surface reaching law according to the error change rate.

7. The laser frequency dynamic compensation and optimization system based on reinforcement learning according to claim 2, characterized in that The convolutional neural network adopts a dilated convolutional pyramid structure to accelerate feature extraction, including: Divide the frequency-environment correlation graph into subgraphs with different resolutions, and each subgraph stores the correlation features of frequency and environmental parameters. In the convolutional layer, use dilated convolution to expand the receptive field, in the pooling layer, use adaptive pooling to retain key features, and in the deconvolution layer, restore the spatial details. Introduce a channel attention mechanism to optimize the weights of the feature map.

8. The laser frequency dynamic compensation and optimization system based on reinforcement learning according to claim 3, characterized in that, The spatial neural network layer adopts a relative position encoding mechanism, including: Define the relative position vector between the laser source node and adjacent nodes, including frequency difference, power transmission difference, and environmental impact difference. Map the relative position vector to the bias parameter of the neural network through a non-linear mapping layer. Superimpose the bias parameter on the standard neural network operation.

9. The laser frequency dynamic compensation and optimization system based on reinforcement learning according to claim 4, wherein The dynamic inertia weight adjustment mechanism is realized based on an adaptive learning strategy, including: Collect the fitness value and search accuracy in the historical optimization process as training samples. Construct a neural network to fit the mapping relationship between the inertia weight coefficient and the optimization state. Update the network parameters online through the stochastic gradient descent method and adjust the inertia weight allocation in real time. When optimization stagnation is detected, trigger the operation of re-initializing the inertia weight.

10. The laser frequency dynamic compensation and optimization system based on reinforcement learning according to claim 5, wherein The adaptive prediction model is realized through Kalman filtering, including: Model the frequency uncertainty as a noise term in the state space model, and the noise covariance is determined by the historical prediction error. Use the Kalman filtering algorithm to estimate and update the state variables, separating the deterministic and uncertain components.

Images