Multi-jitter phase-locked control optimization method and device based on integral points

Abstract

The invention discloses a multi-jitter phase-locked control optimization method and device based on the number of integration points, and relates to the field of phase-locked control, and the method comprises the steps: carrying out the phase-locked control of an optical fiber laser based on a multi-jitter method, and obtaining a current output voltage signal; inputting the current output voltage signal into a preset optimization model to obtain a corresponding optimization integration point number; the preset optimization model is a strategy optimization model based on a Q-learning algorithm; and multiplying the current output voltage signal by a demodulation signal of a corresponding frequency, carrying out integration according to the optimized integration point number to obtain an error signal, and then carrying out phase correction on the fiber laser according to the error signal. The method can dynamically adjust the number of integration points, and achieves the efficient and effective correction of the phase error.

Description

Technical Field

[0001] The present application relates to the field of phase-locked control, and in particular to a multi-jitter phase-locked control optimization method and device based on integration points. Background Art

[0002] The multi-jitter method is a phase-locked technology used for coherent synthesis. It performs small jitters on the phases of the outputs of multiple lasers, monitors the changes in the output interference signals, and then achieves real-time correction of phase errors. The implementation steps of the multi-jitter method on the phase-locked control board are as follows: apply sinusoidal modulation signals of different frequencies to each phase modulator; sample and filter the signal of the photodetector; multiply the filtered photodetector signal with the demodulated signal of the corresponding frequency, and integrate to obtain the error signal; multiply the error signal with the proportional coefficient and output it to the phase modulator for correction. Through this closed-loop control process, real-time correction of phase errors is achieved, thereby maintaining the stability of the coherent synthesis system.

[0003] In the current implementation steps of multi-jitter phase-locked control, when the error signal is demodulated by integration, a shorter integration time may lead to faster control response, but it may also make the system more sensitive to noise; a longer integration time can smooth short-term fluctuations and improve system stability, but it may also cause the system to respond more slowly, which may not be desirable in applications that require fast response. Therefore, choosing the right integration time is a key system design decision. The current method of selecting the right number of integration points (the number of integration points has the same meaning as the integration time) mainly relies on manual adjustment, generally taking 5 times or more of the modulation cycle, and then combining an oscilloscope to observe the peak-to-peak value change of the output voltage to judge the rationality of the number of integration points. However, this method has a complex process and relies on manual experience. It is not only inefficient, but also has large subjectivity and errors, which seriously restricts the automation and accuracy of system optimization. Summary of the invention

[0004] The purpose of this application is to provide a multi-jitter phase-locked control optimization method and device based on the number of integration points, which can dynamically adjust the number of integration points to achieve efficient and effective correction of phase errors.

[0005] To achieve the above objectives, this application provides the following solutions: In a first aspect, the present application provides a multi-jitter phase-locked control optimization method based on integration points, comprising: Perform phase-locked control on the fiber laser based on the multi-jitter method and obtain the current output voltage signal; Inputting the current output voltage signal into a preset optimization model to obtain a corresponding number of optimization integral points; the preset optimization model is a strategy optimization model based on a Q-learning algorithm; The current output voltage signal is multiplied by a demodulated signal of a corresponding frequency, and integrated according to the optimized number of integration points to obtain an error signal, and then the fiber laser is phase corrected according to the error signal.

[0006] In a second aspect, the present application provides a multi-jitter phase-locked control optimization device based on integration points, comprising: The voltage signal output module is used to: perform phase-locked control on the fiber laser based on the multi-jitter method and obtain the current output voltage signal; A reinforcement learning controller, used to: input the current output voltage signal into a preset optimization model to obtain a corresponding number of optimization integral points; the preset optimization model is a strategy optimization model based on a Q-learning algorithm; The phase-locked control board is used to: multiply the current output voltage signal by the demodulated signal of the corresponding frequency, integrate it according to the optimized integration points to obtain an error signal, and then perform phase correction on the fiber laser according to the error signal.

[0007] According to the specific embodiments provided by this application, this application has the following technical effects: This application provides a multi-jitter phase-locked control optimization method and device based on integral points, adopts a preset optimization model, which is a strategy optimization model based on the Q-learning algorithm, and automatically optimizes the integral point setting by introducing the Q-learning algorithm of reinforcement learning, avoiding the cumbersome process of traditional manual adjustment of integral points. Compared with the manual adjustment method in the prior art, this application realizes a more efficient and automated optimization of integral points, greatly improves the automation degree and response speed of phase-locked control, and reduces the errors caused by human factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0009] Figure 1 Schematic diagram of a flow chart of a multi-jitter phase-locked control optimization method based on integration points in one embodiment of the present application.

[0010] Figure 2 This is the principle block diagram of multi-jitter phase locking.

[0011] Figure 3 Schematic diagram of the principle of reinforcement learning.

[0012] Figure 4This is a schematic diagram of simulation results provided in an embodiment of the present application.

[0013] Figure 5 Schematic diagram of a multi-jitter phase-locked control optimization device based on integration points in one embodiment of the present application. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0015] In order to make the purpose, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0016] In an exemplary embodiment, Figure 1 As shown, a multi-jitter phase-locked control optimization method based on integration points is provided, including the following steps 101 to 103.

[0017] Step 101, perform phase-locking control on the fiber laser based on the multi-jitter method, and obtain the current output voltage signal. Figure 2 As shown, the process of phase-locking control of the fiber laser based on the multi-jitter method includes: applying sinusoidal modulation signals of different frequencies to each phase modulator; sampling and filtering the signal of the photodetector; multiplying the filtered photodetector signal with the demodulated signal of the corresponding frequency in the PD feedback controller, and integrating to obtain the error signal, and then multiplying the error signal with the proportional coefficient and outputting it to the phase modulator for correction. Through this closed-loop control process, real-time correction of the phase error is achieved, thereby maintaining the stability of the coherent synthesis system. Corresponding to step 101 of the present application, the output voltage signal obtained can be a signal directly sampled by the photodetector, or it can be a photodetector signal after filtering. Generally, in order to ensure the accuracy of subsequent data, the photodetection signal after filtering will be selected.

[0018] Step 102: Input the current output voltage signal into a preset optimization model to obtain a corresponding number of optimization integral points; the preset optimization model is a strategy optimization model based on a Q-learning algorithm. Figure 3 In an application example, the training iteration process of the preset optimization model in step 102 includes the following steps (21) to (25).

[0019] (21) In one iteration, the fiber laser is phase-locked by using a multi-jitter method to obtain the state at time t; the state at time t represents the peak-to-peak value of the output voltage; specifically, let Represents the output voltage value, state at time t The calculation formula is . min() is the minimum function, is the maximum value function.

[0020] In order to make it easier for the Q-learning algorithm in reinforcement learning to process continuous peak-to-peak data, the peak-to-peak value is discretized into a finite set of states, and then the reinforcement learning algorithm can make decisions based on the discrete states. The calculation formula for the state at time t is: .

[0021] in, is the state at time t, is the output voltage value at time t, is the total number of peak-to-peak discrete states (e.g., it may be 100 peak-to-peak discrete states). is the minimum output voltage, The maximum output voltage can usually be set to an empirical value or estimated by observing data. The minimum and maximum functions are used to ensure the state In 1 to The peak-to-peak discrete state is obtained by discretizing the peak-to-peak value of the output voltage.

[0022] (22) Based on - Greedy strategy, determining the action at time t according to the state at time t and the Q table; the Q table is used to store the value estimation of the state at time t and the action at time t; the action at time t represents the optimization integral points.

[0023] action The agent is in state The actions taken under this application Choose a point number n value, the action space can be set to a finite set of actions: .in, is the first integral point, is the second integral point, is the kth integral point. At the initial stage of the iteration, an action is randomly selected from the set of optional actions. ; In the subsequent iteration stage, in order to avoid falling into the local optimal solution, the commonly used strategy is - Greedy strategy, this strategy chooses the action with the largest current Q value in the Q table most of the time, but it will Explore other actions randomly.

[0024] The formula for determining the action at time t is as follows: .

[0025] in, is the action at time t, For the status Select Action The Q value at Based on -The probability determined by the greedy strategy, () is the maximum value function.

[0026] As training progresses, it gradually decreases , allowing the agent to gradually choose more optimal actions instead of exploring random actions.

[0027] (23) Based on the action at time t, the fiber laser is phase corrected to obtain the state at time t+1 and the corresponding reward at time t; specifically, the state at time t is multiplied by the demodulated signal of the corresponding frequency, and the error signal is obtained by integrating it according to the action at time t, and then the fiber laser is phase corrected according to the error signal. That is, after the action acts on the environment, the environment changes accordingly and feeds back a new state to the intelligent agent. and corresponding rewards .award The agent performs actions at each time step The feedback obtained after the task is used to guide the agent to learn the optimal strategy.

[0028] The reward target of the reward at time t is to minimize the peak-to-peak value of the output voltage, so the reward is designed as: ,and, ,The above two formulas indicate that the larger the peak value, the smaller the reward.

[0029] (24) Based on the state at time t, the action at time t, the state at time t+1 and the reward at time t, the Q table is updated using the Bellman equation.

[0030] The core goal of reinforcement learning is to maximize the cumulative reward of the agent and tend to choose the value of n that minimizes the peak-to-peak value. In Q-learning, the Q-table is a table that stores the value estimate of each state-action pair to help the agent select the optimal action in each state. That is, the Q-table is used to store each state. and each integral point n (i.e. action ), thereby guiding the agent to select the number of integration points that minimizes the peak-to-peak voltage. The Q table is a two-dimensional matrix with each row corresponding to a state , each column corresponds to an action Each element in the Q table Represents in state Next action The cumulative reward (future return) that can be expected to be obtained when

[0031] .

[0032] In the early stages of Q-learning training, the values ​​in the Q table are usually initialized to 0 or random values. After each time step t, the Q table is initialized based on the current state. , Current Action ,award and the next state Specifically, when the Bellman equation is used to update the Q table, the following update formula is used: .

[0033] in, is the state at time t, is the action at time t, For the status Select Action Q value at ; is the learning rate, which controls the step size of each update; is the reward at time t, is the discount factor, is the state at time t+1 All possible actions The meaning of the above update formula is: at each time step t, Q-learning will update the current state-action pair The Q value takes into account the current immediate reward and the future in state Through this cycle, the agent continuously learns and optimizes its strategy to select the optimal action that can obtain the maximum reward in different states.

[0034] (25) If the preset iteration stop condition is not met, the next iteration is performed based on the updated Q table; if the preset iteration stop condition is met, the iteration is stopped and the updated Q table is marked as the preset optimization model. After multiple iterations, the Q value in the Q table will gradually approach the actual state-action value function. Finally, the agent intelligent body in each state Will choose the action that maximizes the long-term return This means that the agent has learned to choose the optimal value of n in each state so as to minimize the peak-to-peak voltage.

[0035] Step 103: multiply the current output voltage signal by the demodulated signal of the corresponding frequency, and integrate according to the optimized integration points to obtain an error signal, and then perform phase correction on the fiber laser according to the error signal.

[0036] In a specific application example, the present application also conducted a simulation under the following conditions: the sampling frequency was 250 MHz, the number of integration points was set to one sampling period at the beginning, a sine wave phase noise with a frequency of 20 kHz was added at the 20th microsecond, and a reinforcement learning controller was added at the 40th microsecond to optimize the number of integration points. The simulation results are shown in Figure 2. Figure 4 As shown, it can be seen from the figure that after optimizing the number of integration points using the method of the present application, the peak-to-peak value of the output voltage is significantly reduced.

[0037] In summary, this application introduces reinforcement learning to automatically optimize the integral point setting, avoiding the cumbersome process of traditional manual adjustment of integral points. Compared with the manual adjustment method in the prior art, this application greatly improves the automation and response speed of phase-locked control optimization and reduces the errors caused by human factors.

[0038] Based on the same inventive concept, the embodiment of the present application also provides a multi-jitter phase-locked control optimization device based on the number of integral points. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more device embodiments provided below can refer to the limitations of the method above, and will not be repeated here. Figure 5 As shown, the multi-jitter phase-locked control optimization device based on integration points of the present application includes a voltage signal output module, a reinforcement learning controller, a phase-locked control board, and forms a control closed loop, which can dynamically adjust the integration points to achieve effective correction of phase errors.

[0039] The voltage signal output module is used to perform phase-locked control on the fiber laser based on the multi-jitter method, and obtain the current output voltage signal, which is then input into the reinforcement learning controller. The current output voltage signal is the peak-to-peak value of the output voltage.

[0040] The reinforcement learning controller is used to: input the current output voltage signal into a preset optimization model to obtain a corresponding number of optimization integral points, and then input the number of optimization integral points into a phase-locked control board; the preset optimization model is a strategy optimization model based on a Q-learning algorithm.

[0041] The phase-locked control board is used to: multiply the current output voltage signal by the demodulated signal of the corresponding frequency, and integrate it according to the optimized integration points to obtain an error signal, and then perform phase correction on the fiber laser according to the error signal, thereby generating the next output voltage signal and transmitting it to the voltage signal output module.

[0042] In a specific practical application, the reinforcement learning controller includes a reinforcement learning training module and an optimization model application module; wherein the reinforcement learning training module is used to perform the following training iteration process, thereby realizing multiple iterative calculations based on input values: In one iteration, the fiber laser is phase-locked and controlled by a multi-jitter method to obtain the state at time t; the state at time t represents the peak-to-peak value of the output voltage; based on - Greedy strategy, determining the action at time t according to the state at time t and the Q table; the Q table is used to store the value estimation of the state at time t and the action at time t; the action at time t represents the optimization integral points; based on the action at time t, the fiber laser is phase corrected to obtain the state at time t+1 and the corresponding reward at time t; based on the state at time t, the action at time t, the state at time t+1 and the reward at time t, the Q table is updated using the Bellman equation; if the preset iteration stop condition is not met, the next iteration is performed based on the updated Q table; if the preset iteration stop condition is met, the iteration is stopped and the updated Q table is marked as the preset optimization model.

[0043] In summary, under the monitoring of the peak-to-peak value of the output voltage, the reinforcement learning controller can effectively ensure the system stability and the optimal phase-locking effect by adaptively adjusting the number of integral points. In addition, the method of automatically adjusting the number of integral points greatly reduces manual intervention and improves the overall reliability and efficiency of the system. Obviously, the application proposes to use a reinforcement learning algorithm to automatically optimize the number of integral points, which can avoid the impact of inaccurate integral points on the phase-locking effect.

[0044] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0045] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A multi-jitter phase-locked control optimization method based on integration points, characterized in that: The multi-jitter phase-locked control optimization method based on the number of integral points includes: Perform phase-locked control on the fiber laser based on the multi-jitter method and obtain the current output voltage signal; Inputting the current output voltage signal into a preset optimization model to obtain a corresponding number of optimization integral points; the preset optimization model is a strategy optimization model based on a Q-learning algorithm; The current output voltage signal is multiplied by a demodulated signal of a corresponding frequency, and integrated according to the optimized number of integration points to obtain an error signal, and then the fiber laser is phase corrected according to the error signal.

2. The multi-jitter phase-locked control optimization method based on the number of integral points according to claim 1, characterized in that: The training iteration process of the preset optimization model includes: In one iteration, a multi-jitter method is used to perform phase-locked control on the fiber laser to obtain a state at time t; the state at time t represents a peak-to-peak value of the output voltage; based on - Greedy strategy, determining the action at time t according to the state at time t and the Q table; the Q table is used to store the value estimation of the state at time t and the action at time t; the action at time t represents the optimization integral points; Based on the action at time t, the fiber laser is phase corrected to obtain the state at time t+1 and the corresponding reward at time t; Based on the state at time t, the action at time t, the state at time t+1 and the reward at time t, the Q table is updated using the Bellman equation; If the preset iteration stop condition is not met, the next iteration is performed based on the updated Q table; if the preset iteration stop condition is met, the iteration is stopped and the updated Q table is marked as the preset optimization model.

3. The multi-jitter phase-locked control optimization method based on the number of integral points according to claim 2, characterized in that: The calculation formula of the state at time t is: ； in, is the state at time t, is the output voltage value at time t, is the total number of peak-to-peak discrete states, is the minimum output voltage, is the maximum output voltage, min () is the minimum function, is the maximum value function; the peak-to-peak discrete state is obtained by discretizing the peak-to-peak value of the output voltage.

4. The multi-jitter phase-locked control optimization method based on the number of integral points according to claim 2, characterized in that: The reward target of the reward at time t is to minimize the peak-to-peak value of the output voltage.

5. The multi-jitter phase-locked control optimization method based on the number of integral points according to claim 2, characterized in that: When the Bellman equation is used to update the Q table, the following update formula is used: ； in, is the state at time t, is the action at time t, For the status Select Action The Q value at is the learning rate, is the reward at time t, is the discount factor, is the state at time t+1 All possible actions The maximum Q value in is the maximum value function.

6. The multi-jitter phase-locked control optimization method based on the number of integral points according to claim 2, characterized in that: The formula for determining the action at time t is as follows: ； in, is the action at time t, For the status Select Action The Q value at Based on - the probability determined by the greedy strategy, () is the maximum value function.

7. A multi-jitter phase-lock control optimization device based on integration points, characterized in that: The multi-jitter phase-locked control optimization device based on the number of integral points comprises: The voltage signal output module is used to: perform phase-locked control on the fiber laser based on the multi-jitter method and obtain the current output voltage signal; A reinforcement learning controller, used to: input the current output voltage signal into a preset optimization model to obtain a corresponding number of optimization integral points; the preset optimization model is a strategy optimization model based on a Q-learning algorithm; The phase-locked control board is used to: multiply the current output voltage signal by the demodulated signal of the corresponding frequency, integrate it according to the optimized integration points to obtain an error signal, and then perform phase correction on the fiber laser according to the error signal.

8. The multi-jitter phase-locked control optimization device based on the number of integral points according to claim 7, characterized in that: The current output voltage signal is the peak-to-peak value of the output voltage.

9. The multi-jitter phase-locked control optimization device based on the number of integral points according to claim 7, characterized in that: The reinforcement learning controller includes a reinforcement learning training module and an optimization model application module; The reinforcement learning training module is used to perform the following training iteration process: In one iteration, a multi-jitter method is used to perform phase-locked control on the fiber laser to obtain a state at time t; the state at time t represents a peak-to-peak value of the output voltage; based on - Greedy strategy, determining the action at time t according to the state at time t and the Q table; the Q table is used to store the value estimation of the state at time t and the action at time t; the action at time t represents the optimization integral points; Based on the action at time t, the fiber laser is phase corrected to obtain the state at time t+1 and the corresponding reward at time t; Based on the state at time t, the action at time t, the state at time t+1 and the reward at time t, the Q table is updated using the Bellman equation; If the preset iteration stop condition is not met, the next iteration is performed based on the updated Q table; if the preset iteration stop condition is met, the iteration is stopped and the updated Q table is marked as the preset optimization model.

Images