Laser coherent combination method based on double-flow network and reinforcement learning framework

Abstract

The invention relates to the technical field of laser coherent combination, and discloses a laser coherent combination method based on a double-flow network and a reinforcement learning framework, which is used for solving the problem of low robustness of a machine learning phase control method. According to the method, the efficiency and robustness of laser coherent synthesis are improved by constructing a double-flow network structure and utilizing information interaction fusion of an interference pattern of a non-focal plane and a focal plane pattern. In the training process, an ActorCritic framework in reinforcement learning is used for updating and optimizing the double-flow network, so that the Actor network can adaptively adjust the phase of each laser beam, and efficient laser coherent combination is realized. The method provided by the invention can effectively realize the phase control of the laser, realizes the high-power output of the coherent combination system, and has high robustness and real-time performance.

Description

Technical Field

[0001] The present invention relates to the technical field of laser coherent synthesis, and in particular to a laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework. Background Art

[0002] Fiber lasers are widely used in military, medical and other fields due to their compact structure, high efficiency, strong portability and excellent beam quality. However, as the power of a single fiber laser continues to increase, physical limitations may lead to a decrease in beam quality. To solve this problem, coherent beam synthesis technology has become an effective method. By synchronizing the phases of the sub-beams of multiple fiber lasers, coherent maximum output is achieved, thereby maximizing the overall output power while ensuring the beam quality.

[0003] However, in actual engineering, due to various noises and interferences, the phase synchronization and stability of sub-beams face challenges, so phase control technology must be used to ensure the optimization of the combination efficiency. Active phase locking is one of the commonly used phase control technologies. It compensates for dynamic phase errors through phase detection and feedback servo control systems to ensure that each sub-beam achieves coherent output in phase and achieves the best effect.

[0004] Active phase-locking methods include heterodyne method, multi-jitter method, stochastic parallel gradient algorithm (SPGD) and other methods. In particular, the SPGD algorithm, as a model-free optimization algorithm based on artificial intelligence, has broad application prospects. The SPGD algorithm does not need to establish a complex mathematical model or strictly restrict variables. It only needs to obtain the far-field light intensity characteristics after synthesis through a photodetector, without measuring the phase of each unit beam. It is simple to implement, low cost, highly scalable and real-time, and has become the mainstream method in the field of phase locking. However, the control bandwidth of the classical active method decreases significantly with the increase in the number of sub-beams, especially in large-scale coherent combination systems (CBC), the bandwidth of phase control is still a challenge.

[0005] In recent years, with the continuous development of deep learning technology, studies have shown that the application of deep learning in active phase-locking technology is feasible, especially supervised learning. However, supervised learning relies on a large number of labels to train a neural network with perfect performance, which is very difficult and cumbersome in practical applications. Environmental changes can also easily affect the performance of deep learning models, resulting in the need for frequent retraining of neural networks, which consumes a lot of time and resources. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework, which can realize the coherent synthesis of multiple lasers, and can simultaneously solve the problems of small control bandwidth of traditional methods, difficulty in acquiring labels of deep learning methods, and long training time of reinforcement learning, while ensuring the real-time performance of the coherent synthesis system.

[0007] The technical solution of the present invention to solve the above technical problems is:

[0008] A laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework comprises the following steps:

[0009] (S1), splitting the seed laser source into N beams through a beam splitter and inputting them into a phase modulator to control the phase;

[0010] (S2), amplifying the power of the N beams passing through the phase modulator and outputting them to a specific aperture beam combiner;

[0011] (S3), after the N laser beams emitted from the specific beam combiner are reflected by the semi-transparent and semi-reflective mirror, a part of the energy is used as the system output, and the other part is reflected to the lens for focusing, and the photodetector CCD1 is placed at the focal point to obtain the diffraction pattern I1 of the focal plane after the coherent combination of the N laser beams, and the photodetector CCD2 is placed at the non-focal point to obtain the diffraction pattern I2 of the non-focal plane after the coherent combination of the N laser beams;

[0012] (S4) Input the obtained diffraction pattern into the trained intelligent agent, and output N phase control signals to the phase controller to correct the phase of each laser so as to improve the coherent synthesis efficiency of the N laser beams.

[0013] Preferably, in step (S4), the training of the agent comprises the following steps:

[0014] (S4-1) Build two networks, Actor and Critic. The Actor network is used to output the probability density function of N correction phases, and the Critic network is used to evaluate the current state.

[0015] (S4-2) Stack the two diffraction images obtained by the photodetector into a state input, and input the feature vector obtained by inputting it into the dual-stream network into the Actor network and the Critic network. After the Actor network outputs N probability density functions of the corrected phase, the N probability density functions are sampled respectively to obtain N beam phase correction signals, and the signals are fed back to the N phase modulators in the optical system to obtain a new state and obtain the corresponding reward;

[0016] (S4-3), the Actor network and the Critic network back-propagate the gradient value of the loss function according to the reinforcement learning proximal strategy optimization algorithm to update the weights and bias parameters of the neural network, output the phase correction signal again, feed the signal back to the phase modulator, repeat steps (S4-2) and (S4-3), update the neural network parameters multiple times, and stop updating when the coherent synthesis evaluation function value is greater than the preset value.

[0017] (S4-4) When the number of synthesis paths is increased, the weights of the N-1 paths that have been trained are loaded into the model as pre-trained weights. The final output layer does not use the pre-trained weights. Steps (S4-1) to (S4-3) are repeated to obtain an intelligent agent with N-path coherent synthesis training.

[0018] Preferably, in step (S4-2), the two diffraction images acquired by the photodetector are stacked into a state input, including but not limited to a focal plane-non-focal plane image pair and a before and after modulation image pair. The state shape is 2*64*64, and is input into a dual-stream network. After the dual-stream network obtains 256 feature vectors, they are input into their respective decoupling heads. The decoupling head of the Actor network is a fully connected layer and two parallel fully connected layers that map the output to the parameters of the probability density function of the action. The decoupling head of the Critic network is two fully connected layers, and the last layer maps the output to a specific value as an estimate of the state value.

[0019] Preferably, in step (S4-2), the dual-stream network consists of two parameter-sharing input streams and a feature exchange module AQC. The image pairs enter from the two input streams respectively, and then pass through the AQC module to output 256 feature vectors. The structure of the input stream consists of three stages: the first part is the Stem layer, which consists of two 3*3 convolutional neural networks and one SPPF module. The input is downsampled to one-fourth of the original, and the number of channels becomes 128; the second part is the Stage1 layer, which consists of a 3*3 convolutional neural network and one SPPF module. The input is downsampled to one-half of the original, and the number of channels becomes 256; the third part is the Stage2 layer, which consists of a 3*3 convolutional neural network and one SPPF module. The input is downsampled to one-half of the original, and the number of channels becomes 256.

[0020] Preferably, in step (S4-2), the AQC module input is two 1 / 16 downsampled feature maps, which are respectively passed through two 3*3 convolutional neural networks to generate Q, K, and V vectors. The K and Q vectors are point-multiplied and then passed through the Sigmoid function and the corresponding elements of the V vector, and then subjected to global average pooling to obtain 256 feature vectors.

[0021] Preferably, in step (S4-2), the reward expression is as follows:

[0022] reward = αPIB t +β(PIB t -PIB t-1 ) (1)

[0023] Among them, α and β are adjustable parameters, PIB t is the normalized power value in the bucket at the current moment, PIB t-1 is the normalized power value in the bucket at the previous moment.

[0024] Preferably, in step (S4-3), the coherent synthesis evaluation function includes but is not limited to the bucket power detected by the photodetector, the maximum output power, the main lobe power, the synthetic beam quality factor and a combination of the above physical quantities.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. The present invention adopts a network structure with a depth of 6 layers, and introduces a dual-stream network and an AQC module to achieve fast fitting through parameter sharing techniques, thereby adapting to the rapid convergence of the reinforcement learning algorithm. The dual-stream network structure not only improves the robustness of the model under environmental changes, but also improves the stability when the position of the non-focal plane detector changes slightly.

[0027] 2. The present invention also solves the problem of being unable to obtain labels in practical applications through a reinforcement learning update method, which makes it possible to effectively train neural networks. During the training process, there is no need for excessive human intervention, nor is there a need to prepare a large number of labels in advance. The algorithm can be directly deployed in the actual environment of the coherent synthesis system, and the neural network can continuously update parameters through multiple random phase inputs. After the training is completed, only a part of the Actor network needs to be extracted as an intelligent agent to achieve efficient phase prediction and correction. In addition, the application of the dual-stream network further improves the robustness of the algorithm.

[0028] 3. Compared with other machine learning methods, the training time of the present invention is no longer increased at an exponential level. By using pre-trained weights, the training time is reduced to a linear level. At the same time, by using the input method of two different diffraction images, the state uniqueness and convergence of the network training are ensured, which provides higher feasibility and efficiency for the laser coherent synthesis method based on reinforcement learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a system block diagram of a laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework of the present invention.

[0030] Figure 2 It is a schematic diagram of the structure of the dual-flow network of the present invention.

[0031] Figure 3 It is a schematic diagram of the structure of each module of the present invention.

[0032] Figure 4 The network structure is composed of (Table 1). DETAILED DESCRIPTION

[0033] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0034] See also Figure 1 , the implementation of a laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework of the present invention includes the following steps:

[0035] (S1), splitting the seed laser into 7 beams through a beam splitter and inputting them into a phase modulator to control the phase;

[0036] (S2), power amplifying the 7 light beams passing through the phase modulator and outputting them to a specific aperture beam combiner;

[0037] (S3), after the 7 laser beams emitted from the specific beam combiner are reflected by a semi-transparent and semi-reflective mirror, a part of the energy is used as the system output, and the other part is reflected to the lens for focusing, and the photodetector CCD1 is placed at the focal point to obtain the diffraction pattern I1 of the focal plane after the 7 laser beams are coherently combined, and the photodetector CCD2 is placed at the non-focal point to obtain the diffraction pattern I2 of the non-focal plane after the 7 laser beams are coherently combined;

[0038] (S4) Input the obtained diffraction pattern into the trained Actor network, and output 7 phase control signals to the phase controller to correct the phase of each laser, thereby improving the coherent synthesis efficiency of the 7 laser beams.

[0039] In step (S4), the training of the Actor network includes the following steps:

[0040] (S4-1) Build two networks, Actor and Critic. The Actor network is used to output the probability density function of N correction phases, and the Critic network is used to evaluate the current state.

[0041] (S4-2) Stack the two diffraction images obtained by the photodetector into a state input, and input the feature vector obtained by inputting it into the dual-stream network into the Actor network and the Critic network. After the Actor network outputs N probability density functions of the corrected phase, the N probability density functions are sampled respectively to obtain N beam phase correction signals, and the signals are fed back to the N phase modulators in the optical system to obtain a new state and obtain the corresponding reward;

[0042] (S4-3), the Actor network and the Critic network back-propagate the gradient value of the loss function according to the reinforcement learning proximal strategy optimization algorithm to update the weights and bias parameters of the neural network, output the phase correction signal again, feed the signal back to the phase modulator, repeat steps (S4-2) and (S4-3), update the neural network parameters multiple times, and stop updating when the coherent synthesis evaluation function value is greater than the preset value.

[0043] (S4-4) When the number of synthesis paths is increased, the weights of the N-1 paths that have been trained are loaded into the model as pre-trained weights. The final output layer does not use the pre-trained weights. Steps (S4-1) to (S4-3) are repeated to obtain an intelligent agent with N-path coherent synthesis training.

[0044] See also Figure 2 As shown in Table 1, the dual-stream network consists of two parameter-sharing input streams and a feature exchange module AQC. The image pairs enter from the two input streams respectively, and then pass through the AQC module to output 256 feature vectors. The structure of the input stream consists of three stages: the first part is the Stem layer, which consists of two 3*3 convolutional neural networks and one SPPF module. The input is downsampled to one-fourth of the original, and the number of channels becomes 128; the second part is the Stage1 layer, which consists of a 3*3 convolutional neural network and an SPPF module. The input is downsampled to one-half of the original, and the number of channels becomes 256; the third part is the Stage2 layer, which consists of a 3*3 convolutional neural network and an SPPF module. The input is downsampled to one-half of the original, and the number of channels becomes 256. The decoupling head of the Actor network is a fully connected layer and two parallel fully connected layers that map the output to the parameters of the probability density function of the action; the decoupling head of the Cr it ic network is two fully connected layers, and the last layer maps the output to a specific value as an estimate of the state value.

[0045] See also Figure 3 The input of the AQC module is two 16th-downsampled feature maps, which are respectively passed through two 3*3 convolutional neural networks to generate Q, K, and V vectors. After the K and Q vectors are dot-multiplied, they are passed through the Sigma function and then multiplied with the V vector elements correspondingly, and then global average pooling is performed to obtain 256 feature vectors. The input of the SPPF module is a feature map, and the channel dimension is reduced or features are fused through a 1*1 convolutional neural network, thereby reducing the amount of calculation and improving the compactness of feature expression; then, the feature map passes through three consecutive maximum pooling layers, each of which can have a different kernel size; next, the output of the three maximum pooling operations is spliced ​​with the output of the original 1x1 convolution in the channel dimension; finally, the spliced ​​features pass through a 1x1 convolution layer, and feature fusion or dimension increase is performed again to output the final feature map.

[0046] In addition, in step (S4-2), the state input is obtained by the photodetector CCD1 of the focal plane and the photodetector CCD2 of the non-focal plane, and the two images are input into the network as a state. Wait for the next action input and repeat the process.

[0047] In addition, in step (S4-2), the reward expression is as follows:

[0048] reward = αPIB t +β(PIB t -PIB t-1 ) (1)

[0049] Among them, α and β are adjustable parameters, PIB t is the normalized power value in the bucket at the current moment, PIB t-1 is the normalized power value in the bucket at the previous moment.

[0050] In addition, in step (S4-3), the coherent synthesis evaluation function includes but is not limited to the bucket power detected by the photodetector, the maximum output power, the main lobe power, the synthetic beam quality factor and a combination of the above physical quantities.

[0051] The above is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited to the above content. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A laser coherent synthesis method based on a dual-stream network and reinforcement learning framework, characterized in that: The following steps are involved: (S1), splitting the seed laser source into N beams through a beam splitter and inputting them into a phase modulator to control the phase; (S2), amplifying the power of the N beams passing through the phase modulator and outputting them to a specific aperture beam combiner; (S3), after the N laser beams emitted from the specific beam combiner are reflected by the semi-transparent and semi-reflective mirror, a part of the energy is used as the system output, and the other part is reflected to the lens for focusing, and the photodetector CCD1 is placed at the focal point to obtain the diffraction pattern I1 of the focal plane after the coherent combination of the N laser beams, and the photodetector CCD2 is placed at the non-focal point to obtain the diffraction pattern I2 of the non-focal plane after the coherent combination of the N laser beams; (S4) Input the obtained diffraction pattern into the trained intelligent agent, and output N phase control signals to the phase controller to correct the phase of each laser so as to improve the coherent synthesis efficiency of the N laser beams.

2. The laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework according to claim 1, characterized in that: The two diffraction images obtained by the photodetector are stacked into a state input, including but not limited to the focal plane-non-focal plane image pair and the image pair before and after modulation, and input into the two-stream network. After the two-stream network, 256 feature vectors are obtained and then input into their respective decoupling heads. The decoupling head of the Actor network is a fully connected layer and two parallel fully connected layers that map the output to the parameters of the probability density function of the action. The decoupling head of the Critic network is two fully connected layers, and the last layer maps the output to a specific value as an estimate of the state value.

3. The laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework according to claim 2, characterized in that: The dual-stream network consists of two parameter-sharing input streams and a feature exchange module AQC. The image pairs enter from the two input streams respectively, and then pass through the AQC module to output 256 feature vectors. The structure of the input stream consists of three stages: the first part is the Stem layer, which consists of two 3*3 convolutional neural networks and one SPPF module. The input is downsampled to one-fourth of the original, and the number of channels becomes 128; the second part is the Stage1 layer, which consists of a 3*3 convolutional neural network and an SPPF module. The input is downsampled to one-half of the original, and the number of channels becomes 256; the third part is the Stage2 layer, which consists of a 3*3 convolutional neural network and an SPPF module. The input is downsampled to one-half of the original, and the number of channels becomes 256.

4. The laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework according to claim 2, characterized in that: The input of the AQC module is two feature maps sampled at a factor of 16, which are respectively passed through two 3*3 convolutional neural networks to generate Q, K, and V vectors. The K and Q vectors are dot-multiplied and then passed through the Sigmoid function and multiplied with the corresponding elements of the V vector. After that, 256 feature vectors are obtained through global average pooling.

5. The laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework according to claim 1, characterized in that: In step (S4), obtaining the trained agent includes the following steps: (S4-1) Build two networks, Actor and Critic. The Actor network is used to output the probability density function of N correction phases, and the Critic network is used to evaluate the current state. (S4-2) Stack the two diffraction images obtained by the photodetector into a state input, and input the two-stream network to obtain the feature vector, which is then input into the Actor network and the Critic network. After the Actor network outputs N probability density functions of the corrected phase, the N probability density functions are sampled respectively to obtain N beam phase correction signals, which are then fed back to the N phase modulators in the optical system to obtain a new state and obtain the corresponding reward. (S4-3), the Actor network and the Critic network back-propagate the gradient value of the loss function according to the reinforcement learning proximal strategy optimization algorithm to update the weights and bias parameters of the neural network, output the phase correction signal again, feed the signal back to the phase modulator, repeat steps (S4-2) and (S4-3), update the neural network parameters multiple times, and stop updating when the coherent synthesis evaluation function value is greater than the preset value. (S4-4) When the number of synthesis paths is increased, the weights of the N-1 paths that have been trained are loaded into the model as pre-trained weights. The final output layer does not use the pre-trained weights. Steps (S4-1) to (S4-3) are repeated to obtain an intelligent agent with N-path coherent synthesis training.

6. The laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework according to claim 5, characterized in that: The reward expression is as follows: reward=αGDP t +β(GDP t -GDP t-1 ) (1) Among them, α and β are adjustable parameters, PIB t is the normalized power value in the bucket at the current moment, PIB t-1 is the normalized power value in the bucket at the previous moment.

7. The laser coherent synthesis method based on a dual-stream network and a reinforcement learning framework according to claim 5, characterized in that: The coherent synthesis evaluation function includes but is not limited to the power in the bucket detected by the photodetector, the maximum output power, the main lobe power, the synthesized beam quality factor and the combination of the above physical quantities.

Images