Laser beam positioning method based on deep neural network

Abstract

The invention discloses a laser beam positioning method based on a deep neural network. The method comprises the following steps: step 1, acquiring a light spot image of a laser beam on a reflector through image acquisition equipment; step 2, a deep neural network DeepLabV3 + model is utilized to perform pixel-by-pixel classification on the light spot image, light spot pixels and non-light-spot pixels are distinguished, a light spot mask is generated, the foreground pixel value of the light spot mask is set to be 1, and the background pixel value of the light spot mask is set to be 0; and step 3, performing morphological processing on the generated light spot mask, and calculating a centroid coordinate of the light spot by using a centroid method. The method has high noise immunity and generalization ability, can accurately and reliably extract the position and area of the light spot, and can give an accurate recognition result under the condition that the laser light spot is abnormal.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and specifically relates to a laser beam positioning method based on a deep neural network. Background Art

[0002] Laser Thomson scattering diagnostics (TS) is the main method used to measure plasma electron density and temperature on the EAST Tokamak device, and is widely recognized as the most accurate measurement method internationally. This diagnostic method determines the characteristics of electrons in the plasma by analyzing the scattered light after the scattered photons interact with the electrons. This method provides accurate measurements of the internal properties of the plasma, which is crucial for understanding plasma behavior and optimizing the performance of plasma equipment.

[0003] The laser beam plays a very important role in the Thomson scattering diagnostic system because it is the main source of scattered photons. Whether the laser beam can be transmitted to a specific area of ​​the plasma, as well as the stability and accuracy of the transmission, have an important impact on the performance and measurement accuracy of the diagnostic system. However, affected by mechanical creep, air turbulence and laser thermal effects, the laser beam often has continuous drift and jitter, which affects the measurement results of the TS system and reduces the accuracy and reliability of the Thomson scattering diagnostic system.

[0004] The laser beam in the TS travels 40 meters through a curved transmission system with eight mirrors to reach the target area in the vacuum chamber. Accurately identifying the position of the laser beam at various locations in the TS system is a prerequisite for correcting the laser offset. In addition, in the TS beam transmission system, each mirror is coated with a strict reflective film layer, and the maximum reflection efficiency can only be obtained at a specific angle of incidence. Therefore, while ensuring the accuracy at the final target, it is necessary to accurately control the posture of each mirror in the transmission system to ensure the efficiency and accuracy of the overall laser beam transmission system.

[0005] To locate the position of the laser beam at each reflector, a camera is usually used to collect the spot image and perform image analysis. There are two main ways to install the camera. One is to install it behind the mirror to capture the leaked laser beam, but this requires a complex focusing optical system and high space requirements; the other is to use a beam splitter on the laser side for collection, but this will affect the transmission efficiency and have a negative impact on the signal-to-noise ratio of TS. In addition, the EAST device site has a complex and changeable electromagnetic environment. The closer the camera is to the device and the longer it is installed, the more white noise points there are. In addition, the background and background light of different camera pictures are also quite different. The TS system has multiple lasers. Due to different hardware, the laser of each laser will also have differences in size and brightness, and will be switched according to the requirements of the EAST experiment. These problems pose considerable challenges to the accurate and automatic identification of the spot. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a laser beam positioning method based on a deep neural network. This method introduces the deep neural network DeepLabV3+ into the field of laser spot detection, and makes corresponding improvements. It realizes the automatic segmentation of the spot through deep learning, and combines traditional geometric methods (such as the centroid method) to accurately calculate the center position of the spot. Specifically, the DeepLabV3+ network adopts advanced semantic segmentation technology, which can effectively extract the pixel information of the spot and generate the corresponding spot mask. Subsequently, the centroid method is used to calculate the centroid coordinates and area of ​​the spot based on the generated spot mask, thereby realizing accurate identification of the spot position and size. This innovative solution significantly improves the accuracy and efficiency of laser spot detection, and provides reliable technical support for the laser automatic alignment of the Thomson scattering diagnostic system of the EAST device and various laser application scenarios. It has broad market prospects, especially in practical applications such as laser beam positioning and automatic correction.

[0007] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0008] A laser beam positioning method based on a deep neural network, the method comprising:

[0009] Step 1, obtaining a spot image of the laser beam on the reflector through an image acquisition device;

[0010] Step 2: Use the trained deep neural network DeepLabV3+ model to classify the spot image pixel by pixel, distinguish spot pixels from non-spot pixels, and generate a spot mask, where the foreground pixel value of the spot mask is set to 1 and the background pixel value is set to 0;

[0011] Step 3: Perform morphological processing on the generated light spot mask and calculate the center of mass coordinates of the light spot using the center of mass method.

[0012] Furthermore, in step 1, the spot image is collected based on a Thomson scattering diagnostic system of the EAST tokamak device.

[0013] Furthermore, in step 2, the network structure of the deep neural network DeepLabV3+ model includes an encoder part and a decoder part.

[0014] Furthermore, the encoder part includes a ResNet101 module as a backbone network, a hole convolution module, and a 1×1 convolution layer.

[0015] Furthermore, the ResNet-101 module is used to extract low-level semantic features from the input spot image and generate high-level semantic features;

[0016] The atrous convolution module performs multi-scale sampling on the high-order semantic features to obtain multi-scale features;

[0017] The multi-scale features are concatenated in the channel dimension, and the channel dimension is reduced through a 1×1 convolutional layer to obtain multi-scale semantic features.

[0018] Furthermore, the dilated convolution module includes multiple parallel 3×3 dilated convolution branches with different dilation rates, which are used to extract multi-scale features of the input spot image.

[0019] Furthermore, the decoder uses jump connections to splice the low-level semantic features extracted by the encoder with the multi-scale semantic features output by the decoder. The spliced ​​feature map is refined by 3×3 convolution and upsampled by a 4-fold upsampling factor using bilinear interpolation to output a spot mask with the same resolution as the input spot image.

[0020] Furthermore, the trained deep neural network DeepLabV3+ model includes a data enhancement method used during training, and the data enhancement method includes performing random brightness adjustment, noise addition, and simulated occlusion on the spot image.

[0021] Furthermore, the morphological processing performed on the generated spot mask in step 3 includes corrosion, expansion, opening operation and closing operation.

[0022] Furthermore, the centroid method is used to calculate the centroid coordinates of the light spot in step 3, including extracting a coordinate set of the light spot pixels through the binarized light spot mask, respectively calculating the weighted average values ​​of the horizontal and vertical pixel coordinates of the light spot area, and determining the position of the light spot centroid.

[0023] The beneficial effects of the present invention are:

[0024] The present invention proposes a laser beam positioning method based on a deep neural network, which effectively overcomes the limitations of the traditional binary threshold method. Traditional methods are often difficult to work effectively under complex light spot shapes or background interference. The present invention uses deep learning technology to automatically identify the light spot area, thereby significantly reducing manual intervention and improving positioning accuracy and automation. This method can accurately and reliably extract the light spot position and area under a variety of actual conditions in the laser Thomson scattering diagnostic optical path of the EAST device and at each reflector, and can adapt to different laser types, reflectors and various background light conditions.

[0025] In addition, the present invention can still provide accurate recognition results when the laser spot is not visible, demonstrating its excellent robustness. This technology is not only suitable for a variety of application scenarios such as laser beam tracking, laser calibration and laser processing positioning, but also has a wide range of industrial and scientific research application prospects, which can promote the further development of laser technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a laser beam positioning method based on a deep neural network of the present invention;

[0027] Figure 2 This is a diagram of the deep neural network architecture of the present invention;

[0028] Figure 3 The light spot recognition result at each reflector of the TS system of the present invention;

[0029] Figure 4 This is a result diagram of laser beam position recognition according to the present invention;

[0030] Figure 5 This is a diagram of the light spot recognition result under special circumstances of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0032] As mentioned above, the present invention proposes a laser beam positioning method based on a deep neural network, which captures the laser spot image on the laser beam side of the reflector. The CCD camera used in this method directly faces the reflector to shoot the scattered light and obtain the spot image. There is no need for a complex optical focusing system, and it has the advantages of small installation space requirements, large field of view, and no impact on the overall laser transmission efficiency. It is particularly suitable for environments with limited space, and can meet the needs of automatic correction when the laser beam occasionally deviates greatly. The method integrates a deep neural network and a centroid method to locate the spot, wherein the deep neural network implements pixel classification of the spot image in a semantic segmentation manner and extracts the effective shape of the spot, while the centroid rule is used to calculate the centroid of the spot based on the extraction results of the previous step, thereby realizing the simultaneous calculation of the centroid and area of ​​the spot. Finally, several existing laser spot position extraction methods, such as the grayscale centroid method, the Otsu method, and the histogram, were compared on the test data set. Figure 3 The angle method, iteration method and adaptive method were compared and tested, and the results showed that the method of the present invention can adapt to the complex field environment of the TS system and accurately locate the laser beam position from various TS spot images. Specifically, it includes the following steps:

[0033] Step 1, obtaining a spot image of the laser beam on the reflector through an image acquisition device;

[0034] Step 2: Use the trained deep neural network DeepLabV3+ model to classify the spot image pixel by pixel, distinguish spot pixels from non-spot pixels, and generate a spot mask, where the foreground pixel value of the spot mask is set to 1 and the background pixel value is set to 0;

[0035] Step 3: Perform morphological processing on the generated light spot mask and calculate the center of mass coordinates of the light spot using the center of mass method.

[0036] like Figure 1 As shown, in the laser beam positioning method of the present invention, a spot image is obtained, that is, the original image collected by the real TS optical path is used as input, and the improved deep neural network DeepLabV3+ in the present invention is used to extract the spot pixels and generate the mask of the spot (that is, the effective spot area), and finally the final result is calculated by the centroid method, including the center coordinates and area of ​​the spot. Specifically, the spot image is collected on the Thomson scattering diagnostic system of the EAST Tokamak device. The laser beam transmission optical path of the system includes 7 reflectors, and the CCD camera used is equipped with a filter to directly face the reflector to shoot the scattered light and obtain the spot image. This method does not require a complex optical focusing system, has the advantages of small installation space requirements and a large field of view, is particularly suitable for environments with limited space, and can meet the automatic correction requirements when the laser beam occasionally deviates greatly.

[0037] like Figure 2 As shown in the figure, the network structure of the deep neural network DeepLabV3+ model includes an encoder part and a decoder part, where:

[0038] (1) The encoder part uses the ResNet-101 backbone network (Backbone) as the feature extraction network to extract the spatial and semantic features of the input light spot image layer by layer. In order to enhance the feature extraction capability, a deeper EfficientNet feature extraction network is introduced. Its dense connection structure effectively eliminates the gradient vanishing problem and enhances the feature representation capability. In addition, the atrous convolution technology is applied in the backbone network. By setting the expansion rate, the receptive field of the convolution kernel is expanded to capture feature information of different scales, while adapting to the diversity of light spot morphology and avoiding increasing the amount of calculation. In order to better focus on global information, a spatial attention module is added after each convolution module of the ResNet101 backbone network and after the output layer of the ASPP module to help the network pay more attention to those areas that are more important in space, thereby improving the effect of light spot segmentation.

[0039] (2) The ASPP module (Atrous Spatial Pyramid Pooling) is used in the encoder part to extract multi-scale features of the light spot area through multiple 3×3 dilated convolutions with different dilation rates. At the same time, 1×1 and 5×5 dilated convolutions are added to capture a wider range of contextual information. In addition, the dilated attention mechanism is introduced to make the network pay more attention to the light spot area during feature extraction, thereby improving the robustness to complex backgrounds. At the same time, the module contains a 1×1 standard convolution to reduce the channel dimension, and combines the global average pooling branch to obtain the global contextual information of the image. The feature maps of each branch are concatenated (Concatenation), and the features are fused through 1×1 convolution to generate a high-quality feature map.

[0040] (3) The decoder part gradually restores the high-level feature map output by the ASPP module to the same spatial resolution as the input image through upsampling operations. In this process, the attention mechanism is combined to adjust the channel importance during upsampling so that more attention is paid to the light spot area when restoring the features, thereby ensuring that fine edge and detail information is not lost. In addition, skip connections are used to splice the low-level features extracted from the shallow layer of the encoder with the decoder feature map to retain the detail information of the light spot edge. The spliced ​​feature map is further refined by 3×3 convolution to improve the accuracy of the segmentation result, and bilinear interpolation is used to output the light spot area mask with the same resolution as the original image with a 4-fold upsampling factor.

[0041] (4) To enhance the robustness of the model, this method introduces data augmentation strategies during the training process, including random brightness adjustment, noise addition, and simulated occlusion on the light spot image. Specifically, random brightness adjustment changes the brightness range of the image so that the model can adapt to light spot recognition under different lighting conditions; noise addition introduces Gaussian noise or salt and pepper noise into the image so that the model can still maintain performance when processing noisy images. In addition, simulated occlusion technology randomly selects some areas for occlusion to reproduce scenes that may appear in the real environment, helping the model learn the characteristics of light spots under different backgrounds and occlusion conditions, thereby enhancing its real-time recognition and positioning capabilities.

[0042] Finally, through a series of convolution, concatenation, and upsampling operations, the decoder is able to integrate high-level semantic features from the encoder into lower-level detail features to achieve pixel-level segmentation output, that is, accurate classification of each pixel in the image.

[0043] After the pixel-level segmentation step, we can get the image of the light spot, map it back to the original image, and calculate the center coordinates of the light spot using the centroid method:

[0044] ,

[0045] ,

[0046] Among them, i and j are the coordinates of the corresponding pixels, is the value of the pixel in row i and column j in the mask.

[0047] The present invention uses various lasers and manually adjusts the position of the laser spot on all mirrors, collects 400 images and manually annotates them. In order to simulate the interference of laser, camera and background light in TS, data enhancement techniques are applied, including affine transformation, random exposure, random cropping, etc. This produces a dataset of 3200 images. In order to ensure that the training data is very close to the actual data, different types of noise (both additive and multiplicative) are introduced in the training samples. During the training process, 70% of the dataset is used to train the laser point extractor, and the remaining 30% is retained as a validation dataset.

[0048] The model used in this paper was built using PyTorch, using the Adam optimizer with an initial learning rate of 0.001. At the end of each epoch, the learning rate was reduced to 80% of its original level. The loss function used was Dice loss. All experiments were performed on NVIDIA RTX A4000 16G with a batch size of 32 and a maximum of 300 iterations. Training was terminated when the validation set loss started to increase while the training set loss stopped decreasing.

[0049] In order to evaluate the performance of the laser spot position calculation module, a test data set of 1,000 spot images was collected. These images were taken at different times during the EAST experiment, using different lasers and manually adjusting the mirror angle to simulate the transformation and drift of the spot in the real scene. This test set was used to test the TS spot position recognition algorithm trained according to the above method. Figure 3 Part of the recognition results are shown, showing the spot images at the reflectors M1 to M7 and the spot position results recognized using the method introduced in the present invention (indicated by red cross lines in the figure).

[0050] The root mean square error (RMSE) is a common indicator for evaluating the difference between the predicted value and the actual value. Here it will be used to quantitatively evaluate the performance of the laser spot position recognition algorithm. RMSE can be expressed as:

[0051] ,

[0052] Where n represents the total number of samples. It is the reference coordinate value of the spot. It is achieved by manually selecting the ROI, isolating the region of interest containing the laser spot in the image, and applying the best binary segmentation threshold to separate the spot from the background. Subsequently, the ellipse fitting method is used to approximate the spot area and fit an ellipse that best fits the laser spot, thereby calculating the centroid coordinates of the spot. It is the laser spot coordinates identified using the method introduced in the present invention.

[0053] The method of the present invention is compared with five mature methods: grayscale centroid method, Otsu method, histogram method, Figure 3 The results are shown in Table 1 (RMSE value comparison of laser beam position recognition results) and Figure 4 As shown. It can be clearly seen that the results automatically calculated by the method of the present invention are more consistent with the standard results. As shown in Table 1, the RMSE values ​​of the x and y coordinates calculated by the method of the present invention are 5.55 and 9.16, respectively, which are the lowest among all the tested methods, indicating that the method of the present invention is more suitable for the calculation of the position of the TS spot.

[0054] Table 1

[0055]

[0056] Furthermore, the algorithm was verified to be robust in situations where the laser spot image was subject to strong stray light, incomplete spots, or missing spots. Despite these challenging conditions, the algorithm consistently produced accurate results, such as Figure 5 shown.

[0057] Experimental results show that the method of the present invention can accurately and reliably extract the position and area of ​​the light spot in the Thomson scattering system on the EAST device under different laser signals, different reflectors and variable background light conditions. Even when the light spot is not visible, the method can provide accurate recognition results, demonstrating its excellent noise resistance and generalization ability.

[0058] In summary, the laser beam positioning algorithm proposed in the present invention has strong robustness and stability: the traditional grayscale centroid method, Otsu method, histogram Figure 3Methods such as the angle method, iteration method, and adaptive method rely on global thresholds and are easily disturbed by stray light or noise in the spot image, resulting in unsatisfactory binarization effects, especially when the spot boundary is blurred or there is a complex background. Using the Deeplab V3+ network, more accurate segmentation of complex spot shapes, light interference, and other situations can be performed through training; High degree of automation: Traditional methods require manual selection of thresholds or parameter adjustment, while the method in the present invention uses an improved deep learning network to automatically learn and generate spot area masks, reducing manual intervention; Multi-scale feature extraction: Deeplab V3+ uses the ASPP module for multi-scale hole convolution, which can extract different scale information of the spot and better identify the spot area in a complex background, while traditional methods cannot achieve this multi-scale feature fusion.

[0059] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A laser beam positioning method based on deep neural network, characterized in that: The method comprises: Step 1, obtaining a spot image of the laser beam on the reflector through an image acquisition device; Step 2: Use the trained deep neural network DeepLabV3+ model to classify the spot image pixel by pixel, distinguish spot pixels from non-spot pixels, and generate a spot mask; Step 3: Perform morphological processing on the generated light spot mask and calculate the center of mass coordinates of the light spot using the center of mass method.

2. The laser beam positioning method based on deep neural network according to claim 1, characterized in that: In the step 1, the light spot image is collected based on the Thomson scattering diagnostic system of the EAST Tokamak device.

3. The laser beam positioning method based on deep neural network according to claim 1, characterized in that: In step 2, the network structure of the deep neural network DeepLabV3+ model includes an encoder part and a decoder part.

4. The laser beam positioning method based on deep neural network according to claim 3, characterized in that: The encoder part includes a ResNet101 module as a backbone network, a hole convolution module, and a 1×1 convolution layer.

5. The laser beam positioning method based on deep neural network according to claim 4, characterized in that: The ResNet-101 module is used to extract low-level semantic features from the input spot image and generate high-level semantic features; The atrous convolution module performs multi-scale sampling on the high-order semantic features to obtain multi-scale features; The multi-scale features are concatenated in the channel dimension, and the channel dimension is reduced through a 1×1 convolutional layer to obtain multi-scale semantic features.

6. The laser beam positioning method based on deep neural network according to claim 5, characterized in that: The dilated convolution module includes multiple parallel 3×3 dilated convolution branches with different dilation rates, which are used to extract multi-scale features of the input spot image.

7. The laser beam positioning method based on deep neural network according to claim 3, characterized in that: The decoder uses jump connections to splice the low-level semantic features extracted by the encoder with the multi-scale semantic features output by the decoder. The spliced ​​feature map is refined through 3×3 convolution to achieve pixel-level segmentation output, and bilinear interpolation is used to upsample with a 4-fold upsampling factor to output a spot mask consistent with the resolution of the input spot image.

8. The laser beam positioning method based on deep neural network according to claim 1, characterized in that: The trained deep neural network DeepLabV3+ model includes a data enhancement method used during training, wherein the data enhancement method includes performing random brightness adjustment, noise addition, and simulated occlusion on the spot image.

9. The laser beam positioning method based on deep neural network according to claim 1, characterized in that: The morphological processing performed on the generated light spot mask in step 3 includes corrosion, expansion, opening operation and closing operation.

10. The laser beam positioning method based on deep neural network according to claim 1, characterized in that: Calculating the centroid coordinates of the light spot by the centroid method in step 3 includes extracting the coordinate set of the light spot pixels through the binarized light spot mask, respectively calculating the weighted average of the horizontal and vertical pixel coordinates of the light spot area, and determining the position of the light spot centroid.

Images