Low-light image enhancement method for online visual monitoring of metal additive manufacturing process

Abstract

The present invention discloses a method for low-light image enhancement for online visual monitoring of metal additive manufacturing process: using a high-speed camera to shoot the video of the metal additive manufacturing process and exporting it frame by frame to construct low-light and bright-light image data sets; constructing a low-light image enhancement network model based on Retinex visual theory; fusing the reflection map and the illumination map into an enhanced image and calculating the loss function value to complete the iterative training of the low-light image enhancement network model; deploying the low-light image enhancement network model to an online monitoring hardware platform to calculate the image quality evaluation indicators PSNR and SSIM. The present invention overcomes the problem of high noise points and poor geometric feature extraction accuracy in online visual monitoring of metal additive manufacturing process under low-light environment. The method of the present invention can be used for online visual monitoring of metal additive manufacturing, effectively improving the quality of online monitoring images of metal additive manufacturing process, and helping to improve the online visual monitoring capability of metal additive manufacturing.

Description

Technical Field

[0001] The present invention belongs to the field of image data processing for online visual monitoring of metal additive manufacturing, and in particular relates to a weak-light image enhancement method for online visual monitoring of a metal additive manufacturing process. Background Art

[0002] As a cutting-edge technology in high-end manufacturing, metal additive manufacturing has the advantages of digitalization, automation, and high material utilization, and has received extensive attention in the fields of national defense engineering, aerospace, etc. The interaction between laser and powder in the metal additive manufacturing process makes the non-equilibrium physical metallurgy and thermal physical processes of the material very complicated, often leading to quality problems such as cracks and pores. Therefore, it is very important to monitor the metal additive manufacturing process online and improve the stability of the manufacturing process.

[0003] At present, online visual monitoring of metal additive manufacturing processes based on optical signals is a common method. Researchers use high-speed cameras to shoot videos of the manufacturing process and further extract key feature information to achieve online visual monitoring of component quality and defects. However, high-speed cameras have high frame rates and short exposure times. Filters of specific wavelengths need to be installed in front of the lens. The captured images will show lower brightness, and information is easily lost and noise is generated during imaging, resulting in low image quality. Improving the hardware performance of the shooting equipment can improve the imaging quality to a certain extent, but this will increase costs. Therefore, in the actual online visual monitoring of metal additive manufacturing processes, it is of great significance to improve the brightness of weak-light images through weak-light image enhancement algorithms.

[0004] At present, the methods of low-light image enhancement are mainly divided into the following categories: grayscale transformation, histogram equalization, frequency domain method, image fusion, defogging model and machine learning method. Traditional low-light image enhancement methods are difficult to improve the brightness of the image while retaining the clear features of the image. It is easy to have unclear image contours, dark area artifacts, color mutations, distortion, noise interference and other problems, and it is difficult to obtain satisfactory results. The machine learning-based method learns the mapping of low-light images to bright-light images from a large number of images to achieve low-light image enhancement, but it has a huge dependence on the amount of data, and the existing algorithms generally use images taken in daily life as the enhancement object. The effect is not good in the online visual monitoring scenario of metal additive manufacturing, and the algorithm performance needs to be improved, so it cannot be directly applied. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a low-light image enhancement method for online visual monitoring of metal additive manufacturing process, which effectively improves the quality of online monitoring images of metal additive manufacturing process and helps to enhance the online monitoring capability of metal additive manufacturing.

[0006] The technical solution provided by the present invention is as follows:

[0007] In a first aspect, the present invention provides a weak light image enhancement method for online visual monitoring of a metal additive manufacturing process, comprising the following steps:

[0008] Step 1: Build an online visual monitoring hardware platform for the metal additive manufacturing process and use a high-speed camera to shoot a video of the metal additive manufacturing process;

[0009] Step 2, exporting the original captured video frame by frame to construct a dataset of low-light and bright-light images; the bright-light image is obtained by brightening the low-light image;

[0010] Step 3, constructing a low-light image enhancement network model based on Retinex visual theory; the low-light image enhancement network model consists of a layer decomposition network, a reflection adjustment network and an illumination adjustment network; the layer decomposition network includes two branches, a reflection map and an illumination map; the reflection map branch is a simplified U-Net structure, which is used to extract the reflection component from the input image, and then extract and combine the low-level and high-level features to generate a reflection map; the illumination map branch includes a convolution layer, a connection layer, a ReLU layer and a Sigmoid layer, the illumination map branch obtains the feature map from the reflection map branch, extracts and enhances the illumination features of the image, and is subsequently processed to extract and enhance the illumination features of the image, and captures and adjusts the illumination changes by performing deep convolution operations and nonlinear activation on the feature map, so that the changes in illumination conditions and details are accurately simulated in the final enhancement process;

[0011] The reflection adjustment network gradually extracts and abstracts image features through a series of convolutional layers and outputs an adjusted reflection image;

[0012] The illumination adjustment network includes a local sub-network and a global sub-network. The local sub-network is used to enhance local details of the image, and the global sub-network is used to capture and adjust global illumination changes of the image. The outputs of the local sub-network and the global sub-network are combined, and the effects of detail enhancement and overall brightness adjustment are balanced by parameter weighting to generate an illumination map to restore the image.

[0013] Step 4: Train the low-light image enhancement network model, decompose the input low-light image into a reflection map and an illumination map through the layer decomposition network, and generate the enhanced reflection map and illumination map through the reflection and illumination adjustment network; calculate the peak signal-to-noise ratio PSNR and the structural similarity index SSIM, and then calculate the loss function to complete the iterative training of the low-light image enhancement network model;

[0014] Step 5: Deploy the low-light image enhancement network model to the online monitoring hardware platform and calculate the image quality evaluation indicators PSNR and SSIM.

[0015] In a possible embodiment, in step 1, the monitoring hardware platform includes a computer, an illumination light source power supply, a high-speed camera, a macro lens, an illumination light source and a metal additive manufacturing device.

[0016] In a possible embodiment, in the step 3, the reflection map branch in the layer decomposition network includes the first convolution layer + ReLU layer, the first deconvolution + ReLU layer, the first connection layer, the second convolution layer + ReLU layer, the second deconvolution + ReLU layer, the second connection layer, the third convolution layer + ReLU layer, and the convolution layer + Sigmoid function in sequence; the first convolution layer + ReLU layer is jump-connected to the first connection layer and the second connection layer respectively;

[0017] The illumination branch includes a convolution + ReLU layer, a connection layer, and a convolution layer + Sigmoid function.

[0018] In a possible embodiment, in step 1, the reflection adjustment network includes:

[0019] The initial convolutional layer is used to extract preliminary features of the image;

[0020] The second convolutional layer is used to capture a wider range of contextual information;

[0021] The third convolutional layer is used to enhance the expressiveness of features;

[0022] The fourth convolutional layer is used to fuse features while preserving details;

[0023] The last convolutional layer, using the ReLU activation function, is used to generate the final adjusted reflection image.

[0024] In a possible embodiment, in step 1, the illumination adjustment network includes a local sub-network and a global sub-network;

[0025] The global sub-network includes an average pooling layer, three convolutional layers + ReLU layers, and convolutional layers + Sigmoid functions, which are used to capture and adjust the global illumination changes of the image. The global sub-network captures the overall brightness information of the image through the average pooling layer, and uses the convolution kernel for feature mapping and combination, and finally generates a global sub-network image.

[0026] The local sub-network includes four convolutional layers + ReLU layers, convolutional layers + Sigmoid functions, which are used to refine and enhance local lighting details and map them to the output channels.

[0027] The local and global features are combined through parameter weighting to balance the effects of detail enhancement and overall brightness adjustment to generate an illumination map to restore the image.

[0028] In a possible embodiment, in step 4, the calculation formula of PSNR is as follows:

[0029]

[0030] Among them, MAX I represents the maximum value of all pixels of image I;

[0031]

[0032] Among them, the size of the original image is m×n, I(i,j) represents the pixel value of the reference image I at the coordinate (i,j), and K(i,j) represents the pixel value of the image K to be evaluated at the coordinate (i,j);

[0033] The calculation formula of SSIM is as follows:

[0034]

[0035] Among them, μ x and μ y represents the average value of image pixels, σ x and σ y represents the variance, σ xy Represents covariance, the coordinates of a point in the image are (x, y), and C1 and C2 are constants that prevent the denominator from being zero.

[0036] In a possible embodiment, in step 4, the calculation formula of the loss function is as follows:

[0037]

[0038] Among them, y true ,y pred Represent the original image and the predicted image respectively, PSNR(y true ,y pred ) represents the peak signal-to-noise ratio of the original image and the predicted image, SSIM(y true ,y pred ) represents the structural similarity index between the original image and the predicted image.

[0039] In a second aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for low-light image enhancement for online visual monitoring of metal additive manufacturing processes as described in the first aspect is implemented.

[0040] In a third aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the low-light image enhancement method for online visual monitoring of metal additive manufacturing processes as described in the first aspect.

[0041] In a fourth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the low-light image enhancement method for online visual monitoring of a metal additive manufacturing process as described in the first aspect.

[0042] The advantages of the present invention compared with the prior art are:

[0043] The present invention combines weak-light image enhancement with online visual monitoring of the metal additive manufacturing process and uses the weak-light images of the metal additive manufacturing process as a training set. When using the same feature extraction algorithm, it is beneficial to capture the key feature geometry of the image and improve the online visual monitoring capability of metal additive manufacturing.

[0044] Based on Retinex visual theory, the present invention decomposes the original low-light image into a reflection map and an illumination map and enhances them separately. Then, the enhanced reflection map and illumination map are combined to enhance the overall brightness of the image, which effectively reduces the difficulty of building and training neural networks. The illumination adjustment network uses a global-local enhancement method to process high dynamic range images to ensure uniform brightness of the output image.

[0045] The present invention reconstructs the loss function in the network training process. Compared with the traditional MSE loss function, it can effectively improve the accuracy of the model and the output image is more in line with the visual perception of the human eye. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 The present invention is a flow chart of a weak-light image enhancement method for online visual monitoring of a metal additive manufacturing process.

[0048] Figure 2 Schematic diagram of the metal additive manufacturing online visual monitoring system of the present invention.

[0049] Figure 3 It is a schematic diagram of the structure of the low-light image enhancement network model based on Retinex vision theory of the present invention.

[0050] Figure 4 This is a comparison chart of the low-light image enhancement effect of the present invention and other low-light image enhancement methods.

[0051] Figure 5 This is a comparison diagram of the effects of using a fully convolutional neural network (FCN) to extract the molten pool before and after low-light image enhancement of the present invention.

[0052] Figure 6 This is a comparison diagram of the effect of sputtering extracted using a fully convolutional neural network (FCN) before and after low-light image enhancement of the present invention. DETAILED DESCRIPTION

[0053] The present invention is further described in detail below in conjunction with the accompanying drawings and specific examples, but the content of the present invention is not limited thereto.

[0054] Example 1

[0055] like Figure 1 The implementation process of the present invention is shown as follows: a weak light image enhancement method for online visual monitoring of metal additive manufacturing process, comprising the following steps:

[0056] Step 1: Build an online monitoring hardware platform for the metal additive manufacturing process and use a high-speed camera to shoot the additive manufacturing process video.

[0057] like Figure 2 As shown, the hardware platform for online visual monitoring of the metal additive manufacturing process includes a computer 1, a lighting source power supply 2, a high-speed camera 3, a macro lens 4, an 808nm lighting source 5, and a metal additive manufacturing device 6. The metal additive manufacturing device 6 used is a TS300A laser powder bed fusion forming device produced by Shanghai Tanzhen Laser Technology Co., Ltd. The powder material used in the printing process is GH4169, and the printing layer thickness is 40μm. The high-speed camera 3 uses a MEMRECAMACS-1 camera produced by Japan's NAC Image Technology Company. The shooting frame rate is 50,000 frames, and the image size of each frame is 1280×896 pixels. A macro lens 4 is installed in front to enlarge the shooting field of view. Set up the high-speed camera 3 and the 808nm lighting source 5, change the scanning speed and power of the processing laser to obtain different processing processes, and transmit the captured video data to the computer 1 for processing.

[0058] Step 2: Export the original video frame by frame to construct low-light and bright-light image datasets. The low-light and bright-light datasets of traditional low-light image enhancement algorithms are to first shoot bright-light images and then manually generate corresponding low-light images; while this embodiment directly obtains low-light videos, outputs them frame by frame as pictures, and then increases the brightness to generate corresponding bright-light images and form a training set.

[0059] Specifically, the low-light image is processed by gamma correction and image processing software to obtain a bright-light image with adjusted brightness; Gaussian noise and Poisson noise are added to simulate the noise in the image taken in a low-light environment to obtain an adjusted low-light image; the image area containing the molten pool and spatter is cropped to a size of 600×400 pixels. The above process is repeated to obtain a large number of paired low-light and bright-light images and construct a data set.

[0060] Step 3: Build a low-light image enhancement network model based on Retinex visual theory.

[0061] As a specific embodiment, Figure 3 As shown, the low-light image enhancement network model consists of a layer decomposition network, a reflection adjustment network, and an illumination adjustment network.

[0062] The layer decomposition network includes two branches: the reflectance map and the illumination map. The reflectance map branch is a simplified U-Net structure (5 layers), which is used to extract the reflection component from the input image and extract low-level features through the initial convolution layer, followed by downsampling through the maximum pooling layer to reduce the spatial dimension of the data. After extracting features through multiple layers of convolution, upsampling is performed using the transposed convolution layer, and then the output of the previous layer is connected back, combining low-level and high-level features in the subsequent convolution layer, and generating a reflectance map in the final convolution layer. The illumination map branch includes convolution layers, connection layers, ReLU layers, and Sigmoid layers. The illumination map branch obtains the feature map from the reflectance map branch and is subsequently processed to extract and enhance the illumination features of the image. By performing deep convolution operations and nonlinear activation on the feature map, the illumination changes are captured in detail, and the changes in illumination conditions and details are more accurately simulated in the final enhancement process.

[0063] Specifically, the reflection map branch in the layer decomposition network includes the first convolution layer + ReLU layer (Conv + ReLU32, Conv + ReLU 64, Conv + ReLU 128), the first deconvolution + ReLU layer, the first connection layer, the second convolution layer + ReLU layer, the second deconvolution + ReLU layer, the second connection layer, the third convolution layer + ReLU layer, and the convolution layer + Sigmoid function; the Conv + ReLU 32 in the first convolution layer + ReLU layer is jump connected to the second connection layer, and the Conv + ReLU 64 in the first convolution layer + ReLU layer is jump connected to the first connection layer.

[0064] Specifically, the illumination branch includes a convolution + ReLU layer, a connection layer, and a convolution layer + Sigmoid function. The illumination branch obtains the feature map from the reflectance map branch (Conv + ReLU 32 in the first convolution layer + ReLU layer) and is subsequently processed to extract and enhance the illumination features of the image. By performing deep convolution operations and nonlinear activation on the feature map, the illumination changes are captured and adjusted in detail to more accurately simulate the changes in illumination conditions and details in the final enhancement process.

[0065] Data flow in layer decomposition network:

[0066] 1. Reflection map branch

[0067] The training set is processed by the first convolution layer + ReLU layer (Conv + ReLU 32, Conv + ReLU 64, Conv + ReLU 128) for feature extraction, and then the extracted low-level features are merged with the output of the subsequent layer through the first connection layer, and then the features are upsampled and nonlinearly activated through the first deconvolution + ReLU layer. Then the second connection layer merges the extracted features again, and then the third convolution layer + ReLU layer continues to deepen the feature extraction. Finally, the convolution layer + Sigmoid function adjusts and outputs the final reflection map.

[0068] 2. Illumination Branch

[0069] After the training set passes through the first convolution layer + ReLU layer Conv+ReLU 32 in the reflectance map branch, the features are further extracted and processed through the convolution + ReLU layer, and then these features are integrated with the outputs of other branches through the connection layer, and finally adjusted and output as a light map through the convolution layer + Sigmoid function.

[0070] As a specific embodiment, the reflection adjustment network is composed of five convolutional layers + ReLU layers, which are used to further extract and refine reflection features from the input image. The reflection network outputs an adjusted and refined reflection map R'(x, y).

[0071] As a specific embodiment, the optical adjustment network is composed of a parallel global sub-network and a local sub-network.

[0072] The global sub-network includes an average pooling layer, three convolutional layers + ReLU layers, and convolutional layers + Sigmoid functions, which are used to capture and adjust the global illumination changes of the image. The global sub-network captures the overall brightness information of the image through the average pooling layer, and uses the convolution kernel for feature mapping and combination, and finally generates the global sub-network image.

[0073] The local sub-network consists of four convolutional layers + ReLU layers, convolutional layers + Sigmoid functions, which are used to refine and enhance local lighting details and map them to the output channels.

[0074] The global sub-network outputs the global illumination enhancement network output G(x,y), and the local sub-network outputs the local illumination enhancement network output C(x,y). After adjustment by weight α, the total output of the illumination adjustment network is I'(x,y).

[0075] R'(x, y) and I'(x, y) are multiplied element by element to obtain the final enhanced image.

[0076] Step 4: Decompose the input low-light image into a reflection map and an illumination map through a layer decomposition network, generate enhanced reflection map and illumination map through a reflection and illumination adjustment network and fuse them into an enhanced image, calculate the loss function value, and implement iterative training of the low-light image enhancement network model.

[0077] Specifically, each pixel in the image can be expressed as the product of the reflection value and the illumination value of the point, that is:

[0078] L(x,y)=R(x,y)·I(x,y)

[0079] Where L(x,y) represents the pixel value of the image at the coordinate (x,y), and R(x,y) and I(x,y) represent the reflection value and illumination value at that position.

[0080] The layer decomposition network decomposes the input low-light image into a reflection map and an illumination map; the reflection adjustment network enhances the reflection map output by the layer decomposition network; the illumination adjustment network inputs the illumination map output by the layer decomposition network into the global enhancement network and the local enhancement network, where the global enhancement network is used to improve the overall brightness of the image, and the local enhancement network is used to enhance the brightness in the high dynamic range area.

[0081] The output of the illumination adjustment network can be expressed as:

[0082] I′(x,y)=α·C(x,y)+(1-α)·G(x,y)

[0083] Where C(x,y) represents the local illumination enhancement network output, G(x,y) represents the global illumination enhancement network output, I'(x,y) represents the total output of the illumination adjustment network, and α represents the weight factor. Modifying the value of α can adjust the proportion of the two components, and its value range is 0 to 1. In this embodiment, the optimal value range of α is 0.75 to 0.85.

[0084] The purpose of low-light image enhancement is to improve the visual quality of the image to be processed, including noise suppression and illumination adjustment. The enhancement effect requires the use of objective, quantitative, and universal evaluation criteria. PSNR and SSIM have the advantages of simple and fast calculation, sensitivity to image distortion, and conformity to human vision, and are therefore used as quality evaluation indicators for low-light image enhancement in metal additive manufacturing of the present invention.

[0085] The PSNR indicator measures the peak signal-to-noise ratio of image quality. For a given original image I of size m×n and a noisy image K after adding noise to it, its MSE can be defined as:

[0086]

[0087] Among them, the size of the original image is m×n, I(i,j) represents the pixel value of the reference image I at the coordinate (i,j), and K(i,j) represents the pixel value of the image K to be evaluated at the coordinate (i,j);

[0088] Then PSNR can be defined as:

[0089]

[0090] Among them, MAX I represents the maximum value of all pixels of image I;

[0091] The SSIM metric measures the similarity between two images. The simplified form of SSIM can be defined as:

[0092]

[0093] Among them, μ x and μ y represents the average value of image pixels, σ x and σ y represents the variance, σ xy Represents covariance, the coordinates of a point in the image are (x, y), and C1 and C2 are constants that prevent the denominator from being zero.

[0094] The loss function takes into account the differences between pixels and the structural and contrast characteristics of the image. Compared with using MSE alone, the combination of PSNR and SSIM can more effectively identify the differences in human visual system perception. Its expression is:

[0095]

[0096] Among them, y true ,y pred Represent the original image and the predicted image respectively, PSNR(y true ,y pred) represents the peak signal-to-noise ratio of the original image and the predicted image, SSIM(y true ,y pred ) represents the structural similarity index between the original image and the predicted image.

[0097] Specifically, we use NVIDIA GeForce RTX 4060Laptop GPU for network model training, select Adam optimizer, and use a learning rate of 0.001. We use a learning rate decay strategy to reduce the learning rate to 95% of the original value after each training round, so that the later training can reach the optimal state.

[0098] Step 5: Apply the low-light image enhancement network model to the S1 online monitoring hardware platform to calculate the image quality evaluation indicators PSNR and SSIM.

[0099] Specifically, the software environment required to call the low-light image enhancement network model, including Python 3.10, Miniconda and TensorFlow 2.10, was installed on the computer of the online monitoring hardware platform built by S1, and the low-light image enhancement program was run.

[0100] like Figure 4 As shown in the figure, three original low-light images are randomly selected from the test set, and the trained model is compared with five existing low-light image enhancement methods, including RetinexNet, DLN, Zero-DCE, KinD, and EnlightenGAN. The input size of the model is set to 600×400 pixels. The PSNR and SSIM values ​​between the bright light images output by each low-light image enhancement method and the corresponding standard bright light image GT are calculated respectively. The results show that the quality evaluation indicators PSNR and SSIM values ​​of the output images of the proposed method are the highest.

[0101] Performance test of low-light image enhancement effect:

[0102] The fully convolutional neural network (FCN) is used to extract the molten pool and sputtering geometric features of the enhanced image, and the extracted results are analyzed.

[0103] Specifically, Figure 5 As shown in (a), five consecutive original low-light images are selected and the melt pool area is cropped out. The shooting interval of each image is 0.2ms. Figure 5 (b) is the bright light image sequence after the original low-light image is enhanced. Figure 5 (c) The outline of the molten pool is manually extracted using Labelme software and used as the true value of the molten pool outline. Figure 5 (d) is the result of directly extracting the original low-light image using the feature extraction algorithm FCN. Figure 5(e) shows the result of performing low-light image enhancement first and then feature extraction. The intersection over union (IoU) is used to measure the accuracy of melt pool extraction. When FCN is used directly, the average IoU is 0.85 and the standard deviation is 2.28×10 -2 ; The IoU of using low-light image enhancement first and then using FCN for feature extraction is significantly improved, with an average value of 0.94 and a standard deviation of 1.53×10 -2 Compared with directly extracting the melt pool from the original low-light image, the result of using the proposed low-light image enhancement algorithm and then extracting it is closer to manual extraction and has a higher accuracy.

[0104] like Figure 6 As shown in (a), five consecutive original low-light images are selected, and the shooting interval of each image is 0.2ms. Figure 6 (b) is the bright light image sequence after the original low-light image is enhanced. Figure 6 (c) The outlines of the molten pool and spatter are manually extracted using Labelme software and taken as the true value. Figure 6 (d) is the result of directly extracting the low-light image using the feature extraction algorithm FCN. Figure 6 (e) is the result of performing low-light image enhancement first and then feature extraction. The balance point (F1) between precision and recall is used to measure the accuracy of sputtering extraction. When FCN is used directly, F1 is 24.93; the F1 of using low-light image enhancement first and then using FCN for feature extraction is significantly improved, with an average value of 52.11. Compared with directly performing sputtering feature extraction on the original low-light image, the number of sputtering extractions after using the proposed low-light image enhancement algorithm is larger and closer to manual extraction.

[0105] Example 2

[0106] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for low-light image enhancement for online visual monitoring of metal additive manufacturing process as described in Example 1 is implemented.

[0107] Example 3

[0108] This embodiment provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for low-light image enhancement for online visual monitoring of a metal additive manufacturing process as described in Example 1 is implemented.

[0109] Example 4

[0110] This embodiment provides a computer program product, including a computer program, which, when executed by a processor, implements the low-light image enhancement method for online visual monitoring of a metal additive manufacturing process as described in Embodiment 1.

[0111] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A low-light image enhancement method for online visual monitoring of metal additive manufacturing process, characterized in that: The following steps are involved: Step 1: Build an online visual monitoring hardware platform for the metal additive manufacturing process and use a high-speed camera to shoot a video of the metal additive manufacturing process; Step 2, exporting the original captured video frame by frame to construct a dataset of low-light and bright-light images; the bright-light image is obtained by brightening the low-light image; Step 3, constructing a low-light image enhancement network model based on Retinex visual theory; the low-light image enhancement network model consists of a layer decomposition network, a reflection adjustment network and an illumination adjustment network; the layer decomposition network includes two branches: a reflection map and an illumination map; the reflection map branch is used to extract reflection components and features from the input image to generate a reflection map; The illumination map branch is used to obtain the feature map from the reflectance map branch to extract and enhance the illumination features of the image; The reflection adjustment network gradually extracts and abstracts image features through a series of convolutional layers, and outputs an adjusted reflection image; the illumination adjustment network is used to enhance local details and global illumination changes of the image to generate an illumination map to restore the image; The reflectance map branch in the layer decomposition network includes the first convolution layer + ReLU layer, the first deconvolution + ReLU layer, the first connection layer, the second convolution layer + ReLU layer, the second deconvolution + ReLU layer, the second connection layer, the third convolution layer + ReLU layer, and the convolution layer + Sigmoid function in sequence; the first convolution layer + ReLU layer is jump-connected to the first connection layer and the second connection layer respectively; the illumination map branch includes the convolution + ReLU layer, the connection layer, and the convolution layer + Sigmoid function; by performing deep convolution operations and nonlinear activation on the feature map, the illumination changes are captured and adjusted to simulate the changes in illumination conditions and details; The reflection adjustment network includes: an initial convolution layer for extracting preliminary features of an image; a second convolution layer for capturing a wider range of contextual information; a third convolution layer for enhancing the expressiveness of features; a fourth convolution layer for performing feature fusion while maintaining details; and a final convolution layer using a ReLU activation function to generate a final adjusted reflection image. The illumination adjustment network includes a local sub-network and a global sub-network; The global sub-network includes three layers, namely, the average pooling layer, the combination of convolution layer + ReLU layer, and convolution layer + Sigmoid function, which are used to capture and adjust the global illumination changes of the image. The global sub-network captures the overall brightness information of the image through the average pooling layer, and uses the convolution kernel for feature mapping and combination, and finally generates the global sub-network image. The local sub-network includes four layers of convolution layer + ReLU layer, convolution layer + Sigmoid function, which are used to refine and enhance local illumination details and map them to the output channel; The local and global features are combined through parameter weighting to balance the effects of detail enhancement and overall brightness adjustment, generating an illumination map to restore the image. Step 4: Train the low-light image enhancement network model and calculate the peak signal-to-noise ratio PSNR and structural similarity index SSIM , and then calculate the loss function to complete the iterative training of the low-light image enhancement network model; Step 5: Deploy the low-light image enhancement network model to the online monitoring hardware platform and calculate the image quality evaluation index PSNR and SSIM .

2. The method for low-light image enhancement for online visual monitoring of metal additive manufacturing process according to claim 1, characterized in that: In step 1, the monitoring hardware platform includes a computer, an illumination light source power supply, a high-speed camera, a macro lens, an illumination light source and a metal additive manufacturing device.

3. The low-light image enhancement method for online visual monitoring of metal additive manufacturing process according to claim 1 is characterized in that: In step 4, PSNR The calculation formula is as follows: in, Representing images I The maximum value of all pixels; The size of the original image is m × n， Represents the reference image I At coordinates ( i , j ), Represents the image to be evaluated K At coordinates ( i , j ) at the pixel value; SSIM The calculation formula is as follows: in, μ x and μ y represents the average value of image pixels, σ x and σ y represents the variance, σ xy represents the covariance, and the coordinates of a point in the image are ( x , y ), C 1 and C 2 is a constant to prevent the denominator from being zero.

4. The method for low-light image enhancement for online visual monitoring of metal additive manufacturing process according to claim 1, characterized in that: In step 4, the calculation formula of the loss function is as follows: in, represent the original image and the predicted image respectively, represents the peak signal-to-noise ratio of the original image and the predicted image, Represents the structural similarity index between the original image and the predicted image.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for low-light image enhancement for online visual monitoring of a metal additive manufacturing process as described in any one of claims 1 to 4 is implemented.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for low-light image enhancement for online visual monitoring of a metal additive manufacturing process as claimed in any one of claims 1 to 4 is implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for low-light image enhancement for online visual monitoring of a metal additive manufacturing process as claimed in any one of claims 1 to 4 is implemented.

Images