A high-fidelity real-time imaging method for laser additive manufacturing molten pool image

Abstract

The present application relates to a kind of laser additive manufacturing molten pool image high-fidelity real-time imaging method, belong to laser additive manufacturing in-situ monitoring and machine vision imaging technical field.The method includes: obtaining molten pool image sequence in laser additive manufacturing process;Determine short exposure acquisition parameters based on molten pool image sensor characteristics;Short exposure image pre-processing is obtained to the molten pool image to be enhanced;Reference enhanced image is generated using lightweight enhancement network;Based on reference enhanced image, perform deep guided adaptive Gamma search, obtain the optimal Gamma parameter corresponding to the current time window;The optimal Gamma parameter is applied to real-time molten pool image stream, and high-fidelity enhanced molten pool image is output.The present application can be based on ordinary commercial camera, suppress molten pool overexposure and restore molten pool boundary, solid-liquid interface and texture information under short exposure condition, while giving consideration to image fidelity and real-time performance, applicable to molten pool high-fidelity real-time imaging in laser additive manufacturing process.

Description

Technical Field

[0001] This invention relates to a high-fidelity real-time imaging method for molten pool images in laser additive manufacturing, and more particularly to a real-time imaging method for molten pool images based on short-exposure acquisition and depth-guided adaptive Gamma enhancement, used to acquire high-fidelity molten pool images in real time during laser additive manufacturing, belonging to the field of in-situ monitoring and machine vision imaging in laser additive manufacturing. Background Technology

[0002] Laser additive manufacturing (LAM) is an advanced manufacturing technology that uses a high-energy laser beam to melt metal powder or filament and deposit it layer by layer to form parts. It is widely used in aerospace, automotive, mold repair, and the manufacturing of complex metal components. In LAM, the molten pool is the core area where material melting, flow, solidification, and forming quality evolve. The geometry, boundary contours, solidification tail, solid-liquid interface, internal texture, and visual features such as spatter and metal vapor are closely related to quality problems such as incomplete fusion, over-melting, porosity, cracks, and forming instability. Therefore, real-time, high-fidelity in-situ visual imaging of the molten pool is a crucial foundation for process monitoring, quality assessment, and subsequent intelligent analysis.

[0003] Existing visual imaging of molten pools typically employs visible light cameras, infrared cameras, or multi-sensor systems to acquire images of the processing area. Among these, visible light visual imaging offers advantages such as high spatial resolution, relatively low equipment cost, and flexible deployment, making it suitable for online monitoring in industrial settings. However, the molten pool in laser additive manufacturing processes is characterized by strong thermal radiation, high dynamic range, and rapid changes. The core region of the molten pool, metal vapor, spatter particles, and substrate reflections can all generate extremely high brightness signals, causing oversaturation or pixel truncation of the image sensor. Once the sensor pixels reach saturation, the grayscale differences and temperature-related textures within the core region of the molten pool will be mapped to the same maximum grayscale value, resulting in the irreversible loss of true physical information.

[0004] To avoid overexposure, camera exposure time can be reduced, aperture decreased, or filters added. Short exposures can suppress sensor saturation to some extent, keeping the core area of ​​the molten pool within a linear dynamic range, thus preserving more realistic morphological information. However, excessively short exposures reduce overall image brightness, making it difficult to directly observe molten pool boundaries, solidification tails, solid-liquid interfaces, and low-brightness textures, resulting in decreased image contrast and discernibility. Therefore, molten pool visual imaging presents a contradiction: "long exposures lead to overexposure truncation, while short exposures result in insignificant details."

[0005] Existing hardware solutions typically improve molten pool image quality through methods such as neutral density filters, narrowband filters, multi-camera synchronization, multi-exposure high dynamic range imaging, or infrared / dual-wavelength thermal imaging. While these methods can compress the dynamic range or isolate specific wavelength signals under certain conditions, they often require complex optical paths, precise calibration, and synchronization control, resulting in high system costs, difficult integration, and strong dependence on materials, process parameters, and equipment structure, which is not conducive to low-cost, universal industrial deployment.

[0006] Existing algorithms typically employ methods such as fixed Gamma correction, histogram equalization, adaptive Gamma, and Retinex enhancement to improve image visual quality. While traditional algorithms are computationally fast, their parameters are usually fixed or dependent on local statistical features, making them ill-suited to rapidly changing molten pool brightness and morphology over time. Furthermore, they are prone to amplifying background noise, spatter, and metal vapor artifacts. Deep learning enhancement methods possess strong nonlinear mapping capabilities, enabling the generation of visually high-quality enhanced images. However, directly performing network inference on each frame of high-resolution molten pool image results in high computational latency, making it difficult to meet the demands of high-frame-rate real-time imaging. Moreover, for overexposed images that have already undergone pixel truncation, no post-processing algorithm can recover the lost original physical information.

[0007] Therefore, existing technologies still require a high-fidelity imaging method for molten pool images that can actively avoid overexposure of the molten pool at the acquisition end, recover key details in low-brightness images in short-exposure images, and meet the requirements for real-time output. Summary of the Invention

[0008] To address the problems of overexposure in molten pool images, insufficient detail in short-exposure images, and poor real-time performance of depth enhancement methods in existing technologies, this invention provides a high-fidelity real-time imaging method for molten pool images in laser additive manufacturing. This method suppresses sensor saturation at the source by acquiring images with short exposures, preserving the original physical information of the molten pool core, boundaries, and low-brightness textures. A lightweight enhancement network is then used to generate a reference enhanced image, and depth-guided adaptive Gamma search is used to obtain enhancement parameters suitable for the current time window. Finally, fast Gamma mapping is employed to enhance the real-time image stream, thus balancing image fidelity, detail visibility, and real-time performance.

[0009] To achieve the above objectives, this invention proposes a high-fidelity real-time imaging method for laser additive manufacturing molten pool images, comprising the following steps:

[0010] S11: Construct a molten pool visual in-situ imaging system to acquire molten pool image sequences during the laser additive manufacturing process; S12: Determine the short-exposure acquisition parameters based on the molten pool radiation intensity and the saturation characteristics of the image sensor, so that the core area of ​​the molten pool is within the linear dynamic range of the image sensor, and acquire a short-exposure molten pool image. S13: Preprocess the short-exposure molten pool image to obtain the molten pool image to be enhanced; S14: Generate a reference enhancement image from the molten pool image to be enhanced using a lightweight enhancement network; S15: Perform a depth-guided adaptive Gamma search based on the reference enhanced image to obtain the optimal Gamma parameter corresponding to the current time window; S16: Apply the optimal Gamma parameter to the real-time molten pool image stream to output a high-fidelity enhanced molten pool image.

[0011] Further, in step S11, the molten pool visual in-situ imaging system includes one or more of the following: a visible light camera, a lens, an aperture assembly, an image acquisition unit, and an installation assembly. The visible light camera may be a CMOS area array camera, and the acquired molten pool image may be an RGB image or a grayscale image.

[0012] Further, in step S12, the short-exposure acquisition parameters are determined based on at least one of the following: saturated pixel ratio, contrast-to-noise ratio, image entropy, boundary gradient magnitude, boundary gradient width, inter-frame segmentation cross-joint ratio, and molten pool area variation coefficient. The short-exposure acquisition parameters determined by the above indicators are used to suppress pixel saturation or pixel truncation caused by the molten pool core, metal vapor, spatter, and matrix reflection, ensuring that molten pool boundaries, solidification tails, solid-liquid interfaces, and low-brightness texture information are preserved in the short-exposure molten pool image.

[0013] Further, in step S13, the preprocessing includes at least one of the following: melt pool region extraction, image cropping, scale normalization, grayscale normalization, and color normalization. The preprocessed image or melt pool region, after being adjusted to a preset size, is then used as input to the lightweight augmentation network and the similarity evaluation process.

[0014] Further, in step S14, the lightweight enhancement network includes a depthwise separable convolution module, a skip connection module, a channel attention module, and a nonlinear activation function module, used to output a pixel-level enhancement parameter map corresponding to the input image. The lightweight enhancement network performs reference enhancement on the molten pool image to be enhanced based on a pixel-level illumination enhancement curve. This pixel-level illumination enhancement curve can enhance low-brightness areas and weaken the enhancement effect in high-brightness areas, thereby preventing oversaturation of the enhanced molten pool core region again.

[0015] Furthermore, the lightweight augmentation network is trained in a zero-reference manner, and the training loss function includes one or more of the following: spatial consistency loss, exposure control loss, color constancy loss, illumination smoothing loss, chromaticity preservation loss, and illumination consistency loss, in order to preserve the molten pool morphology, boundary continuity, and thermal color characteristics.

[0016] Further, in step S15, the real-time melt pool image stream is divided into continuous time windows. Several representative frames are extracted within each time window. A lightweight enhancement network generates a reference enhanced image from these representative frames, and the optimal Gamma parameter for that time window is searched based on the representative frames and the reference enhanced image. A global Gamma search is performed within the first time window, and a local Gamma search is performed within subsequent time windows, centered on the optimal Gamma parameter of the previous time window. The Gamma search is evaluated based on the similarity between the Gamma-mapped image and the reference enhanced image. The similarity evaluation metrics include at least one of mean squared error, peak signal-to-noise ratio, and structural similarity.

[0017] Furthermore, in step S16, an asynchronous processing architecture is used to output the high-fidelity enhanced molten pool image. The main thread performs Gamma mapping on the real-time molten pool image stream and outputs the enhanced image, while the background thread performs lightweight augmented network inference and Gamma parameter search. When the background thread has not completed the Gamma parameter search for the current time window, the main thread uses the most recently effective optimal Gamma parameter to maintain the continuous real-time output of the molten pool image stream.

[0018] The advantages of this invention compared to the prior art are: (1) The present invention actively suppresses sensor saturation during the image formation stage by short exposure acquisition, so that the core area of ​​the molten pool is kept within the linear dynamic range of the image sensor, and enhances the recovery of molten pool boundary, solidification tail, solid-liquid interface and low brightness texture details by depth guidance, thereby improving the observability and physical fidelity of molten pool image.

[0019] (2) The present invention uses a lightweight enhancement network to generate reference enhancement images and performs real-time enhancement through Gamma mapping, which combines the high-quality guidance capability of deep networks with the low-latency advantage of traditional image mapping, avoiding the lack of real-time performance caused by frame-by-frame deep network inference.

[0020] (3) The present invention enables the enhancement parameters to be adaptively updated as the state of the molten pool changes by using a global search in the first window and a local search in the subsequent time window, thereby improving the adaptability to dynamic processing.

[0021] (4) The present invention adopts an asynchronous processing architecture, which enables real-time image acquisition and enhanced output to be unblocked by the background parameter search process, and can maintain a continuous and stable output of molten pool image stream, which is suitable for high frame rate laser additive manufacturing in-situ imaging scenarios.

[0022] (5) This invention does not rely on complex multi-camera synchronization or dedicated high-cost optical hardware. It can achieve high-fidelity real-time imaging of molten pool images based on commercial vision cameras and has good adaptability to different materials, different process parameters and different optical configurations. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the high-fidelity real-time imaging method for molten pool images proposed in this invention.

[0024] Figure 2 A diagram showing the in-situ visual imaging system for the molten pool constructed for this invention.

[0025] Figure 3 This is a graph showing the image quality index calculated by this invention.

[0026] Figure 4 These are comparison images of automatic exposure, short exposure, and images enhanced by the method of this invention.

[0027] Figure 5 This is a structural diagram of the enhanced model proposed in this invention.

[0028] Figure 6 This diagram compares the enhancement method proposed in this invention with other enhancement methods.

[0029] Figure 7 The imaging method proposed in this invention acquires images in materials such as high-temperature alloys, aluminum alloys, and stainless steel.

[0030] Figure 8 The confusion matrix is ​​obtained by classifying the molten pool image acquired in automatic exposure mode.

[0031] Figure 9 This is the confusion matrix of the molten pool image acquired under the short exposure mode determined in S12 of the present invention and identified by the classification model.

[0032] Figure 10 The confusion matrix is ​​obtained by classifying the melt pool image after it has been acquired in short exposure mode and enhanced based on the enhancement model proposed in this invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the following embodiments are only used to illustrate the technical solutions of this invention and are not intended to limit the scope of protection of this invention. Equivalent substitutions made by those skilled in the art regarding imaging device models, exposure parameter ranges, time window lengths, image sizes, or similarity evaluation indicators without departing from the concept of this invention should all fall within the scope of protection of this invention.

[0034] To address the problems existing in current additive manufacturing molten pool imaging technology, this embodiment provides a high-fidelity real-time imaging method for laser additive manufacturing molten pool images. Figure 1This is a schematic flowchart illustrating the process of acquiring molten pool images based on the method provided in this embodiment of the invention. The method first uses short-exposure acquisition to suppress image sensor oversaturation caused by strong radiation from the molten pool, ensuring that the core region of the molten pool and its boundary texture are preserved as much as possible in the original image. Then, a lightweight enhancement network is used to generate a reference enhanced image of the current molten pool state. Next, a depth-guided adaptive Gamma search is used to obtain the optimal Gamma parameters suitable for the current time window. Finally, an asynchronous processing architecture is employed to perform Gamma mapping on the real-time molten pool image stream, outputting a high-fidelity enhanced molten pool image.

[0035] The method in this embodiment includes the following steps.

[0036] S11: Construct a molten pool visual in-situ imaging system to obtain molten pool image sequences during the laser additive manufacturing process.

[0037] In a preferred embodiment, the laser additive manufacturing process is a laser-directed energy deposition process. Figure 2 The molten pool visual in-situ imaging system shown includes laser additive manufacturing equipment, a powder feeding system, a visible light camera, a lens, an image acquisition unit, and a camera mounting assembly. The visible light camera is mounted near the processing head, with its optical axis aligned with the laser-affected area, to continuously acquire images of the molten pool and its surrounding area.

[0038] Preferably, the laser is a continuous fiber laser with a wavelength of 1070 nm, a maximum power of 1 kW, and a focused spot diameter of 0.6 mm. The protective gas is argon with a flow rate of 13.6 L / min and a powder feed rate of 3.2 g / min. The visible light camera is a CMOS area array camera with a sampling frequency of 90 frames / second, a lens aperture of F16, and an angle of 45 degrees between the camera and the substrate. The above equipment parameters are only one preferred embodiment, and the present invention is not limited to this equipment model and parameter combination.

[0039] S12: Determine the short-exposure acquisition parameters based on the molten pool radiation intensity and the saturation characteristics of the image sensor, so that the core area of ​​the molten pool is within the linear dynamic range of the image sensor, and acquire a short-exposure molten pool image.

[0040] In this embodiment, several candidate exposure times are set, and molten pool images are acquired at each candidate exposure time. The candidate exposure times can be set between the minimum exposure time allowed by the camera and the exposure time that would result in significant overexposure. For example, in a preferred embodiment, the candidate exposure time ranges from 65 microseconds to 5000 microseconds.

[0041] To determine suitable short-exposure acquisition parameters for high-fidelity imaging of the molten pool, image quality metrics and physical fidelity metrics were calculated for molten pool images at different exposure times. The image quality metrics include at least one of saturated pixel ratio, contrast-to-noise ratio, and image entropy; the physical fidelity metrics include at least one of boundary gradient magnitude, boundary gradient width, inter-frame segmentation cross-union ratio, and molten pool region area variation coefficient.

[0042] The saturated pixel ratio is used to characterize the proportion of overexposed pixels in the molten pool image. Let the exposure time be... The image of the molten pool acquired below is Convert RGB images to luminance maps ,right A binary mask is obtained by using Otsu thresholding or adaptive thresholding. :

[0043] Where T represents the candidate exposure time. For pixel coordinates, Exposure time The brightness map is in pixels grayscale value at that location This is the segmentation threshold corresponding to the exposure time. The value is set to the binary mask. A value of 1 indicates that the pixel belongs to the candidate region of the melt pool, and a value of 0 indicates that the pixel belongs to the background region.

[0044] Opening, closing, and small connected component removal operations are performed on the binary mask. The largest connected region or the connected region closest to the laser's irradiation center is selected as the molten pool region. The evaluation region of the molten pool neighborhood is obtained by expanding it outward by a preset pixel range. The evaluation region for the molten pool and its neighborhood is... The total number of pixels in the area is saturation threshold is For 8-bit images, The preferred saturation pixel ratio is 240. Represented as:

[0045] in, Exposure time The evaluation area of ​​the lower molten pool and its neighborhood. This represents the total number of pixels in the region. For pixels grayscale or brightness value, The saturation threshold This is an indicator function. The larger the value, the more severe the pixel truncation at that exposure time.

[0046] The contrast-to-noise ratio is used to characterize the distinguishability between the molten pool region and the background region. The Otsu thresholding method is used to divide the image into molten pool regions. and background area Let the average gray levels of the two be respectively , The standard deviations are respectively , Then the contrast noise ratio Represented as:

[0047] Image entropy The information richness used to characterize the gray-level distribution of an image is represented as:

[0048] in, For gray levels, For grayscale indexing, this applies to 8-bit images. , The grayscale value is The probability of a pixel appearing.

[0049] Boundary gradient width and magnitude are used to characterize the clarity of the solid-liquid interface and boundary profile of the molten pool. The boundary gradient width can be expressed as:

[0050] in, Exposure time The boundary gradient width below, For boundary pixels The width of the local grayscale transition along the normal direction. The set of pixels at the boundary of the molten pool. Set of pixels at the boundary of the molten pool The number of pixels in the image. The smaller the value, the narrower the grayscale transition at the boundary.

[0051] To evaluate the sharpness of the melt pool boundary, the Sobel operator is used to calculate the gradient magnitude:

[0052] in, and These represent the gradients of the image in the horizontal and vertical directions, respectively. The average gradient magnitude at the boundary can be expressed as:

[0053] in, Exposure time The set of pixels at the bottom edge of the melt pool. For the number of boundary pixels, For pixels gradient magnitude at that point The larger the value, the clearer the solid-liquid interface and boundary contour of the molten pool.

[0054] Inter-frame segmentation cross-over ratio (COP) and molten pool area variation coefficient are used to evaluate the temporal stability of the molten pool image across consecutive frames. For the molten pool segmentation region in consecutive frames... and The inter-frame segmentation crossover ratio is expressed as:

[0055] Coefficient of variation of molten pool area Represented as:

[0056] in, and These are the mean and standard deviation of the area of ​​the molten pool region in consecutive frames, respectively. The larger and The smaller the value, the better the temporal stability of the melt pool segmentation result at that exposure time.

[0057] Preferably, select to make Lower , IE(T) and Higher, and make higher A lower exposure time is used as a short exposure acquisition parameter. In a preferred embodiment, the above indicators are calculated comprehensively, and the result is as follows: Figure 3 As shown, 65 microseconds was determined as the short exposure acquisition parameter. Figure 4 As shown, this exposure time can significantly reduce pixel saturation caused by the molten pool core, metal vapor, spatter, and substrate reflection, while preserving molten pool boundaries, solidification tails, solid-liquid interfaces, and low-brightness texture information. It should be noted that 65 microseconds is only a preferred value for this embodiment under specific camera and process conditions. Under other camera, material, or process parameters, the short exposure acquisition parameters can be re-determined using the same evaluation method.

[0058] S13: Preprocess the short-exposure molten pool image to obtain the molten pool image to be enhanced.

[0059] Since the original image may contain a large area of ​​matrix, powder, nozzle, or background in addition to the molten pool, this embodiment preprocesses the short-exposure molten pool image to reduce computational load and improve the efficiency of enhancement parameter search. The preprocessing includes at least one of the following: molten pool region extraction, image cropping, scale normalization, grayscale normalization, and color normalization.

[0060] Preferably, the short-exposure melt pool image is first thresholded to determine the region of interest (ROI) containing the melt pool core, melt pool boundary, and surrounding low-brightness texture; then, the ROI is cropped and adjusted to a preset size. The preset size is preferably 224×224 pixels to accommodate the input of the lightweight augmentation network and the subsequent similarity evaluation process. For RGB images, the pixel values ​​of each channel can be normalized to the range [0,1]; for grayscale images, the grayscale values ​​of a single channel can be normalized to the range [0,1].

[0061] S14: Generate a reference enhancement image from the molten pool image to be enhanced using a lightweight enhancement network.

[0062] In this embodiment, the lightweight enhancement network is not directly used as the real-time output path for each frame of high-resolution image. Instead, it serves as a reference image generator to generate a high-quality reference enhancement image for the current molten pool state. This reference enhancement image is used to guide subsequent Gamma parameter searches.

[0063] Preferably, the lightweight enhancement network includes an input convolutional layer, multiple depthwise separable convolutional modules, a skip connection module, a channel attention module, and an output convolutional layer. The input image is first expanded in channel dimension by the convolutional layer, and then sequentially passed through the depthwise separable convolutional modules to extract melt pool brightness, color, boundary, and texture features. Skip connections in the network are used to preserve shallow spatial details, and the channel attention module is used to adaptively weight channels related to melt pool thermal color features. The output convolutional layer outputs a pixel-level enhancement parameter map corresponding to the input image.

[0064] In a preferred embodiment, the lightweight augmentation network includes 7 main convolutional modules, 22 convolutional layers, and 3 SE channel attention modules. The basic channel width is 16, the number of training parameters is approximately 5215, the computational complexity is approximately 0.201 GFLOPs, and the model weight size is approximately 34.8KB. The network structure is shown in Table 1. Figure 5 As shown.

[0065] Table 1 Lightweight Augmentation Network Structure

[0066] The lightweight enhancement network uses a pixel-level illumination enhancement curve to perform reference enhancement on the melt pool image to be enhanced. Let the input pixel value be I, the enhanced pixel value be E, and the pixel-level enhancement parameter be A. Then the illumination enhancement curve can be expressed as:

[0067] Where I and E are normalized pixel values, and A is the pixel-level enhancement parameter output by the lightweight enhancement network. This enhancement curve can be iterated multiple times to improve the enhancement capability for low-brightness regions. In a preferred embodiment, the number of iterations is set to 8. Due to the... When the pixel value is close to 1, it approaches 0. Therefore, this curve can automatically reduce the enhancement effect on the core area of ​​the high-brightness melt pool while enhancing the low-brightness boundary and texture area, thus avoiding oversaturation again after enhancement.

[0068] The lightweight augmentation network employs zero-reference training, eliminating the need for paired short-exposure melt pool images and manually labeled high-quality target images. The training loss function includes one or more of the following: spatial consistency loss, exposure control loss, color constancy loss, illumination smoothing loss, chromaticity preservation loss, and illumination consistency loss. Let the short-exposure input image be... The enhanced reference image is The pixel-level enhancement parameters output by the lightweight enhancement network are shown in the figure. The total loss function is:

[0069] in, , , , , , These are the weights of each loss term.

[0070] Among them, spatial consistency loss is used to constrain the local gradient relationship before and after enhancement, and to maintain the continuity of the melt pool morphology and boundary;

[0071] in, The number of pixels involved in the calculation. For pixel index, For the set of neighborhood directions, The neighborhood offset direction, and The images before and after enhancement are respectively at the 1st... Pixel value at 1 pixel and These are the pixel values ​​at adjacent pixels.

[0072] Exposure control loss is used to prevent the enhanced image from being too dark or too bright, so that the local brightness is close to the preset reasonable exposure level:

[0073] in, This represents the number of local image patches. Indicates the first in the enhanced image A local image patch, The average brightness of this local image patch. The preset target brightness is preferably 0.6.

[0074] Color constant loss is used to maintain color balance between RGB channels:

[0075] in, , , These represent the average intensity of the enhanced image in the R, G, and B channels, respectively.

[0076] Illumination smoothing loss is used to constrain the spatial continuity of the enhanced parametric map and avoid abrupt artifacts.

[0077] in, For the number of pixels, For pixel index, For pixel-level enhancement parameter maps in the first The value at each pixel and These represent the gradients of the enhancement parameter map in the horizontal and vertical directions, respectively.

[0078] Chromaticity preservation loss is used to constrain the differences in hue and saturation between the images before and after enhancement, so that the thermal color features of the molten pool are not significantly distorted.

[0079] in, To enhance the previous image, To enhance the image, and These represent the hue and saturation components of the image after conversion to the HSV color space. Describes the norm 1. and The weighting coefficient is 0.5, and it is preferred that all of them be 0.5.

[0080] The illumination consistency loss is used to constrain the illumination gradient distribution of the enhanced image, avoiding the generation of artificial brightness variations that do not conform to the original physical scene.

[0081] in, Represents the grayscale image of the enhanced image. This represents the gradient of a grayscale image in multiple directions. This indicates variance calculation.

[0082] S15: Perform a depth-guided adaptive Gamma search based on the reference enhanced image to obtain the optimal Gamma parameter corresponding to the current time window.

[0083] In this embodiment, the real-time melt pool image stream is divided into continuous time windows, and the Gamma parameter is updated in units of time windows.

[0084] Let the camera sampling frequency be... The time window length is Then the first The number of frames within each time window is:

[0085] Let the number of representative frames extracted in each time window be . Then the uniform sampling interval is:

[0086] From this time window, by interval Extract representative frames to form a representative frame set:

[0087] in, The number of frames within a single time window. For camera sampling frequency, The length of the time window. The number of representative frames extracted for each time window. To ensure uniform sampling intervals, Indicates rounding down. For the first A set of representative frames for each time window Indicates the first The first time window Frame image.

[0088] Time window length and represent the number of frames The Gamma parameter search time within a single time window is determined based on the camera frame rate, background optimization time, and the rate of change of the molten pool state. This ensures that the search time is less than the time window length and that the representative frame can cover the changes in molten pool brightness and morphology within that time window.

[0089] In a preferred embodiment fps, ,therefore ;set up Then the sampling interval Experiments have shown that this setting can ensure that the representative frame covers the changes in the melt pool state while making the background parameter search time less than the time window length, thus meeting the requirements of real-time imaging.

[0090] For each representative frame, a corresponding reference augmented image is first generated using a lightweight augmentation network; then, Gamma mapping with different Gamma parameters is performed on the original short-exposure image; next, the similarity between the Gamma-mapped image and the reference augmented image is calculated; finally, the Gamma parameter that yields the optimal similarity is selected as the optimal Gamma parameter for the current time window. The Gamma mapping can be represented as:

[0091] Where I is the normalized pixel value of the short exposure image, γ is the Gamma parameter, and G is the pixel value of the image after Gamma mapping.

[0092] Preferably, the similarity evaluation metric is the mean squared error (MSE). Let the reference enhanced image be R, the Gamma-mapped image be G, and the total number of pixels in the image be N, then the mean squared error can be expressed as:

[0093] in, Represents the first Gamma-mapped image The pixel value of each pixel. Indicates the first reference enhancement image The MSE is the pixel value of each pixel. The smaller the MSE, the closer the Gamma-mapped image is to the reference enhanced image. In addition to MSE, Peak Signal-to-Noise Ratio or Structural Similarity can also be used as similarity evaluation metrics.

[0094] In the first time window, since there is no optimal Gamma parameter from the previous time window, a global Gamma search is performed. For example, within a preset range of Gamma parameters, candidate Gamma parameters are traversed with a fixed step size, and the Gamma parameter with the smallest MSE is selected as the optimal Gamma parameter for the first time window. The objective function for the optimal Gamma parameter can be expressed as:

[0095] in, Let Gamma be the optimal parameter for the k-th time window. This is the set of candidate Gamma parameters for this time window. For the first The first time window Each representative frame For the corresponding reference enhanced image, To apply parameters to the representative frame The resulting Gamma mapping image, This represents the number of frames.

[0096] In subsequent time windows, a local Gamma search is performed within a preset local search radius, centered on the optimal Gamma parameter of the previous time window. Preferably, the local search radius is set to 0.5. This local search method utilizes the characteristic that the thermal state of the molten pool changes continuously within adjacent time windows, which can significantly reduce the computational load while ensuring search accuracy.

[0097] S16: Apply the optimal Gamma parameter to the real-time molten pool image stream to output a high-fidelity enhanced molten pool image.

[0098] To meet the requirements of real-time imaging, this embodiment adopts an asynchronous processing architecture. The asynchronous processing architecture includes a main thread and a background thread. The main thread is responsible for real-time image acquisition, reading the most recently valid optimal Gamma parameter, performing Gamma mapping, and outputting the enhanced image; the background thread is responsible for time window frame extraction, lightweight augmented network inference, Gamma parameter search, and optimal Gamma parameter update.

[0099] While the background thread has not yet completed its Gamma parameter search for the current time window, the main thread does not wait for the background thread. Instead, it uses the most recently valid optimal Gamma parameter to perform Gamma mapping on the current frame. Therefore, the real-time image acquisition and enhancement output process is not blocked by the background inference and search processes. Once the background thread completes its optimal Gamma parameter search for the current time window, it updates the parameter to the shared parameter area, and the main thread reads and uses the new optimal Gamma parameter in subsequent frames.

[0100] In one optional embodiment, when the background task queue is full or the background search time exceeds a preset threshold, unprocessed old optimization tasks are discarded, and the latest time window tasks are retained, thereby avoiding task backlog and latency accumulation. Even on low-computing-power devices, the main thread can still use the most recent valid Gamma parameter to maintain continuous enhanced output.

[0101] In a preferred embodiment, gamma mapping is implemented using a lookup table. Since gamma mapping is a pixel-level monotonic transformation, a lookup table corresponding to grayscale values ​​from 0 to 255 can be pre-established based on the current gamma parameter, and then the lookup table mapping is performed on each frame of the image, thereby reducing the enhancement time per frame. In experiments, the gamma application time per frame is approximately 0.113 ms, which meets the requirements of high frame rate fused pool imaging.

[0102] To verify the imaging effect of the method of the present invention, it was compared with methods such as fixed Gamma correction, histogram equalization, adaptive Gamma, and Retinex enhancement. The comparison results are as follows: Figure 6As shown, traditional methods are prone to over-enhancement, background noise amplification, or thermal color distortion; while direct deep network enhancement can improve brightness, it suffers from high frame-by-frame inference latency. This invention generates a reference image through a lightweight enhancement network and applies the enhancement results in real-time via Gamma mapping, balancing enhancement quality and real-time performance.

[0103] Furthermore, the method of this invention can also be implemented with different materials and different laser additive manufacturing processes. For example... Figure 7 As shown, for materials such as high-temperature alloys, aluminum alloys, and stainless steel, the initial brightness of short-exposure images may differ due to variations in material emissivity and molten pool brightness distribution. However, as long as short-exposure acquisition prevents pixel truncation of the molten pool core, depth-guided adaptive Gamma search can adaptively obtain appropriate enhancement parameters based on the current time window image. For high-speed, small-scale molten pool scenarios such as laser powder bed melting, this invention can also freeze molten pool movement through short exposure and recover the boundaries and textures of minute molten pools through adaptive Gamma enhancement.

[0104] The enhanced high-fidelity molten pool image can be used for manual observation, image feature extraction, and subsequent quality analysis. To further verify whether the high-fidelity enhanced molten pool image output by this invention is more conducive to subsequent quality analysis, in one embodiment, a molten pool state classification task is used as an indirect means of verifying imaging quality. It should be noted that the classification task is only used to evaluate the visibility and distinguishability of the physical features of the molten pool in the enhanced image, and does not participate in the short exposure parameter determination, reference image generation, Gamma parameter search, or real-time enhancement output process of this invention, nor does it constitute a necessary step in the imaging method of this invention.

[0105] In this verification embodiment, comparative data were constructed using three methods: automatic exposure acquisition, short exposure parameter acquisition obtained through the exposure optimization parameter method proposed in this invention, and post-exposure enhancement. Figure 4 As shown, in automatically exposed images, due to severe overexposure in the core region of the molten pool, the internal texture, solid-liquid interface, and boundary brightness gradient of the molten pool are truncated, easily resulting in large areas of saturated white spots in the image. In this situation, the classification model struggles to extract stable features from the true morphology of the molten pool and is easily affected by false features such as metal vapor, spatter, and background reflections.

[0106] Using short-exposure acquisition significantly suppresses pixel saturation in the core region of the molten pool, preserving the overall morphology, head, middle, tail, and boundary contours. This allows the classification model to rely more on the true geometry of the molten pool for judgment. However, the overall brightness of the short-exposure image is low, and some low-brightness boundaries, solidification tails, and solid-liquid interface textures are still not clearly defined, leading to some confusion between the normal molten state and slightly unfused or over-melted states.

[0107] After enhancement using the method of this invention, low-brightness details that were originally preserved but difficult to observe in the short-exposure image are further revealed. The brightness distribution in the center of the molten pool, boundary textures, solidification tails, and solid-liquid interface areas are clearer, while the core region of the molten pool does not experience significant oversaturation again. Thus, the enhanced image retains the physical authenticity obtained from the short-exposure acquisition and improves the visibility of key areas of the molten pool, enabling subsequent visual analysis models to more stably focus on the morphology of the molten pool itself, rather than spatter, plumes, or overexposure artifacts.

[0108] In a preferred embodiment, the images before and after enhancement are respectively input into the same type of melt pool state classification model, namely the ResNet50 classification model, for verification. Figure 8 The confusion matrix is ​​obtained by classifying the molten pool image acquired in automatic exposure mode. Figure 9 This is the confusion matrix of images acquired in short-exposure mode after being identified by a classification model. Figure 10 The image shows a confusion matrix of a molten pool image acquired in short-exposure mode and enhanced by the method of this invention, after being identified by a classification model. The confusion matrix reveals that the automatic exposure image suffers from severe overexposure of the molten pool core, resulting in low accuracy in identifying the normal molten state; the model easily confuses it with unfused or over-molten states. After short-exposure acquisition, the saturated pixels in the molten pool core region are significantly reduced, while the overall outline and geometry of the molten pool are preserved, significantly improving the accuracy in identifying various molten pool states. Further enhancement using the method of this invention increases the accuracy in identifying the normal molten state from approximately 83% in the short-exposure image to approximately 88%, indicating that the enhancement process effectively reveals the previously weaker central brightness distribution, boundary texture, and solid-liquid interface features in the short-exposure image, thereby reducing misjudgments between the normal molten state and slightly abnormal states. Therefore, the method of this invention can improve the visibility of the true physical features of the molten pool and the distinguishability of different molten pool states.

[0109] Therefore, the downstream classification experiments demonstrate from an application perspective that the imaging method of this invention can improve the information quality of molten pool images, making the enhanced images more suitable for manual observation, image feature extraction, and subsequent quality analysis. However, the aforementioned classification model, classification categories, and classification results are only used as verification methods for improving imaging quality and do not limit the scope of protection of this invention.

[0110] In summary, this invention achieves high-fidelity, low-latency, and continuous real-time imaging of molten pool images in laser additive manufacturing through short-exposure acquisition, lightweight enhancement network reference generation, depth-guided adaptive Gamma search, and asynchronous real-time enhancement output.

[0111] While the specific implementation method of the present invention has been described above, those skilled in the art, under the guidance of the present invention, can make various similar representations without departing from the spirit and claims of the present invention, and such modifications all fall within the protection scope of the present invention.

Claims

1. A high-fidelity real-time imaging method for laser additive manufacturing molten pool images, characterized in that, Includes the following steps: S11: Construct a molten pool visual in-situ imaging system to acquire molten pool image sequences during the laser additive manufacturing process; S12: Determine the short-exposure acquisition parameters based on the molten pool radiation intensity and the saturation characteristics of the image sensor, so that the core area of ​​the molten pool is within the linear dynamic range of the image sensor, and acquire a short-exposure molten pool image. S13: Preprocess the short-exposure molten pool image to obtain the molten pool image to be enhanced; S14: Generate a reference enhancement image from the molten pool image to be enhanced using a lightweight enhancement network; S15: Perform a depth-guided adaptive Gamma search based on the reference enhanced image to obtain the optimal Gamma parameter corresponding to the current time window; S16: Apply the optimal Gamma parameter to the real-time molten pool image stream to output a high-fidelity enhanced molten pool image.

2. The high-fidelity real-time imaging method for laser additive manufacturing molten pool images according to claim 1, characterized in that, The in-situ visual imaging system for the molten pool described in step S11 includes one or more of the following: a visible light camera, a lens, an aperture assembly, an image acquisition unit, and an installation assembly; the visible light camera is a CMOS area array camera, and the acquired molten pool image is an RGB image or a grayscale image.

3. The high-fidelity real-time imaging method for laser additive manufacturing molten pool images according to claim 1 or 2, characterized in that, The short exposure acquisition parameters in step S12 are determined based on at least one of the following: saturated pixel ratio, contrast-to-noise ratio, image entropy, boundary gradient magnitude, boundary gradient width, inter-frame segmentation crossover ratio, and melt pool area variation coefficient.

4. The high-fidelity real-time imaging method for laser additive manufacturing molten pool images according to claim 1, characterized in that, The short-exposure acquisition parameters described in step S12 are used to suppress pixel saturation or pixel truncation caused by the molten pool core, metal vapor, splash, and matrix reflection, so that the molten pool boundary, solidification tail, solid-liquid interface, and low-brightness texture information are preserved in the short-exposure molten pool image.

5. The high-fidelity real-time imaging method for laser additive manufacturing molten pool images according to claim 1 or 2, characterized in that, The preprocessing in step S13 includes at least one of the following: melt pool region extraction, image cropping, scale normalization, grayscale normalization, and color normalization; and the preprocessed image or melt pool region is adjusted to a preset size and then input into the lightweight enhancement network and similarity evaluation process.

6. The high-fidelity real-time imaging method for laser additive manufacturing molten pool images according to claim 5, characterized in that, The lightweight enhancement network described in step S14 includes a depthwise separable convolutional module, a skip connection module, a channel attention module, and a nonlinear activation function module, which are used to output a pixel-level enhancement parameter map corresponding to the input image.

7. The high-fidelity real-time imaging method for laser additive manufacturing molten pool images according to claim 6, characterized in that, The lightweight enhancement network performs reference enhancement on the image of the molten pool to be enhanced based on a pixel-level illumination enhancement curve. The pixel-level illumination enhancement curve enhances low-brightness areas through multiple iterations and weakens the enhancement effect in high-brightness areas to avoid oversaturation in the enhanced core area of ​​the molten pool.

8. The high-fidelity real-time imaging method for laser additive manufacturing molten pool images according to claim 6, characterized in that, The lightweight augmentation network is trained in a zero-reference manner, and the training loss function includes one or more of the following: spatial consistency loss, exposure control loss, color constancy loss, illumination smoothing loss, chromaticity preservation loss, and illumination consistency loss, in order to preserve the molten pool morphology, boundary continuity, and thermal color characteristics.

9. The method for high-fidelity real-time imaging of molten pool images in laser additive manufacturing according to claim 1 or 2, characterized in that, In step S15, the real-time melt pool image stream is divided into continuous time windows, and several representative frames are extracted in each time window. The lightweight enhancement network generates a reference enhancement image for the representative frames, and searches for the optimal Gamma parameter for the time window based on the representative frames and the reference enhancement image.

10. The method for high-fidelity real-time imaging of molten pool images in laser additive manufacturing according to claim 9, characterized in that, The depth-guided adaptive Gamma search in step S15 includes: performing a global Gamma search within the first time window, and performing a local Gamma search centered on the optimal Gamma parameter of the previous time window in subsequent time windows; the Gamma search is evaluated based on the similarity between the Gamma-mapped image and the reference enhanced image, and the similarity evaluation index includes at least one of mean squared error, peak signal-to-noise ratio and structural similarity.

11. The method for high-fidelity real-time imaging of laser additive manufacturing molten pool images according to claim 1 or 2, characterized in that, In step S16, an asynchronous processing architecture is used to output the high-fidelity enhanced molten pool image. The main thread performs Gamma mapping on the real-time molten pool image stream and outputs the enhanced image, while the background thread performs lightweight augmented network inference and Gamma parameter search. When the background thread has not completed the Gamma parameter search for the current time window, the main thread uses the most recently effective optimal Gamma parameter to maintain the continuous real-time output of the molten pool image stream.

Images