A machine vision-based laser cladding online quality detection system

Abstract

The present application relates to the technical fields of laser cladding detection, in particular to a kind of laser cladding online quality detection system based on machine vision, comprising: acquisition laser cladding processing process molten pool area multi-band synchronous visual image sequence, through space-time alignment and feature layer fusion processing constructs molten pool composite visual feature atlas.Introduce improved region growing algorithm, combined with molten pool thermal gradient distribution and spectral feature similarity dynamic adjustment growth criterion, complete potential defect area segmentation, with multi-feature consistency verification and contour finishing, obtain defect segmentation mask and contour geometric information.According to the segmentation result, extract morphological descriptor, thermal history curve and spectral intensity distribution related multi-dimensional quantitative feature vector, import the pre-trained multi-classification defect recognition model into the feature vector, complete defect type determination and confidence output.Widen molten pool visual information acquisition dimension, optimize defect area segmentation precision, enrich defect feature representation form.

Description

Technical Field

[0001] This invention relates to the field of laser cladding inspection technology, and in particular to an online quality inspection system for laser cladding based on machine vision. Background Technology

[0002] Laser cladding is a dynamic additive manufacturing process where parameters in the molten pool continuously fluctuate throughout the process. Currently, laser cladding quality inspection largely relies on conventional machine vision architectures. Traditional inspection methods use only single-band equipment for image acquisition, resulting in limited information coverage and insufficient ability to simultaneously acquire and integrate multi-dimensional visual data. The image data processing flow lacks a unified spatiotemporal calibration step, leading to poor temporal correspondence of image data from different acquisition nodes in similar scenarios. Deep fusion processing of various visual features is impossible, and multiple related physical parameters in the molten pool area are difficult to present synchronously, making it unsuitable for the refined inspection requirements of complex processing conditions.

[0003] Conventional image segmentation algorithms employ fixed operational logic, and their parameter settings cannot be dynamically adjusted to reflect the actual state of the molten pool. They also cannot incorporate the thermal gradient distribution and spectral feature differences of the molten pool to adjust the computational rules. After initial defect region segmentation, a standardized multi-feature verification process and contour optimization steps are lacking, resulting in deviations in the defect region segmentation boundaries. The extraction of defect features is relatively simplistic, focusing only on basic morphological parameters and ignoring crucial data such as thermal history changes and spectral distribution, leading to an inadequate defect quantification and characterization system. This single-dimensional feature data input mode limits the discriminative power of defect classification and identification, failing to meet the requirements of continuous online laser cladding inspection. The industry demands a novel multi-feature fusion detection architecture to address the shortcomings of existing technologies in information acquisition area segmentation and feature characterization. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose an online quality inspection system for laser cladding based on machine vision.

[0005] The present invention discloses a machine vision-based online quality inspection system for laser cladding, comprising: a composite feature extraction module, which acquires a multi-band synchronous visual image sequence of the molten pool region during the laser cladding process, performs spatiotemporal alignment and feature layer fusion processing on the multi-band synchronous visual image sequence, and generates a composite visual feature map of the molten pool region;

[0006] The defect region segmentation module calls an improved region growing algorithm to perform preliminary segmentation of the potential defect region in the composite visual feature map. The improved region growing algorithm dynamically adjusts the growth criteria based on the thermal gradient distribution and spectral feature similarity of the molten pool region, performs multi-feature consistency verification and contour trimming on the potential defect region obtained from the preliminary segmentation, and generates a defect region segmentation mask and defect contour geometric information.

[0007] The multidimensional feature quantization module extracts multidimensional quantized feature vectors of the defect region from the composite visual feature map based on the defect region segmentation mask. The multidimensional quantized feature vectors include the morphological descriptor of the defect, thermal history curve features, and spectral intensity distribution features.

[0008] The defect intelligent identification module inputs the multi-dimensional quantized feature vector into a pre-trained multi-class defect identification model and outputs the defect type identifier and the corresponding confidence score.

[0009] As a further aspect of the present invention, the multi-band synchronous visual image sequence is subjected to spatiotemporal alignment and feature layer fusion processing to generate a composite visual feature map of the molten pool region, including:

[0010] The multi-band synchronous visual image sequence includes visible light band images, near-infrared thermal radiation images, and laser-induced emission images of specific wavelengths;

[0011] The composite visual feature map contains pixel-level fusion information of morphological features, thermal features, and spectral features;

[0012] For the visible light band images, near-infrared thermal radiation images and laser-induced emission images acquired at the same time, image spatial registration is performed based on a pre-calibrated multi-camera extrinsic matrix to ensure that the pixel positions of the molten pool region in the images of different bands correspond one-to-one.

[0013] For the spatially registered image sequence, based on the shooting time of the laser-induced emission image, interpolation compensation and alignment are performed on the temporal deviation caused by the exposure time difference between images of different bands to generate a set of multi-band image pairs that are strictly synchronized in time and space.

[0014] Extract edge gradient feature maps and texture feature maps of the molten pool and surrounding area from the spatiotemporally synchronized visible light band image;

[0015] From the spatiotemporally synchronized near-infrared thermal radiation image, extract the isotherm distribution feature map and heat flux density feature map of the molten pool region;

[0016] Extract specific spectral emission intensity feature maps and spectral distribution uniformity feature maps from the spatiotemporally synchronized laser-induced emission images;

[0017] The edge gradient feature map, texture feature map, isotherm distribution feature map, heat flux density feature map, spectral emission intensity feature map, and spectral distribution uniformity feature map are spliced ​​and normalized along the feature channel dimension to generate the composite visual feature map.

[0018] As a further aspect of the present invention, the improved region growing algorithm dynamically adjusts the growth criteria based on the similarity between the thermal gradient distribution and spectral characteristics of the molten pool region. The principle of the improved region growing algorithm includes:

[0019] In the composite visual feature map, the pixels in the high-temperature region at the center of the molten pool are used as the initial seed point set;

[0020] For each seed point to be grown, the feature similarity between its neighboring pixels and the seed point in the composite visual feature map is calculated. The feature similarity is determined by the weighted sum of thermal gradient difference and spectral feature difference.

[0021] Among them, the thermal gradient difference is calculated based on the temperature value and spatial gradient of the pixel in the isotherm distribution feature map, and the spectral feature difference is calculated based on the value of the pixel in the spectral emission intensity feature map and the spectral distribution uniformity feature map.

[0022] A growth threshold is dynamically set, which is negatively correlated with the thermal gradient value at the location of the seed point. The thermal gradient value is quantified by the temperature standard deviation of the seed point in its eight neighborhoods. A stricter similarity threshold is used in areas with large thermal gradients to control growth, while a relatively looser similarity threshold is used in areas with gentle thermal gradients.

[0023] When the feature similarity between a neighboring pixel and a seed point is less than the current dynamic growth threshold, the neighboring pixel is included in the growth region and used as a new seed point.

[0024] The growth process is iteratively executed until no new pixels satisfy the growth criteria, and the resulting connected region is the potential defect region.

[0025] As a further aspect of the present invention, the step of performing multi-feature consistency verification and contour trimming on the initially segmented potential defect region to generate a defect region segmentation mask and defect contour geometric information includes:

[0026] Extract all feature channel data of the potential defect region at the corresponding position in the composite visual feature map;

[0027] Calculate the statistical consistency measure of pixels in each feature channel within the potential defect region. The statistical consistency measure includes the variance and skewness of pixel values ​​within the region.

[0028] If the statistical consistency measure of a certain feature channel exceeds the preset consistency threshold, the potential defect region is determined to be heterogeneous in terms of features, and the preliminary segmentation result is valid.

[0029] For potential defect regions that have passed the consistency verification, active contour evolution is performed using their edge gradient feature maps to drive the initial contour to evolve towards the true edge of the defect, thus obtaining the defect contour.

[0030] The evolved defect contour is approximated by polygons and key points are extracted to generate the geometric information of the defect contour composed of an ordered set of points.

[0031] Based on the geometric information of the defect contour, a binary mask image with the same size as the original image is generated, wherein the pixel value within the defect area is a set value, and the pixel value outside the area is zero. This binary mask image is the defect area segmentation mask.

[0032] As a further aspect of the present invention, based on the defect region segmentation mask, extracting a multi-dimensional quantized feature vector of the defect region from the composite visual feature map includes:

[0033] Using the defect region segmentation mask, a sub-map containing only the defect region is cropped from the composite visual feature map;

[0034] The seven Hu invariant moments of the defect region submap in the morphological feature channel are calculated to form the morphological descriptor;

[0035] Sampling is performed along the defect contour angle, and the thermal history curve of the sampling point in the direction of the inner molten pool is extracted. The thermal history curve is composed of the temperature change data of the corresponding pixel in the near-infrared thermal radiation image sequence over time. Features including the peak heating rate, peak temperature and average cooling rate are extracted from the thermal history curve to form thermal history curve features.

[0036] On the spectral feature channels of the defect region sub-map, the average value, standard deviation, and ratio of the highest intensity to the lowest intensity of the spectral intensity within the defect region are calculated to form the spectral intensity distribution characteristics.

[0037] The morphological descriptor, thermal history curve features, and spectral intensity distribution features are sequentially connected to form a fixed-dimensional multidimensional quantized feature vector.

[0038] As a further aspect of the present invention, the multidimensional quantized feature vector is input into a pre-trained multi-class defect recognition model, and the defect type identifier and corresponding confidence score are output, including:

[0039] The pre-trained multi-class defect recognition model is a classification network trained based on the feature vectors of samples with defect type labels collected from historical laser cladding processes.

[0040] After standardizing and preprocessing the multidimensional quantized feature vector to be identified, it is input into the input layer of the classification network.

[0041] The multidimensional quantized feature vectors are sequentially passed through multiple fully connected layers of the classification network for nonlinear transformation and feature abstraction, and finally the probability distributions belonging to each preset defect category are obtained in the output layer.

[0042] The defect category with the highest probability value is selected as the defect type identifier, and the highest probability value is output as the confidence score.

[0043] As a further aspect of the present invention, the system further includes:

[0044] The real-time decision-making module, based on the defect type identifier, confidence score, and defect contour geometry information, generates real-time quality control instructions for the current laser cladding process, specifically including:

[0045] The real-time quality control commands include laser power adjustment, powder feeding rate correction, and scanning path fine-tuning commands.

[0046] Establish a knowledge base for mapping the relationship between defect types and process parameter deviations. The knowledge base defines which abnormal process parameters cause different defect types.

[0047] Based on the defect type identifier, the mapping relationship knowledge base is queried to obtain candidate process parameter adjustment items and suggested adjustment directions;

[0048] Based on the confidence score, the suggested adjustment amount of the candidate process parameter adjustment item is weighted and corrected. The higher the confidence score, the closer the value is to the suggested adjustment amount.

[0049] Simultaneously, the geometric information of the defect contour is analyzed. If the defect contour is narrow or has a specific direction, it is determined that the defect is related to the scanning path or the flow direction of the molten pool, and a scanning path fine-tuning instruction is generated.

[0050] The weighted and corrected process parameter adjustment amount is combined with the scan path fine-tuning instruction to form the real-time quality control instruction containing specific parameter values ​​and execution timing.

[0051] As a further aspect of the present invention, the thermal gradient difference is calculated based on the temperature value and spatial gradient of the pixel in the isotherm distribution feature map, and the spectral feature difference is calculated based on the values ​​of the pixel in the spectral emission intensity feature map and the spectral distribution uniformity feature map, including:

[0052] For a seed point and its neighboring pixels, the corresponding temperature value is obtained from the isotherm distribution feature map.

[0053] The spatial gradient of the seed point temperature value is calculated, which is approximated by the first-order difference between the temperature values ​​of the pixel and its eight neighboring pixels.

[0054] The thermal gradient difference is defined as the absolute value of the temperature difference between the neighboring pixel and the seed point, plus the Euclidean distance of the spatial gradient vector difference between the neighboring pixel and the seed point, and then normalized.

[0055] The feature values ​​corresponding to the seed point and its neighboring pixels are obtained from the spectral emission intensity feature map and the spectral distribution uniformity feature map, respectively.

[0056] The spectral feature difference is defined as the weighted sum of the differences in feature values ​​between neighboring pixels and seed points in the spectral emission intensity feature map and the differences in feature values ​​in the spectral distribution uniformity feature map.

[0057] The thermal gradient difference and the spectral feature difference are assigned preset weighting coefficients, and the feature similarity is obtained by summing them.

[0058] As a further aspect of the present invention, on the spectral feature channels of the defect region sub-map, the average value, standard deviation, and ratio of the highest intensity to the lowest intensity within the defect region are calculated to constitute spectral intensity distribution characteristics, including:

[0059] Using the defect region segmentation mask, feature values ​​of all pixels within the defect region are extracted from the spectral intensity related feature channel of the composite visual feature map to form a set of spectral intensity values.

[0060] The average value of the spectral intensity is obtained by summing all the values ​​in the set of spectral intensity values ​​and dividing by the total number of pixels.

[0061] Calculate the square of the difference between each value in the set of spectral intensity values ​​and the average value, sum the squared values, divide by the total number of pixels, and then take the square root to obtain the standard deviation of the spectral intensity.

[0062] Traverse the set of spectral intensity values, find the maximum and minimum values, and divide the maximum value by the minimum value to obtain the ratio of the highest intensity to the lowest intensity.

[0063] The calculated average value, standard deviation, and ratio of the spectral intensity are arranged in a predetermined order to form the spectral intensity distribution characteristics.

[0064] As a further aspect of the present invention, the training steps of the pre-trained multi-class defect recognition model include:

[0065] Collect multi-band synchronous visual image sequence samples from historical laser cladding processes, and manually annotate the defect areas in each image sample to obtain a training sample set with defect type labels;

[0066] For each image sample in the training sample set, the same processing flow of the composite feature extraction module, defect region segmentation module, and multidimensional feature quantization module is followed to obtain a training feature set consisting of the multidimensional quantized feature vectors corresponding to all image samples and their defect type labels.

[0067] Construct a fully connected neural network containing an input layer, multiple hidden layers, and an output layer as the initial classification network model;

[0068] The training feature set is input into the initial classification network model for forward propagation, the probability distribution of defect categories predicted by the model is calculated, and the cross-entropy loss function is calculated based on the defect type label and the probability distribution of defect categories predicted by the model.

[0069] The gradient of the cross-entropy loss function with respect to the parameters of each layer in the network is calculated using the backpropagation algorithm, and the network parameters are iteratively updated using an optimizer to minimize the loss function;

[0070] The classification accuracy of the model is monitored on an independent validation set. Training is stopped when the accuracy no longer improves, and the pre-trained multi-class defect recognition model is obtained.

[0071] The parameters of the pre-trained multi-class defect recognition model are stored and saved for loading and use during online quality inspection.

[0072] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0073] A multi-band synchronous visual image sequence of the laser cladding molten pool area was acquired. Spatiotemporal alignment processing was performed on multiple sets of image data to eliminate temporal discrepancies between multi-source images and unify the time reference of data from different acquisition dimensions. Effective information contained in the multi-band images was integrated through feature layer fusion to generate a complete composite visual feature map of the molten pool area. This expands the data dimensions of visual inspection, compensates for information gaps in single-band imaging modes, fully preserves diverse visual information during the molten pool processing, and improves the complete recording of the molten pool's dynamic state, providing diversified basic data for subsequent defect analysis.

[0074] Based on the similarity between the thermal gradient distribution and spectral characteristics of the molten pool region, the judgment criteria of the region growth algorithm are adjusted in real time, breaking the limitations of fixed algorithm parameters and adapting to the dynamic changes of molten pool parameters during the cladding process. Multi-feature consistency verification is performed on the initially segmented target region, and defect contour trimming is completed simultaneously to standardize the boundary range of the defect region, correct the range deviation generated during the region division process, and improve the regularity of the defect region division.

[0075] Based on defect region segmentation masks, directional feature extraction is completed, simultaneously acquiring defect morphological descriptors, thermal history curve features, and spectral intensity distribution features. These are combined to form a multi-dimensional quantified feature vector, expanding the parameterized representation dimension of defect states. Multiple types of feature parameters complement each other, enriching the quantitative expression of defect attributes. A multi-class defect recognition model, combined with multi-dimensional features, completes computational analysis, improves the defect category discrimination logic, refines the attribute distinction of different defect states, and adapts to the defect recognition operation logic under complex laser cladding processing conditions, as well as the normalized detection operation logic for continuous processing. Attached Figure Description

[0076] Figure 1 This is a state diagram of an online quality inspection system for laser cladding based on machine vision, as described in this invention.

[0077] Figure 2 Flowchart for generating composite visual feature maps;

[0078] Figure 3 This is a flowchart for extracting multidimensional quantized feature vectors. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0080] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0081] See Figure 1This invention provides an online quality inspection system for laser cladding based on machine vision. The overall implementation scheme of the system is as follows: A composite feature extraction module acquires a multi-band synchronous visual image sequence of the molten pool area during the laser cladding process. This image sequence undergoes spatiotemporal alignment and feature layer fusion processing to generate a composite visual feature map of the molten pool area. A defect region segmentation module uses an improved region growing algorithm to perform preliminary segmentation of potential defect regions on the composite visual feature map. This improved region growing algorithm dynamically adjusts the growth criteria based on the thermal gradient distribution and spectral feature similarity of the molten pool area. The preliminarily segmented potential defect regions undergo multi-feature consistency verification and contour trimming processing to generate a defect region segmentation mask and defect contour geometric information. A multi-dimensional feature quantization module extracts a multi-dimensional quantized feature vector of the defect region from the composite visual feature map based on the defect region segmentation mask. This vector includes the morphological descriptor of the defect, thermal history curve features, and spectral intensity distribution features. A defect intelligent recognition module inputs this multi-dimensional quantized feature vector into a pre-trained multi-class defect recognition model and outputs a defect type identifier and corresponding confidence score.

[0082] In one embodiment of the present invention, see [reference] Figure 2 The multi-band synchronized visual image sequence includes visible light images, near-infrared thermal radiation images, and laser-induced emission images at specific wavelengths. The composite visual feature map contains pixel-level fusion information of morphological, thermal, and spectral features. For visible light images, near-infrared thermal radiation images, and laser-induced emission images acquired at the same time, spatial registration is performed based on a pre-calibrated multi-camera extrinsic matrix to ensure a one-to-one correspondence of pixel positions in the molten pool region across different image bands. For the spatially registered image sequence, using the capture time of the laser-induced emission image as a reference, interpolation compensation is performed to align the temporal deviations caused by exposure time differences between different image bands, generating a set of spatiotemporally strictly synchronized multi-band image pairs. Edge gradient feature maps and texture feature maps of the molten pool and surrounding areas are extracted from the spatiotemporally synchronized visible light images. Isotherm distribution feature maps and heat flux density feature maps of the molten pool region are extracted from the spatiotemporally synchronized near-infrared thermal radiation images. Specific spectral emission intensity feature maps and spectral distribution uniformity feature maps are extracted from the spatiotemporally synchronized laser-induced emission images. The edge gradient feature map, texture feature map, isotherm distribution feature map, heat flux density feature map, spectral emission intensity feature map, and spectral distribution uniformity feature map are spliced ​​and normalized along the feature channel dimension to generate a composite visual feature map.

[0083] In practical implementation, the composite feature extraction module acquires a multi-band synchronous visual image sequence of the molten pool region during the laser cladding process. This sequence includes visible light images captured by a visible light camera, near-infrared thermal radiation images captured by a near-infrared thermal imager, and laser-induced emission images of a specific wavelength captured by a camera equipped with a specific narrowband filter. In an example of cobalt-based alloy powder cladding on a 304 stainless steel substrate, three imaging devices achieve microsecond-level synchronous acquisition via hardware triggers, obtaining a timestamp-aligned original image sequence. The visible light images clearly present the liquid metal flow morphology and splashing sparks in the molten pool, the near-infrared thermal radiation images record the temperature field distribution of the molten pool and heat-affected zone, and the laser-induced emission images reflect the spatial variation information of plasma electron temperature and density in the molten pool region.

[0084] In practical implementation, image spatial registration is performed on visible light images, near-infrared thermal radiation images, and laser-induced emission images acquired simultaneously, based on a pre-calibrated multi-camera extrinsic parameter matrix. During system installation, a calibration board with a precise checkerboard pattern is placed on the working plane of the laser processing head. Multiple sets of images of the calibration board in different postures are simultaneously acquired by three imaging devices, and the extrinsic parameter matrix of each camera relative to the world coordinate system and the homography transformation matrix between cameras are calculated. During online inspection, based on these matrices, the near-infrared thermal radiation image and the laser-induced emission image are uniformly mapped to the pixel coordinate system of the visible light image through perspective transformation, so that the pixel positions of the molten pool region in the images of different wavelengths are one-to-one. After spatial registration, the high-temperature pixel point at the center of the molten pool with coordinates (x1, y1) in the near-infrared image and the center point of the molten pool with coordinates (x2, y2) in the visible light image are mapped to the same image coordinates (u, v).

[0085] In some embodiments, for the spatially registered image sequence, interpolation compensation and alignment are performed to compensate for temporal deviations caused by exposure time differences between different band images, using the exposure start time of the laser-induced emission image as a reference. Due to slight differences in the response and integration times of different sensors, even with synchronous triggering, the optical information captured by each band image is not necessarily from the same physical moment. In a specific implementation, the system records the precise timestamp of each frame. Before and after the time point of the laser-induced emission image, time-based linear interpolation is performed on the visible light band image and near-infrared thermal radiation image sequences at adjacent times to generate virtual visible light band image frames and virtual near-infrared thermal radiation image frames that strictly correspond to the time of the laser-induced emission image capture, thus forming a set of multi-band image pairs that are strictly synchronized in time and space. For example, if the laser-induced emission image is captured at time T0, and the two preceding and following visible light images are captured at times T0-Δt and T0+Δt respectively, then a weighted average is performed between the two frames at times T0-Δt and T0+Δt to calculate the virtual visible light band image at time T0.

[0086] In specific implementation, edge gradient feature maps and texture feature maps of the molten pool and surrounding area are extracted from the spatiotemporally synchronized visible light images. Edge gradient feature maps are obtained by applying the Sobel operator to calculate the gradient magnitudes of the image in the horizontal and vertical directions, used to characterize the contour information of the molten pool boundary, ripples, and surface abrupt changes. Texture feature maps are obtained by calculating the gray-level co-occurrence matrix within a local window of the image and extracting its contrast feature values, used to quantify the solidification roughness and stripe distribution of the molten pool surface. From the spatiotemporally synchronized near-infrared thermal radiation images, isotherm distribution feature maps and heat flux density feature maps of the molten pool region are extracted. The isotherm distribution feature maps are obtained by thresholding and edge tracking, transforming the continuous temperature field into a discrete set of closed contour lines representing regions of the same temperature. The heat flux density feature maps are obtained by calculating the second derivative of the temperature field in space, used to reflect the direction and intensity of heat conduction. From the spatiotemporally synchronized laser-induced emission images, specific spectral emission intensity feature maps and spectral distribution uniformity feature maps are extracted. The specific spectral emission intensity feature map, which is the result of grayscale processing of the laser-induced emission image itself, directly reflects the spatial distribution of spectral radiation intensity at the selected wavelength. The spectral distribution uniformity feature map is generated by calculating the intensity standard deviation in the local neighborhood of each pixel in the laser-induced emission image, and is used to characterize the local stability of spectral emission. Optionally, the edge gradient feature map, texture feature map, isotherm distribution feature map, heat flux density feature map, spectral emission intensity feature map, and spectral distribution uniformity feature map are stitched and normalized along the feature channel dimension. In specific implementation, the six feature maps are stacked pixel by pixel to form a preliminary fused data cube with six channels. To eliminate the differences in physical dimensions and numerical ranges of different features, each feature channel is independently normalized based on Z-score to generate the final composite visual feature map. Each pixel position in this map contains a six-dimensional feature vector, which comprehensively describes the fused features of that point in terms of morphology, thermal, and spectral properties. The formula used for feature channel normalization is:

[0087]

[0088] Where: symbol Represents the standardized eigenvalues, with the sign... Represents the original eigenvalue, symbol The arithmetic mean of all pixel values ​​in the current feature channel, with the sign... This represents the standard deviation of all pixel values ​​in the current feature channel.

[0089] In one embodiment of the present invention, pixels in the high-temperature region at the center of the molten pool are used as the initial set of seed points in the composite visual feature map. For each seed point to be grown, the feature similarity between its neighboring pixels and the seed point in the composite visual feature map is calculated. This feature similarity is determined by the weighted sum of thermal gradient difference and spectral feature difference. A growth threshold is dynamically set, which is negatively correlated with the thermal gradient value at the location of the seed point. A stricter similarity threshold is used in regions with large thermal gradients to control growth, while a relatively lenient similarity threshold is used in regions with gentle thermal gradients. When the feature similarity between a neighboring pixel and the seed point is less than the current dynamic growth threshold, the neighboring pixel is included in the growth region and used as a new seed point. The growth process is iteratively executed until no new pixels meet the growth criteria, and the resulting connected region is the potential defect region. The thermal gradient difference is calculated based on the temperature value and spatial gradient of the pixel in the isotherm distribution feature map, and the spectral feature difference is calculated based on the values ​​of the pixel in the spectral emission intensity feature map and the spectral distribution uniformity feature map. For the seed point and its neighboring pixels, their corresponding temperature values ​​are obtained from the isotherm distribution feature map, respectively. The spatial gradient of the seed point's temperature value is calculated, approximated by the first-order difference between the pixel's temperature value and that of its eight neighboring pixels. The thermal gradient difference is defined as the absolute value of the temperature difference between the neighboring pixels and the seed point, plus the Euclidean distance between the spatial gradient vector differences of the neighboring pixels and the seed point, and then normalized. Feature values ​​corresponding to the seed point and its neighboring pixels are obtained from the spectral emission intensity feature map and the spectral distribution uniformity feature map, respectively. The spectral feature difference is defined as the weighted sum of the differences in feature values ​​between the neighboring pixels and the seed point in the spectral emission intensity feature map and in the spectral distribution uniformity feature map. Preset weighting coefficients are assigned to the thermal gradient difference and the spectral feature difference, and the sum is used to obtain the feature similarity.

[0090] In practical implementation, the defect region segmentation module calls an improved region growing algorithm to perform preliminary segmentation of potential defect regions in the composite visual feature map. The improved algorithm uses pixels in the high-temperature region at the center of the molten pool as the initial seed point set in the composite visual feature map. In an example scenario for laser cladding of Inconel 718 nickel-based alloy, the system identifies all pixels with temperatures higher than a preset threshold from the isotherm distribution feature map channel of the composite visual feature map. The geometric center point of the connected domain formed by these pixels and its eight neighboring pixels are then included in the initial seed point set, which represents the core high-temperature region of the molten pool. Data comparison shows that the temperature values ​​of pixels in the central region of the molten pool in the isotherm distribution feature map are between 1750°C and 1950°C, while the temperature values ​​in the edge region of the molten pool rapidly drop below 1400°C. This significant difference in thermal gradient is the basis for the algorithm to initiate segmentation.

[0091] In practice, for each seed point to be grown, the feature similarity between its neighboring pixels and the seed point in the composite visual feature map is calculated. This feature similarity is determined by a weighted sum of thermal gradient difference and spectral feature difference. In one processing iteration, a seed point located in the transition zone at the edge of the melt pool with a temperature of 1520°C is selected. The system then iterates through every pixel in its eight neighborhoods. For a given neighboring pixel with a temperature of 1450°C, the system reads the values ​​of these two pixels from the corresponding positions in the isotherm distribution feature map, spectral emission intensity feature map, and spectral distribution uniformity feature map of the composite visual feature map. The thermal gradient difference is calculated based on the pixel's temperature value and its spatial gradient in the isotherm distribution feature map, while the spectral feature difference is calculated based on the pixel's values ​​in the spectral emission intensity feature map and the spectral distribution uniformity feature map. The system calculates the spatial gradient of the seed point's temperature value, which is approximated by the first-order difference between the pixel's temperature value and that of its eight neighboring pixels, generating a two-dimensional gradient vector. The thermal gradient difference is defined as the absolute value of the temperature difference between a neighboring pixel and a seed point, plus the Euclidean distance between the spatial gradient vector differences between the neighboring pixel and the seed point, and then normalized. For example, if the absolute value of the temperature difference between the seed point and the neighboring pixel is 70°C, and the Euclidean distance between the two gradient vectors is calculated to be 15°C / pixel, these two values ​​are normalized to the [0,1] interval and then weighted and summed to obtain the thermal gradient difference. .

[0092] In some embodiments, feature values ​​corresponding to the seed point and its neighboring pixels are obtained from the spectral emission intensity feature map and the spectral distribution uniformity feature map, respectively. The spectral feature difference is defined as the weighted sum of the squared differences between the feature values ​​of the neighboring pixels and the seed point in the spectral emission intensity feature map and the feature values ​​in the spectral distribution uniformity feature map. For example, the normalized intensity value of the seed point in the spectral emission intensity feature map is 0.85, and that of the neighboring pixels is 0.72; in the spectral distribution uniformity feature map, the seed point value is 0.10 (indicating uniformity), and that of the neighboring pixels is 0.25 (indicating non-uniformity). The system assigns weight coefficients w1 and w2 to these two differences and then calculates the weighted sum of squares. Finally, the thermal gradient difference is... Difference from spectral characteristics Assign preset weight coefficients α and β respectively, and sum them to obtain the final feature similarity S. The calculation formula is as follows:

[0093]

[0094] Where: symbol This represents the calculated feature similarity; the smaller the value, the more similar the features. Represents the normalized difference in thermal gradients; symbol Represents the degree of difference in spectral characteristics after normalization; symbol and These are preset weighting coefficients that satisfy... In specific implementation, α=0.6 and β=0.4 can be used. The preset values ​​of the weight coefficients α and β need to be pre-calibrated experimentally based on the specific laser cladding process parameters and the characteristics of the processed material. The process parameters include key variables such as laser power and scanning speed, while the processed materials cover various types such as stainless steel, titanium alloys, and nickel-based alloys. The calibration method involves collecting multi-band synchronous visual image samples of typical defect states under the material process to form a training set; traversing different (α, β) combinations within a set numerical range (e.g., α from 0.1 to 0.9, with a step size of 0.1); for each weight set, executing the improved region growing algorithm and subsequent processing flow, with the accuracy and recall of defect segmentation as optimization objectives, and selecting the weight combination that optimizes the objective function as the preset value under the material process. The finally determined (α, β) values ​​will be fixed in the system configuration parameters for use during online detection.

[0095] In practice, a growth threshold is dynamically set, which is negatively correlated with the thermal gradient value at the seed point's location. A stricter similarity threshold is used to control growth in regions with large thermal gradients, while a more lenient similarity threshold is used in regions with gentle thermal gradients. The system quantifies the local thermal gradient value based on the temperature standard deviation of the seed point within its eight neighborhoods. For example, in regions with drastic temperature changes at the melt pool boundary, the local temperature standard deviation of the seed point can reach over 80°C, at which point the corresponding dynamic growth threshold... The threshold value is set to 0.15; however, in the heat-affected zone outside the molten pool, the temperature changes gradually, and the local temperature standard deviation may only be 10°C. In this case, the dynamic growth threshold value is... It can be set to 0.25. This dynamic mechanism ensures that the growth process can respond sensitively to the differences in physical properties of different regions of the molten pool, prevent over-segmentation at the boundaries of drastic temperature changes, and allow sufficient region merging in uniform regions to capture potential porosity defects.

[0096] When the feature similarity between a neighboring pixel and a seed point is less than the current dynamic growth threshold, the neighboring pixel is included in the growth region and used as a new seed point. The growth process is iteratively executed until no new pixels satisfy the growth criteria; the resulting connected region is the potential defect region. During algorithm execution, the system maintains an active seed point queue and a binary mask of the grown region. Taking the aforementioned neighboring pixels as an example, if the calculated feature similarity S is 0.20, and the current dynamic growth threshold... The value is 0.25, since S < The neighboring pixel is marked as grown (set to 1 in the mask) and added to the tail of the active seed point queue. The algorithm takes the next seed point from the head of the queue and repeats the process of calculating the similarity of its neighboring pixel features and comparing it with the current dynamic growth threshold. The iteration continues until the active seed point queue is empty. In the Inconel718 cladding example, when the iteration finally stops, the generated connected region may be a region that has a significant discontinuity with the surrounding melt pool region in terms of thermal gradient and spectral features. This region is marked as a potential defect region.

[0097] In one embodiment of the present invention, all feature channel data of the corresponding position of the potential defect region in the composite visual feature map are extracted. A statistical consistency measure of pixels within the potential defect region on each feature channel is calculated. This statistical consistency measure includes the variance and skewness of pixel values ​​within the region. If the statistical consistency measure of a certain feature channel exceeds a preset consistency threshold, the potential defect region is determined to be heterogeneous in features, and the preliminary segmentation result is valid. For potential defect regions that pass the consistency verification, active contour evolution is performed using their edge gradient feature maps to drive the initial contour to evolve towards the true edge of the defect, obtaining the defect contour. Polygon approximation and key point extraction are performed on the evolved defect contour to generate defect contour geometric information composed of an ordered set of points. Based on the defect contour geometric information, a binary mask image with the same size as the original image is generated, where pixel values ​​within the defect region are set values, and pixel values ​​outside the region are zero. This binary mask image is the defect region segmentation mask. (See also...) Figure 3 Using a defect region segmentation mask, a sub-map containing only the defect region is cropped from the composite visual feature map. Seven Hu invariant moments in the morphological feature channel of the defect region sub-map are calculated to form a morphological descriptor. Sampling is performed along the defect contour angle to extract the thermal history curve of the sampling point in the direction of the inner molten pool. This thermal history curve is composed of the temperature change data of the corresponding pixels in the near-infrared thermal radiation image sequence over time. Features including the peak heating rate, peak temperature, and average cooling rate are extracted from this thermal history curve to form thermal history curve features. In the spectral feature channel of the defect region sub-map, the average value, standard deviation, and ratio of the highest to lowest intensity of the spectral intensity within the defect region are calculated to form spectral intensity distribution features. The morphological descriptor, thermal history curve features, and spectral intensity distribution features are sequentially concatenated to form a fixed-dimensional multi-dimensional quantized feature vector.

[0098] Specifically, in the spectral feature channels of the defect region sub-map, the average value, standard deviation, and ratio of the highest to lowest intensity of the spectral intensity within the defect region are calculated to form the spectral intensity distribution feature. This includes: using a defect region segmentation mask, extracting feature values ​​of all pixels within the defect region in the spectral intensity-related feature channels of the composite visual feature map to form a set of spectral intensity values; summing all values ​​in the spectral intensity value set and dividing by the total number of pixels to obtain the average spectral intensity; calculating the square of the difference between each value in the spectral intensity value set and the average value, summing the squared values, dividing by the total number of pixels, and taking the square root to obtain the standard deviation of the spectral intensity; traversing the spectral intensity value set to find the maximum and minimum values, and dividing the maximum value by the minimum value to obtain the ratio of the highest to the lowest intensity; arranging the calculated average value, standard deviation, and ratio of the spectral intensity in a predetermined order to form the spectral intensity distribution feature.

[0099] In practical implementation, all feature channel data of the potential defect area at the corresponding position in the composite visual feature map are extracted. The composite visual feature map includes six feature channels: edge gradient feature map, texture feature map, isotherm distribution feature map, heat flux density feature map, spectral emission intensity feature map, and spectral distribution uniformity feature map. In a scenario involving laser cladding of titanium alloys, the improved region growing algorithm initially segments an approximately circular region suspected of having pores. The system maps this region back to the composite visual feature map and extracts the values ​​of all pixels within the region in the six channels. The statistical consistency measure of pixels within the potential defect area in each feature channel is calculated. The statistical consistency measure includes the variance and skewness of the pixel values ​​within the region. Variance measures the dispersion of feature values ​​within the region, and skewness describes the asymmetry of the feature value distribution. For this initially segmented suspected pore area, the system calculates its variance and skewness values ​​in each of the six feature channels. For example, in the isotherm distribution feature map channel, the temperature values ​​within the region are relatively uniform, and the variance is small; while in the texture feature map channel, due to the difference in texture between the inside of the pore and the surrounding material, the variance value increases significantly. For a detailed data comparison, please refer to Table 1, which shows the statistical measures of a potential defect area on two different feature channels:

[0100] Table 1: Statistical Consistency Measurement Table for Feature Channels of Potential Defect Regions

[0101] isotherm profile 12.5 -0.1 texture profile 85.7 1.8

[0102] In some embodiments, if the statistical consistency measure of a certain feature channel exceeds a preset consistency threshold, the potential defect region is determined to have heterogeneity in features, and the preliminary segmentation result is valid. The preset consistency threshold includes a variance threshold and a skewness threshold. Based on statistical analysis of a large number of known defect samples, the system sets the variance threshold of the texture feature map to 50.0 and the skewness threshold to 1.5. Comparing the data in the table above, the variance value of the potential defect region in the texture feature map channel is 85.7, which is greater than the variance threshold of 50.0, and the skewness value is 1.8, which is greater than the skewness threshold of 1.5. This indicates that the region exhibits significant internal heterogeneity in texture features and has statistical inconsistency with the surrounding melt pool region. Therefore, the preliminary segmentation result passes the validity verification. If the statistical consistency measure of a region is lower than the threshold in all feature channels, the region is considered a uniform melt pool normal region, and the segmentation result will be discarded.

[0103] In practical implementation, for potential defect regions that have passed consistency verification, active contour evolution is performed using their edge gradient feature maps, driving the initial contour to evolve towards the true edge of the defect. The system uses the region boundary obtained from the initial segmentation as the initial contour curve of the active contour model, and the edge gradient feature map provides the strong field of the image gradient as external energy. During the iterative evolution process, the contour curve deforms under the combined action of internal elastic energy and external image gradient energy, and is attracted to the edge position where the gradient change between the defect and the substrate is most drastic, resulting in an accurate defect contour. The evolved defect contour is then approximated with polygons and keypoints are extracted to generate the geometric information of the defect contour composed of an ordered set of points. The system uses the Douglas-Peucker algorithm to approximate the contour curve with polygons, using fewer vertex sequences to represent the contour shape within the allowable fitting error, and then extracting the corner points of the polygon as keypoints. For example, an approximately circular pore defect may ultimately be represented as a polygonal contour composed of 36 ordered points and its geometric center coordinates. Based on the geometric information of the defect contour, a binary mask image with the same size as the original image is generated, in which the pixel value in the defect area is set to 1, and the pixel value outside the area is 0. This binary mask image is the defect area segmentation mask.

[0104] Optionally, a defect region segmentation mask is used to crop a sub-map containing only the defect region from the composite visual feature map. Seven Hu invariant moments in the morphological feature channels of the defect region sub-map are calculated to form a morphological descriptor. The Hu invariant moments are constructed using second and third-order central moments, resulting in seven moment invariants invariant to translation, rotation, and scaling. The system calculates these seven moment values ​​for the binarized regions of the morphological feature channels (mainly derived from the edge and texture information of the original visible light image) on the cropped defect region sub-map. For example, the seven-dimensional Hu moment vectors calculated for a narrow crack defect and a circular pore defect will differ significantly in value. Sampling is performed along the defect contour angle, and the thermal history curves of the sampling points in the direction of the inner molten pool are extracted. The thermal history curves are composed of the temperature change data of the corresponding pixels in the near-infrared thermal radiation image sequence over time. The system sets a sampling point at regular angles along the defect contour. Extending a short distance from each sampling point towards the center of the molten pool, it acquires the temperature sequence of all pixels along this short line segment over a previous period (e.g., the most recent 0.5 seconds). These temperature sequences are averaged to obtain the thermal history curve corresponding to that sampling angle. Features including the peak heating rate, peak temperature, and average cooling rate are extracted from the thermal history curve to form its features. For example, at the edge of a porosity defect, due to faster heat loss, the average cooling rate will be significantly higher than in normal areas.

[0105] In practical implementation, on the spectral feature channels of the defect region sub-map, the average value, standard deviation, and ratio of the highest to lowest intensity of the spectral intensity within the defect region are calculated to form the spectral intensity distribution characteristics. Using a defect region segmentation mask, feature values ​​of all pixels within the defect region are extracted on the spectral intensity-related feature channels of the composite visual feature map, forming a set of spectral intensity values. The average spectral intensity is obtained by summing all values ​​in the set and dividing by the total number of pixels. The square of the difference between each value and the average is calculated, and the sum of these squares is divided by the total number of pixels, then the square root is taken to obtain the standard deviation of the spectral intensity. The formula for calculating the standard deviation is:

[0106]

[0107] Where: symbol Represents the calculated standard deviation of spectral intensity; symbol Represents the total number of pixels within the defective area; symbol The first value in the set of spectral intensity values Feature values ​​of each pixel; symbol This represents the average value of the set of spectral intensity values. The process iterates through the set of spectral intensity values ​​to find the maximum value. and minimum value Dividing the maximum value by the minimum value yields the ratio of the highest strength to the lowest strength. It is understandable that a plasma scintillation region exhibiting hyperspectral emission has a ratio... It will be much larger than a spectrally uniform pore region. The average value of the calculated spectral intensity will be... Standard deviation and ratio The morphological descriptors (seven Hu moments), thermal history curve features (e.g., three features), and spectral intensity distribution features (three features) are arranged in a predetermined order to form a fixed-dimensional multidimensional quantized feature vector. For example, the final generated multidimensional quantized feature vector may be a 13-dimensional numerical vector.

[0108] In one embodiment of the present invention, the pre-trained multi-class defect recognition model is a classification network trained based on sample feature vectors with defect type labels collected from historical laser cladding processes. The multi-dimensional quantized feature vectors to be identified are standardized and preprocessed before being input to the input layer of the classification network. The multi-dimensional quantized feature vectors are sequentially passed through multiple fully connected layers of the classification network for nonlinear transformation and feature abstraction, resulting in a probability distribution belonging to each preset defect category at the output layer. The defect category with the highest probability value is selected as the defect type identifier, and the highest probability value is output as the confidence score.

[0109] The training steps of the pre-trained multi-class defect recognition model include: collecting multi-band synchronous visual image sequence samples from historical laser cladding processes, and manually annotating the defect regions in each image sample to obtain a training sample set with defect type labels. For each image sample in the training sample set, the same processing flow as the composite feature extraction module, defect region segmentation module, and multi-dimensional feature quantization module is used to obtain a training feature set composed of multi-dimensional quantized feature vectors corresponding to all image samples and their defect type labels. A fully connected neural network containing an input layer, multiple hidden layers, and an output layer is constructed as the initial classification network model. The training feature set is input into the initial classification network model for forward propagation, calculating the probability distribution of the defect category predicted by the model. Based on the defect type labels and the probability distribution of the defect category predicted by the model, the cross-entropy loss function is calculated. The gradient of the cross-entropy loss function with respect to the parameters of each layer in the network is calculated using the backpropagation algorithm, and the network parameters are iteratively updated using an optimizer to minimize the loss function. The classification accuracy of the model is monitored on an independent validation set. Training stops when the accuracy no longer improves, resulting in the pre-trained multi-class defect recognition model. The parameters of the pre-trained multi-class defect recognition model are stored and saved for loading and use during online quality inspection.

[0110] In practical implementation, the intelligent defect identification module inputs the multidimensional quantized feature vectors generated by the multidimensional feature quantization module into the pre-trained multi-class defect identification model, outputting the defect type identifier and corresponding confidence score. The pre-trained multi-class defect identification model is a classification network trained based on sample feature vectors with defect type labels collected from historical laser cladding processes. In applications targeting nickel-based superalloy cladding layers, the historically collected defect types include porosity, cracks, lack of fusion, spheroidization, and "no defects," among others. Each defect type has hundreds of corresponding sample feature vectors and their manually labeled attributes, forming the basis for model training.

[0111] In practice, the multidimensional quantized feature vector to be identified is standardized and preprocessed before being input into the input layer of the classification network. The multidimensional quantized feature vector is a 13-dimensional numerical vector containing morphological, thermal history, and spectral features extracted from the currently detected molten pool region. The standardization preprocessing uses Z-score normalization based on the mean and standard deviation of each feature dimension calculated during model training, ensuring that each feature dimension has zero mean and unit variance to suit the numerical characteristics of the classification network. For example, a feature vector to be identified extracted from the current cladding channel is first subtracted from the mean vector of the training set and then divided by the standard deviation vector of the training set before being input into the network. The multidimensional quantized feature vector undergoes nonlinear transformation and feature abstraction through multiple fully connected layers of the classification network, ultimately yielding the probability distribution belonging to each preset defect category at the output layer. A specific network structure consists of a 13-dimensional input layer, two hidden layers with 64 and 32 neurons respectively, each followed by a ReLU activation function and a Dropout layer, and an output layer equal to the number of defect categories. The output layer uses a Softmax activation function to normalize the output values ​​into a probability distribution. It can be understood that the forward propagation process of the network is essentially the process by which the input feature vector is mapped to the defect category space through a series of weighted sums and nonlinear mappings.

[0112] In some embodiments, the defect category with the highest probability value is selected as the defect type identifier, and the highest probability value is output as the confidence score. For example, for an input feature vector, the probability distribution calculated by the output layer of the classification network is: porosity 0.02, crack 0.85, lack of fusion 0.10, and spheroidization 0.03. The system selects the defect category "crack" corresponding to the highest probability value of 0.85 as the defect type identifier for this identification, and uses 0.85 as the confidence score. This score reflects the model's grasp of the classification result; a high score usually means that the input features are very close to the historical sample features of the "crack" category in the feature space learned by the model.

[0113] The training steps of the pre-trained multi-class defect recognition model include: collecting multi-band synchronous visual image sequence samples from historical laser cladding processes, and manually annotating the defect regions in each image sample to obtain a training sample set with defect type labels. For each image sample in the training sample set, the same processing flow as the composite feature extraction module, defect region segmentation module, and multi-dimensional feature quantization module is followed to obtain a training feature set composed of multi-dimensional quantized feature vectors corresponding to all image samples and their defect type labels. A fully connected neural network containing an input layer, multiple hidden layers, and an output layer is constructed as the initial classification network model. See Table 2 for a simplified small training feature set.

[0114] Table 2: Multidimensional quantized feature vector table of partial samples in the training feature set

[0115] 1 0.12 0.05 120.5 1560.2 0.87 0.05 pores 2 0.01 0.20 85.3 1480.7 0.45 0.25 cracks 3 0.30 0.01 95.1 1620.8 0.91 0.08 no defects

[0116] Optionally, the training feature set is input into the initial classification network model for forward propagation. The probability distribution of the defect categories predicted by the model is calculated. Based on the defect type label and the probability distribution of the defect categories predicted by the model, the cross-entropy loss function is calculated. The specific formula for calculating the loss function is as follows:

[0117]

[0118] Where: symbol Represents the average cross-entropy loss of a batch; symbol Represents the number of samples in the current training batch; symbol Represents the total number of defect categories; symbol It is an indicator function, if the sample The true category is The value is 1 if it is 1, otherwise it is 0; symbol Representative model predicts samples Category The probability of loss is calculated. The gradient of the cross-entropy loss function with respect to the parameters of each layer in the network is calculated using the backpropagation algorithm, and an optimizer is used to iteratively update the network parameters to minimize the loss function. In the specific implementation, the Adam optimizer is used, with a learning rate of 0.001 and a batch size of 32, to perform multiple rounds of iterative updates to the network parameters.

[0119] The model's classification accuracy is monitored on a validation set independent of the training set. Training stops when the accuracy no longer improves, resulting in a pre-trained multi-class defect recognition model. For example, if the training process lasts 200 epochs, the validation set accuracy reaches 98.5% after 170 epochs and stabilizes, fluctuating between 98.3% and 98.6% in subsequent epochs, then the system triggers an early stop mechanism at the 180th epoch to terminate training. The validation set serves to evaluate the model's generalization ability on unseen data, preventing overfitting on the training set. The pre-trained multi-class defect recognition model's network structure definition file, layer weights, and biases are saved as a model file for online quality detection. When the online detection system starts, the defect intelligent recognition module reads this model file from storage, loads the network parameters into memory, and can then directly perform rapid classification inference on newly input multi-dimensional quantized feature vectors.

[0120] In one embodiment of the present invention, the real-time quality control command includes laser power adjustment, powder feed rate correction, and scanning path fine-tuning. A knowledge base mapping defect types to process parameter deviations is established, defining which process parameter anomalies cause different defect types. The knowledge base is queried based on the defect type identifier to obtain candidate process parameter adjustment items and suggested adjustment directions. The suggested adjustment amounts of the candidate process parameter adjustment items are weighted and corrected based on a confidence score; the higher the confidence score, the closer the value is to the suggested adjustment amount. Simultaneously, the geometric information of the defect contour is analyzed. If the defect contour is elongated or has a specific orientation, it is determined that the defect is related to the scanning path or the melt pool flow direction, generating a scanning path fine-tuning command. The weighted and corrected process parameter adjustment amounts are combined with the scanning path fine-tuning command to form a real-time quality control command containing specific parameter values ​​and execution timing.

[0121] In practical implementation, the system's real-time decision-making module generates real-time quality control instructions for the current laser cladding process based on the defect type identifier, corresponding confidence score, and defect contour geometry information provided by the defect region segmentation module, all output by the intelligent defect identification module. These instructions include laser power adjustment, powder feed rate correction, and scanning path fine-tuning. In an online inspection example of 316L stainless steel cladding, the intelligent defect identification module identified a "porosity" defect in the current molten pool area, identifying it as "Porosity," and outputting a confidence score of 0.92. The defect contour geometry information described the defect as an approximately circular region.

[0122] In practical implementation, a knowledge base mapping defect types to process parameter deviations is established. This knowledge base defines which abnormal process parameters cause different defect types. The knowledge base is stored in the form of lookup tables or rule sets, and its content is derived from process knowledge and historical data. For example, one record in the knowledge base associates the "porosity" defect with "insufficient protective gas flow" and "excessive powder feed rate leading to incomplete powder melting," with corresponding process parameter adjustment suggestions of "increasing laser power" and "decreasing powder feed rate." Another record associates the "crack" defect with "excessive cooling rate" and "insufficient heat input," with suggested adjustments of "decreasing scanning speed" and "preheating the substrate." The knowledge base is queried based on the defect type identifier to obtain candidate process parameter adjustment items and suggested adjustment directions. In the example where the "porosity" defect was identified, the candidate adjustment items obtained after querying the knowledge base are laser power and powder feed rate, with corresponding suggested adjustment directions of positive adjustment (increase) and negative adjustment (decrease), respectively.

[0123] In some embodiments, the suggested adjustment amounts for candidate process parameter adjustment items are weighted and adjusted based on confidence scores; the higher the confidence score, the closer the value is to the suggested adjustment amount. The mapping relationship knowledge base not only defines the adjustment direction but also presets a basic suggested adjustment amount for each adjustment item of each type of defect. For example, for "porosity" defects, the basic recommended adjustment for laser power is an increase of 50W, and the basic recommended adjustment for powder feed rate is a decrease of 1.5g / min. Weighted correction is achieved by directly multiplying the basic recommended adjustment by the confidence score to generate the final adjustment. The corrected formula is expressed as:

[0124]

[0125] Where: symbol The amount of process parameter adjustment representing the final decision to be implemented; symbol This represents the basic suggested adjustment amount extracted from the mapping knowledge base for a specific defect type and adjustment item; its value can be positive or negative to indicate an increase or decrease; (symbol) This represents the confidence score output by the defect intelligent identification module, with a value between 0 and 1. Taking the aforementioned "porosity" defect as an example, the confidence score c=0.92, and the laser power base adjustment... =+50W, Basic adjustment amount for powder delivery rate =-1.5g / min. After weighted correction, the final laser power adjustment amount was determined. =50 * 0.92 = 46W (increase), powder feeding rate correction amount =-1.5*0.92=-1.38g / min (decrease). This is understandable; if the confidence score is 0.5, the adjustment amount will be halved, reflecting the decision's reliance on the reliability of the identification results.

[0126] In practice, the system simultaneously analyzes the geometric information of the defect contour. If the defect contour is elongated or has a specific orientation, it is determined that the defect is related to the scanning path or the flow direction of the molten pool, and a scan path fine-tuning instruction is generated. The system performs principal component analysis on the ordered set of points in the defect contour geometry to calculate the aspect ratio and principal orientation angle of the defect contour. The aspect ratio is defined as the ratio of the length of the major axis to the minor axis of the circumscribed rectangle of the contour. In another example where a "crack" defect was identified, the defect contour geometry showed an aspect ratio of 8:1, and the principal orientation angle was basically parallel to the current cladding scan direction. This meets the criteria of being "elongated and having an orientation related to the scan direction," and the system therefore generates a scan path fine-tuning instruction. The content of this fine-tuning instruction may be "to superimpose periodic micro-amplitude vibrations perpendicular to the cladding direction in subsequent scan paths" to disperse the energy distribution and prevent crack propagation.

[0127] Optionally, the weighted adjusted process parameters can be combined with the scan path fine-tuning command to form a real-time quality control command containing specific parameter values ​​and execution timing. The system superimposes the calculated adjustment amount with the real-time process parameters of the current equipment operation. For example, if the current laser power is 2000W, the powder feeding rate is 15g / min, and the scan path is a straight line, the generated real-time quality control command, based on the aforementioned "porosity" defect decision, would be: "After 0.5 seconds, adjust the laser power setting to 2046W, adjust the powder feeding rate setting to 13.62g / min, and keep the scan path mode unchanged." For the aforementioned "crack" defect, assuming its weighted adjusted power adjustment is a reduction of 30W, the command might be: "After 0.3 seconds, adjust the laser power setting to 1970W, and simultaneously enable a vibration scan mode with a lateral amplitude of 0.2mm and a frequency of 200Hz." It can be understood that the execution timing setting considers the control system response delay and process stability, ensuring that the adjustment command takes effect at the appropriate process cycle node. Real-time quality control commands are transmitted in real time to the motion controller, laser power controller, and powder feeder flow controller of the laser cladding equipment in the form of digital signals via fieldbus or industrial Ethernet interface.

[0128] The online quality inspection system and the laser cladding equipment's actuators are physically connected and interact with each other via an industrial communication network. The system's real-time decision-making module typically runs on an industrial control computer. This computer establishes a physical link with the controllers at the equipment's actuator end, including motion controllers, laser power controllers, and powder feeder flow controllers, by adding or integrating appropriate industrial communication interface cards, such as PROFIBUS communication cards or EtherCAT master cards. The physical connection is usually achieved using industrial-standard communication cables (such as twisted-pair cables or fiber optic cables). The communication protocol depends on the selected industrial network standard and can be a fieldbus protocol (such as PROFIBUS-DP or CANopen) or an industrial Ethernet protocol (such as PROFINET, EtherNet / IP, or Modbus TCP / IP). Following the specifications of the selected protocol, the system encapsulates real-time quality control commands containing specific adjustment parameters into data frames and sends them to the corresponding equipment controllers through the established communication link. Each controller receives and parses the commands, then drives the corresponding actuators (such as the laser power supply, powder feeder motor, and scanning galvanometer) to complete the parameter adjustments, thereby achieving closed-loop control of the processing quality.

[0129] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A machine vision-based online quality inspection system for laser cladding, characterized in that, The system includes: The composite feature extraction module acquires a multi-band synchronous visual image sequence of the molten pool region during the laser cladding process, performs spatiotemporal alignment and feature layer fusion processing on the multi-band synchronous visual image sequence, and generates a composite visual feature map of the molten pool region. The defect region segmentation module calls an improved region growing algorithm to perform preliminary segmentation of the potential defect region in the composite visual feature map. The improved region growing algorithm dynamically adjusts the growth criteria based on the thermal gradient distribution and spectral feature similarity of the molten pool region, performs multi-feature consistency verification and contour trimming on the potential defect region obtained from the preliminary segmentation, and generates a defect region segmentation mask and defect contour geometric information. The multidimensional feature quantization module extracts multidimensional quantized feature vectors of the defect region from the composite visual feature map based on the defect region segmentation mask. The multidimensional quantized feature vectors include the morphological descriptor of the defect, thermal history curve features, and spectral intensity distribution features. The defect intelligent identification module inputs the multi-dimensional quantized feature vector into a pre-trained multi-class defect identification model and outputs the defect type identifier and the corresponding confidence score.

2. The machine vision-based online quality inspection system for laser cladding as described in claim 1, characterized in that, The multi-band synchronized visual image sequence is subjected to spatiotemporal alignment and feature layer fusion processing to generate a composite visual feature map of the molten pool region, including: The multi-band synchronous visual image sequence includes visible light band images, near-infrared thermal radiation images, and laser-induced emission images of specific wavelengths; The composite visual feature map contains pixel-level fusion information of morphological features, thermal features, and spectral features; For the visible light band images, near-infrared thermal radiation images and laser-induced emission images acquired at the same time, image spatial registration is performed based on a pre-calibrated multi-camera extrinsic matrix to ensure that the pixel positions of the molten pool region in the images of different bands correspond one-to-one. For the spatially registered image sequence, based on the shooting time of the laser-induced emission image, interpolation compensation and alignment are performed on the temporal deviation caused by the exposure time difference between images of different bands to generate a set of multi-band image pairs that are strictly synchronized in time and space. Extract edge gradient feature maps and texture feature maps of the molten pool and surrounding area from the spatiotemporally synchronized visible light band image; From the spatiotemporally synchronized near-infrared thermal radiation image, extract the isotherm distribution feature map and heat flux density feature map of the molten pool region; Extract specific spectral emission intensity feature maps and spectral distribution uniformity feature maps from the spatiotemporally synchronized laser-induced emission images; The edge gradient feature map, texture feature map, isotherm distribution feature map, heat flux density feature map, spectral emission intensity feature map, and spectral distribution uniformity feature map are spliced ​​and normalized along the feature channel dimension to generate the composite visual feature map.

3. The machine vision-based online quality inspection system for laser cladding as described in claim 1, characterized in that, The improved region growing algorithm dynamically adjusts the growth criteria based on the similarity between the thermal gradient distribution and spectral characteristics of the molten pool region. The principle of the improved region growing algorithm includes: In the composite visual feature map, the pixels in the high-temperature region at the center of the molten pool are used as the initial seed point set; For each seed point to be grown, the feature similarity between its neighboring pixels and the seed point in the composite visual feature map is calculated. The feature similarity is determined by the weighted sum of thermal gradient difference and spectral feature difference. Among them, the thermal gradient difference is calculated based on the temperature value and spatial gradient of the pixel in the isotherm distribution feature map, and the spectral feature difference is calculated based on the value of the pixel in the spectral emission intensity feature map and the spectral distribution uniformity feature map. A growth threshold is dynamically set, which is negatively correlated with the thermal gradient value at the location of the seed point. The thermal gradient value is quantified by the temperature standard deviation of the seed point in its eight neighborhoods. A stricter similarity threshold is used in areas with large thermal gradients to control growth, while a relatively looser similarity threshold is used in areas with gentle thermal gradients. When the feature similarity between a neighboring pixel and a seed point is less than the current dynamic growth threshold, the neighboring pixel is included in the growth region and used as a new seed point. The growth process is iteratively executed until no new pixels satisfy the growth criteria, and the resulting connected region is the potential defect region.

4. The machine vision-based online quality inspection system for laser cladding as described in claim 1, characterized in that, The process of performing multi-feature consistency verification and contour trimming on the initially segmented potential defect regions to generate defect region segmentation masks and defect contour geometric information includes: Extract all feature channel data of the potential defect region at the corresponding position in the composite visual feature map; Calculate the statistical consistency measure of pixels in each feature channel within the potential defect region. The statistical consistency measure includes the variance and skewness of pixel values ​​within the region. If the statistical consistency measure of a certain feature channel exceeds the preset consistency threshold, the potential defect region is determined to be heterogeneous in terms of features, and the preliminary segmentation result is valid. For potential defect regions that have passed the consistency verification, active contour evolution is performed using their edge gradient feature maps to drive the initial contour to evolve towards the true edge of the defect, thus obtaining the defect contour. The evolved defect contour is approximated by polygons and key points are extracted to generate the geometric information of the defect contour composed of an ordered set of points. Based on the geometric information of the defect contour, a binary mask image with the same size as the original image is generated, wherein the pixel value within the defect area is a set value, and the pixel value outside the area is zero. This binary mask image is the defect area segmentation mask.

5. The machine vision-based online quality inspection system for laser cladding according to claim 4, characterized in that, Based on the defect region segmentation mask, extract the multi-dimensional quantized feature vector of the defect region from the composite visual feature map, including: Using the defect region segmentation mask, a sub-map containing only the defect region is cropped from the composite visual feature map; The seven Hu invariant moments of the defect region submap in the morphological feature channel are calculated to form the morphological descriptor; Sampling is performed along the defect contour angle, and the thermal history curve of the sampling point in the direction of the inner molten pool is extracted. The thermal history curve is composed of the temperature change data of the corresponding pixel in the near-infrared thermal radiation image sequence over time. Features including the peak heating rate, peak temperature and average cooling rate are extracted from the thermal history curve to form thermal history curve features. On the spectral feature channels of the defect region sub-map, the average value, standard deviation, and ratio of the highest intensity to the lowest intensity of the spectral intensity within the defect region are calculated to form the spectral intensity distribution characteristics. The morphological descriptor, thermal history curve features, and spectral intensity distribution features are sequentially connected to form a fixed-dimensional multidimensional quantized feature vector.

6. The machine vision-based online quality inspection system for laser cladding according to claim 1, characterized in that, The multidimensional quantized feature vector is input into a pre-trained multi-class defect recognition model, which outputs a defect type identifier and a corresponding confidence score, including: The pre-trained multi-class defect recognition model is a classification network trained based on the feature vectors of samples with defect type labels collected from historical laser cladding processes. After standardizing and preprocessing the multidimensional quantized feature vector to be identified, it is input into the input layer of the classification network. The multidimensional quantized feature vectors are sequentially passed through multiple fully connected layers of the classification network for nonlinear transformation and feature abstraction, and finally the probability distributions belonging to each preset defect category are obtained in the output layer. The defect category with the highest probability value is selected as the defect type identifier, and the highest probability value is output as the confidence score.

7. The machine vision-based online quality inspection system for laser cladding according to claim 6, characterized in that, The system also includes: The real-time decision-making module, based on the defect type identifier, confidence score, and defect contour geometry information, generates real-time quality control instructions for the current laser cladding process, specifically including: The real-time quality control commands include laser power adjustment, powder feeding rate correction, and scanning path fine-tuning commands. Establish a knowledge base for mapping the relationship between defect types and process parameter deviations. The knowledge base defines which abnormal process parameters cause different defect types. Based on the defect type identifier, the mapping relationship knowledge base is queried to obtain candidate process parameter adjustment items and suggested adjustment directions; Based on the confidence score, the suggested adjustment amount of the candidate process parameter adjustment item is weighted and corrected. The higher the confidence score, the closer the value is to the suggested adjustment amount. Simultaneously, the geometric information of the defect contour is analyzed. If the defect contour is narrow or has a specific direction, it is determined that the defect is related to the scanning path or the flow direction of the molten pool, and a scanning path fine-tuning instruction is generated. The weighted and corrected process parameter adjustment amount is combined with the scan path fine-tuning instruction to form the real-time quality control instruction containing specific parameter values ​​and execution timing.

8. The machine vision-based online quality inspection system for laser cladding according to claim 3, characterized in that, The thermal gradient difference is calculated based on the temperature value and spatial gradient of the pixel in the isotherm distribution feature map, and the spectral feature difference is calculated based on the pixel values ​​in the spectral emission intensity feature map and the spectral distribution uniformity feature map, including: For a seed point and its neighboring pixels, the corresponding temperature value is obtained from the isotherm distribution feature map. The spatial gradient of the seed point temperature value is calculated, which is approximated by the first-order difference between the temperature values ​​of the pixel and its eight neighboring pixels. The thermal gradient difference is defined as the absolute value of the temperature difference between the neighboring pixel and the seed point, plus the Euclidean distance of the spatial gradient vector difference between the neighboring pixel and the seed point, and then normalized. The feature values ​​corresponding to the seed point and its neighboring pixels are obtained from the spectral emission intensity feature map and the spectral distribution uniformity feature map, respectively. The spectral feature difference is defined as the weighted sum of the differences in feature values ​​between neighboring pixels and seed points in the spectral emission intensity feature map and the differences in feature values ​​in the spectral distribution uniformity feature map. The thermal gradient difference and the spectral feature difference are assigned preset weighting coefficients, and the feature similarity is obtained by summing them.

9. The machine vision-based online quality inspection system for laser cladding according to claim 5, characterized in that, On the spectral feature channels of the defect region sub-map, the average value, standard deviation, and ratio of the highest to lowest intensity of the spectral intensity within the defect region are calculated to constitute the spectral intensity distribution characteristics, including: Using the defect region segmentation mask, feature values ​​of all pixels within the defect region are extracted from the spectral intensity related feature channel of the composite visual feature map to form a set of spectral intensity values. The average value of the spectral intensity is obtained by summing all the values ​​in the set of spectral intensity values ​​and dividing by the total number of pixels. Calculate the square of the difference between each value in the set of spectral intensity values ​​and the average value, sum the squared values, divide by the total number of pixels, and then take the square root to obtain the standard deviation of the spectral intensity. Traverse the set of spectral intensity values, find the maximum and minimum values, and divide the maximum value by the minimum value to obtain the ratio of the highest intensity to the lowest intensity. The calculated average value, standard deviation, and ratio of the spectral intensity are arranged in a predetermined order to form the spectral intensity distribution characteristics.

10. The machine vision-based online quality inspection system for laser cladding according to claim 6, characterized in that, The training steps of the pre-trained multi-class defect recognition model include: Collect multi-band synchronous visual image sequence samples from historical laser cladding processes, and manually annotate the defect areas in each image sample to obtain a training sample set with defect type labels; For each image sample in the training sample set, the same processing flow of the composite feature extraction module, defect region segmentation module, and multidimensional feature quantization module is followed to obtain a training feature set consisting of the multidimensional quantized feature vectors corresponding to all image samples and their defect type labels. Construct a fully connected neural network containing an input layer, multiple hidden layers, and an output layer as the initial classification network model; The training feature set is input into the initial classification network model for forward propagation, the probability distribution of defect categories predicted by the model is calculated, and the cross-entropy loss function is calculated based on the defect type label and the probability distribution of defect categories predicted by the model. The gradient of the cross-entropy loss function with respect to the parameters of each layer in the network is calculated using the backpropagation algorithm, and the network parameters are iteratively updated using an optimizer to minimize the loss function; The classification accuracy of the model is monitored on an independent validation set. Training is stopped when the accuracy no longer improves, and the pre-trained multi-class defect recognition model is obtained. The parameters of the pre-trained multi-class defect recognition model are stored and saved for loading and use during online quality inspection.

Images