A weld defect detection method and system based on visual detection

Abstract

The application discloses a kind of welding seam defect detection method and system based on visual inspection, it is related to industrial nondestructive testing technical field, disclose is the welding seam defect detection method and system based on visual inspection, through fusion visible light and infrared image generation reflection interference coefficient distribution chart, based on reflection interference coefficient dynamic distribution fusion weight and extract multispectral feature and reflection suppression feature, combined with double branch processing and defect recognition algorithm, effectively solve the problem of false detection and missed detection caused by strong reflection interference, with improve the accuracy and reliability of welding seam defect detection, effectively suppress strong reflection interference, reduce false detection rate and missed detection rate.In addition, the present application makes full use of the complementary characteristics of visible light and infrared image, improves the detection performance under complex conditions.

Description

Technical Field

[0001] This application relates to the field of industrial nondestructive testing technology, and in particular to a method and system for detecting weld defects based on visual inspection. Background Technology

[0002] In existing technologies, weld defect detection typically employs manual visual inspection or traditional image processing techniques. Manual visual inspection is not only time-consuming and labor-intensive but also susceptible to human error, leading to low accuracy and reliability of the results. While traditional image processing techniques can improve detection efficiency to some extent, their limited ability to identify complex weld defects often fails to meet the needs of practical applications. Particularly when dealing with weld areas exhibiting strong reflection interference, traditional methods struggle to effectively distinguish between genuine defects and reflection artifacts, resulting in high false positive and false negative rates.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this application is to provide a visual inspection-based method and system for detecting weld defects, aiming to improve the accuracy and reliability of weld defect detection.

[0005] To achieve the above objectives, this application proposes a visual inspection-based weld defect detection method, the method comprising:

[0006] Acquire visible light and infrared images of the weld area;

[0007] A reflection interference coefficient distribution map is generated based on the visible light image;

[0008] The visible light image and the infrared image are assigned fusion weights according to the reflection interference coefficient distribution map, and the visible light image and the infrared image are fused according to the fusion weights to generate a first fused image;

[0009] The first fused image is subjected to bi-branch feature processing to extract the corresponding multispectral feature map and reflectance suppression feature map, respectively;

[0010] The multispectral feature map and the reflection suppression feature map are fused to generate a second fused image, and the second fused image is subjected to defect identification processing to determine the location and type of weld defects.

[0011] In one embodiment, the step of generating a reflection interference coefficient distribution map based on the visible light image and the infrared image includes:

[0012] Divide the visible light image into multiple local regions;

[0013] Calculate the grayscale variance value for each of the local regions;

[0014] The reflection interference coefficient of the corresponding local area is determined based on the gray-scale variance value;

[0015] A reflection interference coefficient distribution map is generated based on the distribution of reflection interference coefficients in multiple local areas.

[0016] In one embodiment, the step of determining the reflection interference coefficient of the corresponding local region based on the grayscale variance value includes determining the reflection interference coefficient of the corresponding local region according to the following formula:

[0017] ;

[0018] in, Indicates the reflection interference coefficient. This represents the grayscale variance value. This represents the first preset threshold. This indicates the second preset threshold.

[0019] In one embodiment, the step of allocating fusion weights for the visible light image and the infrared image according to the reflection interference coefficient distribution map, and fusing the visible light image and the infrared image according to the fusion weights to generate a first fused image includes:

[0020] Read the reflection interference coefficient of each pixel in the reflection interference coefficient distribution map;

[0021] The adjustment factor is determined based on the material composition of the weld area;

[0022] The first fusion weight corresponding to the infrared image and the second fusion weight corresponding to the visible light image are determined based on the reflection interference coefficient and the adjustment factor.

[0023] The visible light image and the infrared image are fused pixel by pixel based on the first fusion weight and the second fusion weight to generate a first fused image.

[0024] In one embodiment, the step of fusing the visible light image and the infrared image pixel-by-pixel based on the first fusion weight and the second fusion weight to generate a first fused image includes fusing pixel-by-pixel according to the following formula to generate the first fused image:

[0025] ;

[0026] in, Indicates the location of the first fused image. Pixel values; Indicates the first fusion weight. ,in, Indicates the reflection interference coefficient. Indicates the regulating factor; Indicates the second fusion weight. ; Indicates the location of the infrared image. Pixel values; Indicates the position of the visible light image. The pixel value.

[0027] In one embodiment, the step of performing bi-branch feature processing on the first fused image to extract the corresponding multispectral feature map and reflectance suppression feature map respectively includes:

[0028] The first branch performs a three-layer convolution operation on the first fused image to extract multispectral feature maps; wherein, the first layer of the first branch has a kernel size of 3×3, the second layer has a kernel size of 5×5, and the third layer has a kernel size of 3×3.

[0029] The first fused image is subjected to reflection suppression processing through the second branch, and the first fused image after reflection suppression processing is subjected to two convolution operations to extract the reflection suppression feature map; wherein, the first convolution kernel size of the second branch is 3×3, and the second convolution kernel size is 3×3.

[0030] In one embodiment, the step of performing reflection suppression processing on the first fused image includes:

[0031] In the first fused image, locate the strong reflection region corresponding to the reflection interference coefficient greater than the third preset threshold in the reflection interference coefficient distribution map;

[0032] The pixel values ​​in the highly reflective area are adjusted based on a preset attenuation coefficient.

[0033] Gaussian smoothing is applied to the edges of the strongly reflective regions after pixel value adjustment to complete the reflection suppression processing of the first fused image.

[0034] In one embodiment, the step of fusing the multispectral feature map and the reflection suppression feature map to generate a second fused image, and performing defect identification processing on the second fused image to determine the location and type of weld defects includes:

[0035] Calculate the similarity matrix between the multispectral feature map and the reflection suppression feature map;

[0036] A feature weight allocation map is generated based on the similarity matrix;

[0037] The multispectral feature map and the reflection suppression feature map are weighted and summed according to the feature weight allocation map to generate a second fused image;

[0038] The second fused image is input into a pre-trained classification network model for defect identification processing to output the location and type of weld defects.

[0039] In one embodiment, prior to the step of generating a reflection interference coefficient distribution map based on the visible light image, the method further includes:

[0040] Detecting oil stain areas in visible light images;

[0041] Brightness compensation is applied to the oil-stained area;

[0042] Replace the original visible light image with the brightness-compensated visible light image.

[0043] Furthermore, to achieve the above objectives, this application also proposes a weld defect detection system based on vision inspection. The system includes: a memory, a processor, and a weld defect detection program based on vision inspection stored in the memory and executable on the processor. The weld defect detection program based on vision inspection is configured to implement the steps of the weld defect detection method based on vision inspection.

[0044] This application proposes a vision-based weld defect detection method and system. It generates a reflection interference coefficient distribution map by fusing visible light and infrared images, dynamically allocates fusion weights based on the reflection interference coefficient, and extracts multispectral features and reflection suppression features. Combined with dual-branch processing and a defect recognition algorithm, it effectively solves the problems of false detection and missed detection caused by strong reflection interference. This improves the accuracy and reliability of weld defect detection, effectively suppresses strong reflection interference, and reduces the false detection and missed detection rates. Furthermore, this application fully utilizes the complementary advantages of visible light and infrared images to enhance detection performance under complex working conditions. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating an embodiment of the visual inspection-based weld defect detection method of this application.

[0048] Figure 2 This is a structural schematic diagram of an embodiment of the weld defect detection system based on visual inspection according to this application.

[0049] Explanation of icon numbers:

[0050] 10. Memory; 20. Processor.

[0051] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0053] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0054] The main solution of this application embodiment is as follows: acquiring visible light and infrared images of the weld area; generating a reflection interference coefficient distribution map based on the visible light image; assigning fusion weights to the visible light and infrared images according to the reflection interference coefficient distribution map, and fusing the visible light and infrared images according to the fusion weights to generate a first fused image; performing bi-branch feature processing on the first fused image to extract corresponding multispectral feature maps and reflection suppression feature maps respectively; fusing the multispectral feature maps and reflection suppression feature maps to generate a second fused image, and performing defect identification processing on the second fused image to determine the location and type of weld defects.

[0055] In this embodiment, for ease of description, the following description will focus on a weld defect detection system based on vision inspection.

[0056] Currently, weld defect detection typically relies on manual visual inspection or traditional image processing techniques. Manual visual inspection is not only time-consuming and labor-intensive but also susceptible to human error, leading to low accuracy and reliability of the results. While traditional image processing techniques can improve detection efficiency to some extent, their limited ability to identify complex weld defects often fails to meet the needs of practical applications. Particularly when dealing with weld areas exhibiting strong reflection interference, traditional methods struggle to effectively distinguish between genuine defects and reflection artifacts, resulting in high false positive and false negative rates.

[0057] The solution provided in this application generates a reflection interference coefficient distribution map by fusing visible light and infrared images. Based on the reflection interference coefficient, it dynamically allocates fusion weights and extracts multispectral features and reflection suppression features. Combined with dual-branch processing and defect recognition algorithms, it effectively solves the problems of false detection and missed detection caused by strong reflection interference. This improves the accuracy and reliability of weld defect detection, effectively suppresses strong reflection interference, and reduces the false detection and missed detection rates. Furthermore, this application fully utilizes the complementary advantages of visible light and infrared images to enhance detection performance under complex working conditions.

[0058] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a weld defect detection system based on vision inspection. The following description uses a weld defect detection system based on vision inspection as an example to illustrate this embodiment and the subsequent embodiments.

[0059] Based on this, embodiments of this application provide a visual inspection-based weld defect detection method, referring to... Figure 1 The visual inspection-based weld defect detection method includes steps S100 to S500, wherein:

[0060] Step S100: Acquire visible light and infrared images of the weld area;

[0061] Step S200: Generate a reflection interference coefficient distribution map based on the visible light image;

[0062] Step S300: Assign fusion weights to the visible light image and the infrared image according to the reflection interference coefficient distribution map, and fuse the visible light image and the infrared image according to the fusion weights to generate a first fused image;

[0063] Step S400: Perform dual-branch feature processing on the first fused image to extract the corresponding multispectral feature map and reflectance suppression feature map respectively;

[0064] Step S500: The multispectral feature map and the reflection suppression feature map are fused to generate a second fused image, and the second fused image is subjected to defect identification processing to determine the location and type of weld defects.

[0065] In this embodiment, the reflection interference coefficient distribution map refers to a quantitative distribution map reflecting the degree of reflection interference in local areas of a visible light image. Specifically, it can be achieved by dividing the visible light image into multiple local regions. The specific number of local regions can be set according to actual needs, such as dividing it into grids of 7×7, 9×9, etc. Then, the gray-level variance value of each local region is calculated and mapped to the reflection interference coefficient according to a preset threshold. This distribution map is used to dynamically adjust the fusion weights and reduce the dependence of strongly reflective areas on the visible light image. The fusion weight allocation can be dynamically determined based on the reflection interference coefficient to determine the contribution ratio of visible light and infrared images. Specifically, a linear interpolation method can be used, combined with material-related adjustment factors, to achieve pixel-level weight allocation. Dual-branch feature processing refers to extracting multispectral features and reflection suppression features separately through independent branch networks. Specifically, multi-scale feature extraction can be achieved by combining convolutional layers with different kernel sizes, while region attenuation and edge smoothing operations are used to suppress reflection interference.

[0066] In this embodiment, visible light images and infrared images are used to capture the optical properties and thermal radiation information of the weld surface, respectively. By analyzing the local grayscale changes in the visible light images, high-reflection interference areas are identified, and a reflection interference coefficient distribution map is generated. In the image fusion stage, for high-reflection areas, the weight of the visible light image is reduced, and infrared image information is relied upon instead to reduce misjudgments caused by reflection. In the dual-branch processing, the first branch extracts multispectral features using multi-scale convolution operations to retain detailed information in different bands; the second branch locates strong reflection areas and adjusts pixel values, while combining Gaussian smoothing to eliminate edge artifacts, thereby generating reflection suppression features. In the secondary feature fusion stage, feature weights are dynamically allocated through a similarity matrix to enhance the saliency of defect-related features, and finally, a classification network is used to achieve accurate identification.

[0067] In this embodiment, a dynamic adjustment fusion strategy based on the reflection interference coefficient, combined with the complementarity of bi-branch features, effectively distinguishes between real defects and reflection artifacts, reducing the false detection rate. This allows the system to adapt to the weld inspection needs in complex industrial environments, accurately identifying defect types such as cracks and porosity under strong reflection conditions, while precisely locating defect positions. Furthermore, the multi-modal data fusion and dynamic feature enhancement mechanism improve the robustness and generalization ability of the detection system, providing reliable technical support for automated inspection.

[0068] In one feasible implementation, the step of generating a reflection interference coefficient distribution map based on the visible light image and the infrared image includes: dividing the visible light image into multiple local regions; calculating the gray-level variance value of each local region; determining the reflection interference coefficient of the corresponding local region based on the gray-level variance value; and generating a reflection interference coefficient distribution map according to the distribution of the reflection interference coefficients of the multiple local regions.

[0069] In this embodiment, a local region refers to dividing the visible light image into several rectangular blocks of equal size. This can be achieved using sliding window segmentation or grid partitioning. By dividing the image into local regions, the differences in reflectance characteristics between different areas can be analyzed specifically. The gray-level variance value is a numerical value obtained by calculating the variance of the gray-level values ​​of all pixels within the local region. This can be implemented using the discrete variance formula. This value reflects the degree of dispersion of the gray-level distribution within the region and is used to measure the intensity of reflection interference. The reflection interference coefficient is a normalized coefficient obtained by mapping the gray-level variance value. This can be implemented using a piecewise linear function. This coefficient is used to quantify the degree of reflection interference in the local region, providing a weighting basis for subsequent image fusion.

[0070] In this embodiment, in visible light images, high-reflectivity areas generally exhibit uneven grayscale value distribution or drastic fluctuations, while low-reflectivity areas have a relatively uniform grayscale distribution. By dividing the image into several local regions and calculating the grayscale variance value of each region, areas with strong reflection interference can be effectively identified. For example, if the grayscale variance value of a certain local region is high, it indicates that there may be metallic reflection or high-brightness noise in that region, and the reflection interference coefficient will be set to a larger value; conversely, regions with lower grayscale variance values ​​have smaller reflection interference coefficients. The final generated reflection interference coefficient distribution map is presented in the form of a two-dimensional matrix, where each element corresponds to the reflection interference intensity of a certain local region in the image, providing data support for subsequent fusion weight allocation.

[0071] In this embodiment, by dividing the local area and calculating the gray-scale variance, the intensity of reflection interference in different areas can be dynamically identified, thereby more accurately locating the interference source. For example, in the scenario of weld inspection, metal surface oxidation may only exist in local areas. Traditional methods would reduce the accuracy of defect identification due to global processing, while this method can avoid such interference through local analysis.

[0072] In one feasible implementation, the step of determining the reflection interference coefficient of the corresponding local region based on the grayscale variance value includes determining the reflection interference coefficient of the corresponding local region according to the following formula:

[0073] ;

[0074] in, Indicates the reflection interference coefficient. This represents the grayscale variance value. This represents the first preset threshold. This indicates the second preset threshold.

[0075] In this embodiment, the grayscale variance value is used to characterize the dispersion of pixel grayscale values ​​within a local area. Specifically, it can be achieved by calculating the mean of the squared differences between the grayscale values ​​of all pixels within the area and the average value, reflecting the change in reflection intensity in that area. The reflection interference coefficient is an indicator that quantifies the degree of reflection interference in a local area. It is specifically obtained by mapping the grayscale variance value through a piecewise function and is used for subsequent image fusion weight allocation. The first preset threshold and the second preset threshold are used to define the range of grayscale variance values, thereby determining the piecewise mapping strategy for the reflection interference coefficient. Specifically, they can be set according to the reflection characteristics of welds of different materials or historical detection data. For example, the first preset threshold can be the lower quartile of the grayscale variance value range, and the second preset threshold can be the upper quartile; their specific values ​​are not limited here. Specifically, when the grayscale variance value is below a first preset threshold, it indicates that the reflection interference in the local area is weak, and the reflection interference coefficient is set to a small value. When the grayscale variance value is between the first and second preset thresholds, the reflection interference coefficient increases with the increase of the grayscale variance value. When the grayscale variance value is above the second preset threshold, it indicates that the reflection interference in the local area is strong, and the reflection interference coefficient is set to a large value. By introducing a piecewise linear function, this embodiment can characterize the degree of reflection interference corresponding to different grayscale variance values, providing a more accurate basis for weight allocation in subsequent image fusion. Compared with traditional global processing or fixed weight allocation strategies, this fine-grained reflection interference coefficient calculation method can more effectively cope with the complex and ever-changing weld inspection environment, improving the robustness and accuracy of the inspection system.

[0076] In this embodiment, after calculating the grayscale variance value of a local area, this value is first compared with a pre-set first and second preset thresholds. If the grayscale variance value is less than the first preset threshold, the reflection interference coefficient is set to 0, meaning that the area does not require infrared image compensation. If the grayscale variance value is between the first and second preset thresholds, the reflection interference coefficient is calculated using linear interpolation to achieve a smooth transition of weights. If the grayscale variance value exceeds the second preset threshold, the reflection interference coefficient is directly set to the maximum value of 1, indicating that the area must rely entirely on infrared image data. This segmented processing method can dynamically adapt to areas with different reflection intensities, avoiding the problem of sudden weight changes caused by using a single threshold.

[0077] In this embodiment, by combining dual-threshold interval division with linear interpolation, visible light details in low-interference regions are preserved while reflection noise is effectively suppressed in high-interference regions, significantly enhancing the robustness of weight allocation. This application can accurately quantify the degree of reflection interference in different regions of weld images, providing a reliable weighting basis for multispectral image fusion, thereby reducing artifacts and detail loss in the fused image and ultimately improving the accuracy of weld defect identification.

[0078] In one feasible implementation, the steps of assigning fusion weights to the visible light image and the infrared image according to the reflection interference coefficient distribution map, and fusing the visible light image and the infrared image according to the fusion weights to generate a first fused image include: reading the reflection interference coefficient of each pixel in the reflection interference coefficient distribution map; determining an adjustment factor based on the material of the weld area; determining a first fusion weight corresponding to the infrared image and a second fusion weight corresponding to the visible light image based on the reflection interference coefficient and the adjustment factor; and fusing the visible light image and the infrared image pixel by pixel based on the first fusion weight and the second fusion weight to generate the first fused image.

[0079] In this embodiment, the reflection interference coefficient is a parameter used to quantify the degree of reflection interference in an image. Specifically, it can be calculated using the grayscale variance value and then mapped to the interval of 0 to 1 using a piecewise function. The larger the reflection interference coefficient, the more severe the reflection interference in that area. The adjustment factor is a parameter that dynamically adjusts the fusion weights according to the characteristics of the weld material. Specifically, it can be obtained by querying a material's optical property database or through experimental calibration. The differences in the reflection characteristics of different materials for visible and infrared light need to be compensated for by the adjustment factor. The first fusion weight and the second fusion weight refer to the contribution ratio of each pixel in the fusion process of the visible light and infrared images. Specifically, it can be calculated by a linear combination of the reflection interference coefficient and the adjustment factor. The weight allocation needs to satisfy that the visible light image dominates in the low reflection interference area and the infrared image dominates in the high reflection interference area.

[0080] In this embodiment, when generating the first fused image, the reflection interference coefficient value of each pixel is first extracted from the reflection interference coefficient distribution map. The reflection interference coefficient value reflects the image noise intensity caused by surface reflection at that location. Based on the actual material type of the weld, such as stainless steel, carbon steel, or aluminum alloy, the corresponding adjustment factor is obtained from a pre-set material parameter table. Some specific material parameters are shown in Table 1.

[0081] Table 1 Material Parameter Table

[0082] Material type Regulatory factors Stainless steel 0.8 carbon steel 1.2 aluminum alloy 1.0 copper alloy 0.7 Titanium alloy 0.9 High temperature alloy 1.1

[0083] The adjustment factors provided above are used to correct for differences in the visible and infrared light reflectance characteristics of different materials. Subsequently, the reflection interference coefficient and the adjustment factors are substituted into the weighting calculation formula to obtain the first fusion weight for the infrared image and the second fusion weight for the visible light image. For example, when the material is high-reflectivity stainless steel, the adjustment factor can be set to 0.8 to reduce the visible light weight; when the material is low-reflectivity carbon steel, the adjustment factor can be set to 1.2 to increase the visible light weight. Finally, the visible light image and the infrared image are fused using a pixel-by-pixel weighted summation method to generate a first fused image that suppresses reflection interference.

[0084] In this embodiment, by combining dynamic weight allocation with material characteristic compensation, noise interference in high-reflectivity areas can be effectively suppressed, while retaining the detailed information of visible light images in low-reflectivity areas. This solves the problem of image quality degradation caused by reflection interference and material differences in weld inspection by traditional fusion methods, improves the clarity and defect contrast of the fused image in strong-reflectivity areas, and provides more reliable input data for subsequent defect identification.

[0085] In one feasible implementation, the step of fusing the visible light image and the infrared image pixel-by-pixel based on the first fusion weight and the second fusion weight to generate a first fused image includes fusing pixel-by-pixel according to the following formula to generate the first fused image:

[0086] ;

[0087] in, Indicates the location of the first fused image. Pixel values; Indicates the first fusion weight. ,in, Indicates the reflection interference coefficient. Indicates the regulating factor; Indicates the second fusion weight. ; Indicates the position of the infrared image Pixel values; Indicates the position of the visible light image. The pixel value.

[0088] In this embodiment, the reflection interference coefficient is a parameter used to quantify the degree of light interference caused by surface reflection in a local area. Specifically, it can be obtained by calculating the gray-level variance of a local area in the visible light image and mapping it piecewise based on a preset threshold. Its function is to dynamically reflect the intensity of reflection interference in different areas, providing a basis for the allocation of fusion weights. The adjustment factor is a coefficient that adjusts the weight allocation ratio according to the optical characteristics of the weld material. Specifically, it can be determined by experimentally calibrating the sensitivity of different materials to infrared and visible light reflection. Its function is to adapt to the differences in reflection characteristics of different materials and optimize the adaptability of the fusion weight allocation. The fusion weight is a parameter representing the proportion of contribution of the visible light image and the infrared image during the fusion process. Specifically, it can be calculated by a linear combination of the reflection interference coefficient and the adjustment factor. Its function is to dynamically balance the information complementarity of the two images, suppressing reflection interference while retaining effective features.

[0089] In this embodiment, when generating the first fused image, for each pixel location, the reflection interference coefficient corresponding to the current pixel is first obtained from the reflection interference coefficient distribution map. Then, combined with an adjustment factor pre-set according to the material, the fusion weight of the infrared image and the visible light image is calculated using a formula. For example, when the reflection interference coefficient is high, the fusion weight is calculated according to the formula. The larger the calculated first fusion weight, the greater the weight corresponding to the infrared image, thus reducing the impact of reflection interference in the visible light image; while when the reflection interference coefficient is low, according to the formula... The smaller the calculated first fusion weight, the better. Accordingly, the larger the second fusion weight, the greater the weight of the visible light image, thus preserving its high-resolution detail information. Subsequently, the infrared image and the visible light image are fused using a pixel-by-pixel weighted summation method to finally generate the first fused image.

[0090] In this embodiment, by dynamically allocating fusion weights and combining the reflection interference coefficient and the material adjustment factor, the contribution ratio of the two images can be adaptively adjusted. While suppressing reflection interference, the complementary advantages of multispectral information are preserved, thereby improving the quality of the fused image. That is, this application can dynamically optimize the fusion weights according to the reflection interference intensity and material characteristics of different regions, effectively suppressing the interference noise caused by surface reflection in the visible light image, while making full use of the sensitivity of infrared images to internal defects of materials, so that the generated first fused image has both high-resolution details and deep feature information, providing a more reliable data foundation for subsequent defect identification.

[0091] In one feasible implementation, the step of performing bi-branch feature processing on the first fused image to extract the corresponding multispectral feature map and reflectance suppression feature map respectively includes:

[0092] The first branch performs a three-layer convolution operation on the first fused image to extract multispectral feature maps; wherein the first layer convolution kernel size of the first branch is 3×3, the second layer convolution kernel size is 5×5, and the third layer convolution kernel size is 3×3. The second branch performs reflection suppression processing on the first fused image, and then performs a two-layer convolution operation on the first fused image after reflection suppression processing to extract reflection suppression feature maps; wherein the first layer convolution kernel size of the second branch is 3×3, and the second layer convolution kernel size is 3×3.

[0093] In this embodiment, the multispectral feature map refers to a feature map containing visible and infrared spectral information extracted layer by layer through different convolutional kernel sizes. Specifically, it can be implemented using three convolutional operations: the first 3×3 convolutional kernel is used to capture local details, the second 5×5 convolutional kernel is used to expand the receptive field to extract global features, and the third 3×3 convolutional kernel is used to optimize feature representation, thereby achieving the fusion of multi-scale features. The reflection suppression feature map refers to a robust feature map extracted after suppressing strong reflection interference. Specifically, it can be implemented by combining reflection suppression processing with two convolutional operations. The reflection suppression processing locates and adjusts the pixel values ​​of strong reflection regions, combines Gaussian smoothing to eliminate edge abrupt changes, and then extracts stable features after reflection suppression through two 3×3 convolutional kernels.

[0094] In this embodiment, the first branch processes the first fused image layer by layer using convolutional kernels of different sizes. First, a 3×3 convolutional kernel is used to extract the texture details of the weld surface. Then, a 5×5 convolutional kernel is used to capture a larger range of regional features. Finally, a 3×3 convolutional kernel is used to integrate multi-scale information, thereby generating a multispectral feature map. The second branch first performs reflection suppression processing on the first fused image, locating regions in the reflection interference coefficient distribution map that exceed a preset threshold, reducing the brightness of pixels in these regions, and performing Gaussian smoothing to eliminate edge artifacts. Subsequently, two layers of 3×3 convolutional kernels are used to extract the reflection-suppressed features, generating a reflection suppression feature map that suppresses interference. The two branches optimize spectral feature enhancement and interference suppression respectively, providing complementary feature information for subsequent fusion.

[0095] In this embodiment, a dual-branch parallel processing structure is used. The first branch utilizes combinations of different convolutional kernel sizes to enhance the representational ability of multispectral features, while the second branch enhances feature stability through reflection suppression preprocessing combined with fixed-size convolutional kernels. The first and second branches work together to solve the problem of insufficient feature extraction under complex lighting conditions. Thus, this application can effectively separate multispectral features from reflection interference features, suppressing noise interference in strong reflection regions while retaining complementary information from visible light and infrared images, thereby improving the accuracy of weld defect identification, especially in scenarios with metallic reflection, reducing false positive and false negative rates.

[0096] In one feasible implementation, the step of performing reflection suppression processing on the first fused image includes: locating a strong reflection region in the first fused image corresponding to a reflection interference coefficient greater than a third preset threshold in the reflection interference coefficient distribution map; adjusting the pixel values ​​within the strong reflection region based on a preset attenuation coefficient; and performing Gaussian smoothing processing on the edges of the strong reflection region after pixel value adjustment to complete the reflection suppression processing of the first fused image.

[0097] In this embodiment, a strong reflection region refers to an image region where the reflection interference coefficient exceeds a preset threshold. This can be achieved by combining image segmentation techniques with threshold filtering based on the reflection interference coefficient distribution map, used to identify areas where surface reflection causes loss of image detail or noise interference. The attenuation coefficient is an adjustment parameter used to reduce pixel values ​​in strong reflection regions. It can be implemented using a fixed ratio or dynamic calculation, for example, set to a value between 0.5 and 0.8 based on the material's reflectivity, used to suppress brightness interference in overexposed areas. Gaussian smoothing refers to blurring the edges of the adjusted region, specifically using Gaussian kernel convolution operations, such as using a 3×3 or 5×5 convolution kernel, to eliminate edge artifacts caused by abrupt changes in pixel values.

[0098] In this embodiment, after generating the first fused image, a reflection interference coefficient distribution map is used to identify the location of areas with strong reflections. By setting a third preset threshold, such as areas with a reflection interference coefficient greater than 0.7, local areas severely affected by reflection interference can be filtered out. Subsequently, the pixel values ​​of the selected areas are linearly or non-linearly adjusted using an attenuation coefficient, for example, multiplying the pixel values ​​by 0.6 to reduce brightness. The edges of the adjusted areas may exhibit jagged edges or artifacts due to abrupt changes in pixel values, so Gaussian smoothing is further applied, for example, using a Gaussian kernel with a standard deviation of 1.0 for convolution operations to ensure natural edge transitions. Thus, the image after reflection suppression processing retains key details while reducing the interference of reflections on defect detection.

[0099] In this embodiment, by locating and specifically adjusting highly reflective areas, pixel value attenuation is performed only in the affected areas, avoiding information loss caused by global processing. Furthermore, edge smoothing further resolves the artifact problem that may be introduced by local adjustments, avoiding local overexposure in weld images caused by surface reflection, while preserving detail information in non-reflective areas, thereby improving the accuracy of defect identification. Through the synergistic effect of local adjustments and edge smoothing, the image quality degradation caused by global processing in traditional methods is avoided, enabling subsequent defect detection algorithms to more reliably identify minute defects such as cracks and porosity.

[0100] In one feasible implementation, the steps of fusing the multispectral feature map and the reflection suppression feature map to generate a second fused image, and performing defect identification processing on the second fused image to determine the location and type of weld defects include:

[0101] Calculate the similarity matrix between the multispectral feature map and the reflection suppression feature map; generate a feature weight allocation map based on the similarity matrix; perform a weighted summation of the multispectral feature map and the reflection suppression feature map according to the feature weight allocation map to generate a second fused image; input the second fused image into a pre-trained classification network model for defect identification processing to output the location and type of weld defects.

[0102] In this embodiment, the similarity matrix refers to a two-dimensional data matrix formed by calculating the correlation metric between two feature maps pixel by pixel. Specifically, it can be implemented using cosine similarity or Euclidean distance algorithms, used to quantify the degree of correlation between multispectral features and reflectance suppression features at different spatial locations. The feature weight allocation map refers to a weight distribution map that dynamically adjusts the feature fusion ratio based on the similarity matrix. Specifically, it can be generated by normalizing similarity values ​​and mapping them to the 0-1 interval, used to assign higher weights to highly correlated regions during the fusion process to strengthen effective features. Weighted summation refers to the linear superposition of corresponding pixels in the two feature maps according to the weight allocation map, specifically implemented using pixel-by-pixel multiplication and addition operations, used to fuse complementary features and suppress redundant information. The classification network model refers to a deep learning model trained for image classification and object detection, specifically implemented using convolutional neural networks or Transformer architectures, used to identify defect morphologies from the fused feature maps and output their location and category labels.

[0103] In this embodiment, during the feature fusion stage, a similarity matrix reflecting the degree of correlation between the multispectral feature map and the reflection suppression feature map is generated by calculating the similarity values ​​at each pixel position. Then, the similarity matrix is ​​normalized and transformed into a feature weight allocation map. In this map, the weight value of each pixel represents the contribution ratio of the multispectral feature at the corresponding position in the fusion process. Based on the weight allocation map, a pixel-by-pixel weighted summation operation is performed on the multispectral feature map and the reflection suppression feature map, thereby enhancing the expression of multispectral features in areas with high feature correlation, while emphasizing the information of reflection suppression features in areas with low correlation. Finally, the generated second fused image is input into a pre-trained classification network model. After processing through multiple convolutional and fully connected layers of the model, the precise location coordinates of the weld defect and the type classification result are output.

[0104] In this embodiment, a dynamic weight allocation mechanism adaptively adjusts the fusion ratio based on the correlation between features, effectively preserving complementary features and suppressing interference, thereby improving the accuracy of defect identification. Furthermore, using a pre-trained classification network model instead of the traditional threshold segmentation algorithm better adapts to complex and varied defect morphologies, reducing the false positive rate. This solves the problem of insufficient identification accuracy caused by the single feature fusion method in traditional weld defect detection, achieving efficient fusion and accurate classification of multi-source features, significantly improving the detection rate of minute defects such as porosity and cracks, while reducing false positives and false negatives caused by reflection interference or noise.

[0105] In one feasible implementation, prior to the step of generating a reflection interference coefficient distribution map based on the visible light image, the method further includes: detecting oil stain areas in the visible light image; performing brightness compensation on the oil stain areas; and replacing the original visible light image with the brightness-compensated visible light image.

[0106] In this embodiment, oil stain area detection refers to identifying low-brightness or high-reflectivity areas in a visible light image caused by oil stains through image processing algorithms. Specifically, edge detection combined with region growing algorithms can be used to locate oil stain areas by analyzing pixel brightness gradient changes and region connectivity. Brightness compensation refers to enhancing the pixel values ​​within the oil stain area, which can be achieved using histogram equalization or adaptive gamma correction algorithms. This adjusts the pixel brightness distribution to restore the detailed information of the oil stain area. Replacing the original visible light image involves covering the corresponding positions of the original image with the brightness-compensated oil stain area. This can be achieved using pixel-level replacement or region fusion algorithms, which eliminates oil stain interference to improve the accuracy of subsequent reflection interference coefficient calculations.

[0107] In this embodiment, after the visible light image is acquired, an edge detection algorithm is first used to identify the contour boundaries of the oil-stained area, and then a region growing algorithm is used to determine the oil-stained coverage area. Subsequently, histogram equalization is performed on the pixels within the oil-stained area to enhance their brightness distribution and restore the texture details obscured by the oil. Finally, the processed oil-stained area is seamlessly fused with the non-oil-stained area of ​​the original image to generate a brightness-compensated visible light image. This image is then used to replace the original image in the subsequent reflection interference coefficient distribution map generation process.

[0108] In some specific implementations, oil stain area detection can employ a threshold-based segmentation method, such as marking pixels with brightness below a preset value as candidate oil stain areas, and then eliminating noise interference through morphological closing operations. Brightness compensation can employ a local contrast enhancement algorithm, such as applying dynamic range stretching to each pixel within the oil stain area to make its brightness distribution consistent with the surrounding normal area. The replacement operation can employ a Poisson fusion algorithm, achieving a natural transition between the compensated area and the original image through gradient domain editing.

[0109] Understandably, existing technologies typically generate reflection interference coefficient distribution maps directly from the original image. However, the low brightness or high reflectivity of oil-stained areas can lead to deviations in the calculation of reflection interference coefficients, thus affecting image fusion weight allocation and defect recognition accuracy. This solution, through oil stain detection and compensation, effectively eliminates the influence of oil stains on the reflection interference coefficient distribution, solving the problem of local reflection characteristic distortion in visible light images caused by oil stain coverage. It avoids misjudgments of reflection coefficients due to oil stain interference, thereby improving the accuracy and robustness of weld defect detection.

[0110] In this embodiment, the weld defect detection method based on visual inspection generates a reflection interference coefficient distribution map by fusing visible light and infrared images. Based on the reflection interference coefficient, it dynamically allocates fusion weights and extracts multispectral features and reflection suppression features. Combined with dual-branch processing and defect recognition algorithms, it effectively solves the problems of false detection and missed detection caused by strong reflection interference. It improves the accuracy and reliability of weld defect detection, effectively suppresses strong reflection interference, and reduces the false detection rate and missed detection rate. In addition, it makes full use of the complementary advantages of visible light and infrared images to improve the detection performance under complex working conditions.

[0111] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the visual inspection-based weld defect detection method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0112] This application also provides a weld defect detection system based on vision inspection, referenced Figure 2 The system includes: a memory 10, a processor 20, and a vision-based weld defect detection program stored in the memory 10 and executable on the processor 20, wherein the vision-based weld defect detection program is configured to implement the steps of the vision-based weld defect detection method.

[0113] The weld defect detection system provided in this application, employing the weld defect detection method described in the above embodiments, can improve the accuracy and reliability of weld defect detection. Compared with the prior art, the beneficial effects of the weld defect detection system provided in this application are the same as those of the weld defect detection method provided in the above embodiments, and other technical features of the weld defect detection system are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0114] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for detecting weld defects based on visual inspection, characterized in that, The method includes: Acquire visible light and infrared images of the weld area; A reflection interference coefficient distribution map is generated based on the visible light image; The visible light image and the infrared image are assigned fusion weights according to the reflection interference coefficient distribution map, and the visible light image and the infrared image are fused according to the fusion weights to generate a first fused image; The first fused image is subjected to bi-branch feature processing to extract the corresponding multispectral feature map and reflectance suppression feature map, respectively; The multispectral feature map and the reflection suppression feature map are fused to generate a second fused image, and the second fused image is subjected to defect identification processing to determine the location and type of weld defects.

2. The weld defect detection method based on vision inspection as described in claim 1, characterized in that, The step of generating a reflection interference coefficient distribution map based on the visible light image and the infrared image includes: Divide the visible light image into multiple local regions; Calculate the grayscale variance value for each of the local regions; The reflection interference coefficient of the corresponding local area is determined based on the gray-scale variance value; A reflection interference coefficient distribution map is generated based on the distribution of reflection interference coefficients in multiple local areas.

3. The weld defect detection method based on vision inspection as described in claim 2, characterized in that, The step of determining the reflection interference coefficient of the corresponding local region based on the grayscale variance value includes determining the reflection interference coefficient of the corresponding local region according to the following formula: ； in, Indicates the reflection interference coefficient. This represents the grayscale variance value. This represents the first preset threshold. This indicates the second preset threshold.

4. The weld defect detection method based on vision inspection as described in claim 1, characterized in that, The step of assigning fusion weights to the visible light image and the infrared image according to the reflection interference coefficient distribution map, and fusing the visible light image and the infrared image according to the fusion weights to generate a first fused image includes: Read the reflection interference coefficient of each pixel in the reflection interference coefficient distribution map; The adjustment factor is determined based on the material composition of the weld area; The first fusion weight corresponding to the infrared image and the second fusion weight corresponding to the visible light image are determined based on the reflection interference coefficient and the adjustment factor. The visible light image and the infrared image are fused pixel by pixel based on the first fusion weight and the second fusion weight to generate a first fused image.

5. The weld defect detection method based on vision inspection as described in claim 4, characterized in that, The step of fusing the visible light image and the infrared image pixel by pixel based on the first fusion weight and the second fusion weight to generate a first fused image includes fusing pixel by pixel according to the following formula to generate the first fused image: ； in, Indicates the location of the first fused image. Pixel values; Indicates the first fusion weight. ,in, Indicates the reflection interference coefficient. Indicates the regulating factor; Indicates the second fusion weight. ; Indicates the position of the infrared image Pixel values; Indicates the position of the visible light image. The pixel value.

6. The weld defect detection method based on vision inspection as described in claim 1, characterized in that, The step of performing bi-branch feature processing on the first fused image to extract the corresponding multispectral feature map and reflectance suppression feature map respectively includes: The first branch performs a three-layer convolution operation on the first fused image to extract multispectral feature maps; wherein, the first layer of the first branch has a kernel size of 3×3, the second layer has a kernel size of 5×5, and the third layer has a kernel size of 3×3. The first fused image is subjected to reflection suppression processing through the second branch, and the first fused image after reflection suppression processing is subjected to two convolution operations to extract the reflection suppression feature map; wherein, the first convolution kernel size of the second branch is 3×3, and the second convolution kernel size is 3×3.

7. The weld defect detection method based on vision inspection as described in claim 6, characterized in that, The step of performing reflection suppression processing on the first fused image includes: In the first fused image, locate the strong reflection region corresponding to the reflection interference coefficient greater than the third preset threshold in the reflection interference coefficient distribution map; The pixel values ​​in the highly reflective area are adjusted based on a preset attenuation coefficient. Gaussian smoothing is applied to the edges of the strongly reflective regions after pixel value adjustment to complete the reflection suppression processing of the first fused image.

8. The weld defect detection method based on vision inspection as described in claim 1, characterized in that, The steps of fusing the multispectral feature map and the reflection suppression feature map to generate a second fused image, and performing defect identification processing on the second fused image to determine the location and type of weld defects include: Calculate the similarity matrix between the multispectral feature map and the reflection suppression feature map; A feature weight allocation map is generated based on the similarity matrix; The multispectral feature map and the reflection suppression feature map are weighted and summed according to the feature weight allocation map to generate a second fused image; The second fused image is input into a pre-trained classification network model for defect identification processing to output the location and type of weld defects.

9. The weld defect detection method based on vision inspection as described in any one of claims 1 to 8, characterized in that, Prior to the step of generating a reflection interference coefficient distribution map based on the visible light image, the method further includes: Detecting oil stain areas in visible light images; Brightness compensation is applied to the oil-stained area; Replace the original visible light image with the brightness-compensated visible light image.

10. A weld defect detection system based on vision inspection, characterized in that, The system includes: a memory, a processor, and a vision-based weld defect detection program stored in the memory and executable on the processor, the vision-based weld defect detection program being configured to implement the steps of the vision-based weld defect detection method as described in any one of claims 1 to 9.

Images