Weld defect detection system and method based on infrared thermal imaging and deep learning

Abstract

The present invention belongs to the technical field of welding detection. The present invention discloses a weld defect detection system and method based on infrared thermal imaging and deep learning, including: a data acquisition and processing module: acquiring weld thermal imaging data, performing enhanced preprocessing on the weld thermal imaging data, and obtaining an enhanced heat map; an adaptive segmentation module: performing adaptive segmentation on the enhanced heat map to obtain a weld area map; a defect feature extraction module: performing sparse feature extraction on the weld area map to obtain weld features; a model construction module: constructing a weld defect detection model based on weld defect features and defect labels, and performing defect detection on the weld based on the constructed weld defect detection model to obtain accurate detection results; ensuring the efficiency and real-time performance of the detection process and improving the accuracy of weld defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of welding detection, and more specifically, to a weld defect detection system and method based on infrared thermal imaging and deep learning. Background Art

[0002] In modern industry, welding is an indispensable process step in many manufacturing processes and is widely used in aerospace, automobile manufacturing, shipbuilding, petrochemical and other fields. The quality of welding directly affects the strength, safety and service life of structural parts. Therefore, the inspection and evaluation of weld quality is crucial. During the welding process, various defects such as cracks, pores, inclusions, incomplete penetration, overburning, etc. may occur in the weld due to the influence of factors such as materials, processes, equipment, and operations. If these defects are not discovered and repaired in time, they may lead to failure, damage and even major safety accidents of the structural parts.

[0003] Existing infrared thermal imaging systems mostly rely on static thermal images after welding for analysis, but these images often fail to take into account the instantaneous and dynamic nature of temperature changes during the welding process. This means that some short-term, small thermal changes during the welding process can easily be masked by the thermal diffusion effect, resulting in defects not being captured in time; existing defect detection systems are often unable to simultaneously handle the complex interactive effects between multiple defects. For example, the presence of pores may cause local overheating, while cracks may cause uneven temperature distribution. These factors are superimposed on each other to generate "pseudo-defect" signals, leading to misjudgment of defects; when the current fluctuates or the welding speed changes suddenly during the welding process, the infrared thermal imaging image may show a thermal pattern similar to the defect, such as a sudden increase or decrease in surface temperature. This change may lead to "false defects" and "false defects" will be treated as real defects.

[0004] In view of this, the present invention proposes a weld defect detection system and method based on infrared thermal imaging and deep learning to solve the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned objectives, the present invention provides the following technical solution: a weld defect detection system based on infrared thermal imaging and deep learning, comprising:

[0006] Data acquisition and processing module: collects weld thermal imaging data, performs enhancement preprocessing on the weld thermal imaging data, and obtains enhanced thermal maps;

[0007] Adaptive segmentation module: adaptively segment the enhanced heat map to obtain the weld area map;

[0008] Defect feature extraction module: performs sparse feature extraction on the weld area map to obtain weld features;

[0009] Model building module: Build a weld defect detection model based on weld defect characteristics and defect labels, and perform defect detection on the weld based on the built weld defect detection model to obtain accurate detection results.

[0010] Furthermore, the weld thermal imaging data includes a weld thermal map and defect labels.

[0011] Furthermore, the method of performing enhanced preprocessing on the weld thermal imaging data includes:

[0012] The block size is preset based on the size of the weld heat map in the weld thermal imaging data, and the weld heat map is evenly divided into blocks based on the block size to obtain heat map sub-blocks. Each heat map sub-block is numbered incrementally from left to right and from top to bottom based on the corresponding position in the weld heat map to obtain the sub-block number of each heat map sub-block; the mean of the grayscale values ​​of the pixels in each heat map sub-block is used as the initial grayscale center, and the average attribution calculation is performed on each pixel in each heat map sub-block based on the initial grayscale center to obtain the average attribution degree; the mean of the average attribution degrees is used as the comparison threshold , the standard deviation of the grayscale values ​​of the pixels in each heat map sub-block is used as the sub-block contrast. Based on the contrast threshold, the heat map sub-blocks with a sub-block contrast greater than or equal to the contrast threshold are recorded as high-definition sub-blocks, and the heat map sub-blocks with a sub-block contrast less than the contrast threshold are recorded as sub-blocks to be enhanced. The sub-blocks to be enhanced are contrast enhanced to obtain enhanced sub-blocks. The enhanced sub-blocks and the high-definition sub-blocks are sequentially spliced ​​based on the sub-block numbers to obtain an enhanced heat map. The pixels at the splicing edge of the enhanced sub-blocks and the high-definition sub-blocks are recorded as boundary pixels. The boundary pixels are smoothed. The formula for boundary smoothing is: ;in, Representative The smoothed pixel values ​​of the boundary pixels, Representative The pixel values ​​of the boundary pixels, represents pi, Represents the smoothing factor, which is used to control the smoothness of the boundary pixels. Representative The pixel values ​​of the boundary pixels, represents the set of boundary pixels, Representative The boundary pixels and The distance of the boundary pixels, and .

[0013] Furthermore, the formula for calculating the average attribution is:

[0014] ;in, represents the average degree of belonging, Represents the number of initial grayscale centers, Representative The initial gray center, Represents the attribution width.

[0015] Furthermore, the method of contrast enhancing the sub-block to be enhanced includes:

[0016] Perform histogram distribution statistics on each sub-block to be enhanced. The formula for performing histogram distribution statistics is: ;in, Representatives belong to The number of pixels with gray levels, Represents the first The gray value of a pixel, Represents the grayscale, and the grayscale meets the sub-block range condition: , Represents the grayscale value set of pixels appearing in the sub-block to be enhanced, Represents the number of pixels in the sub-block to be enhanced, Represents the indicator function; a grayscale histogram is constructed based on the histogram distribution statistics, and the grayscale value of each pixel in the enhanced sub-block is enhanced based on the grayscale histogram. The formula for the enhancement mapping is: ;in, represents the enhanced grayscale value, Represents the original grayscale value of the pixel, Represents the grayscale value corresponding to the minimum columnar value in the grayscale histogram, Represents the number of pixels in the sub-block to be enhanced, The minimum value of the column in the grayscale histogram, Represents the total number of gray levels, represents the cut-off function, and the sub-block to be enhanced after the enhancement mapping is recorded as the enhanced sub-block.

[0017] Furthermore, the method of adaptively segmenting the enhanced heat map includes:

[0018] The lower left corner of the enhanced heat map is used as the origin of the pixel map coordinates, the horizontal index of the pixel in the enhanced heat map is used as the horizontal coordinate value of the pixel map coordinates, and the vertical index of the pixel in the enhanced heat map is used as the vertical coordinate value of the pixel map coordinates, and each pixel in the enhanced heat map is mapped to the pixel map coordinates; pixel gradients of the pixels in the enhanced heat map are calculated based on the map coordinates to obtain pixel gradient values; a weld point selection window is preset, and the weld point selection window is moved without covering based on the weld point selection window with the upper left corner of the enhanced heat map as the starting point, and the pixels in the weld point selection window are recorded as window pixels, and the gradient marking threshold and pixel marking threshold are calculated for the window pixels after each without covering movement; an initial weld point set is preset, and the initial weld point set is initialized to be empty. The window pixels are double-screened based on the gradient marking threshold and the pixel marking threshold, and the window pixels whose pixel gradient values ​​are greater than or equal to the gradient calibration threshold and whose pixel values ​​are greater than or equal to the pixel marking threshold are counted into the initial weld point set;

[0019] Based on the pixels in the initial solder joint set, the pixel map coordinates are binary mapped, and the corresponding positions of the pixels in the initial solder joint set in the pixel map coordinates are marked as value one, and the remaining positions in the pixel map coordinates are marked as zero to obtain a binary mapping image. An extended window is preset, and each pixel in the binary mapping image is window eroded based on the extended window to obtain an eroded binary image. The window erosion method includes: for a selected pixel, based on the extended window, the selected pixel is used as the erosion center of the extended window, and the other pixels framed in the extended window are used as erosion edges. When the value of any erosion edge is zero, the value of the erosion center is zero. When When the values ​​of all corrosion edges are one, the value of the corrosion center is one; based on the expanded window, each pixel in the corrosion binary image is expanded to obtain an expanded binary image, and the window expansion method includes: for a selected pixel, based on the expanded window, the selected pixel is used as the expansion center of the expanded window, and other pixels framed in the expanded window are used as expansion edges. When the value of any expanded edge is one, the value of the pixel center is one, and when the values ​​of all window edges are zero, the value of the pixel center is zero; based on the expanded binary image, the enhanced heat map is segmented according to the area with the most concentrated values ​​of one in the expanded binary image to obtain a weld area map.

[0020] Furthermore, the method of extracting features from the weld area map includes:

[0021] Perform linear temperature transformation on the weld area map to obtain the weld temperature map. Based on the weld temperature map, the temperature gradient of each pixel in the weld temperature map is calculated using a derivative formula. A sparse grid is preset. Based on the sparse grid, the pixel in the upper left corner of the weld temperature map is used as the starting feature point. The sparse grid is used as the selection span to select the grid points. All selected grid points and the starting feature point constitute a feature point set. The four neighborhoods are used as the neighborhood selection window to calculate the significance score of each pixel in the feature point set to obtain the significance score. A significance threshold is preset, and pixels in the feature point set whose significance score is greater than or equal to the significance threshold are retained as features. Points are generated for each retained feature point based on the eight neighborhoods. The temperature description set includes the average temperature in the eight neighborhoods, the mean of the temperature gradient in the eight neighborhoods, the standard deviation of the temperature gradient in the eight neighborhoods, and the ratio of the maximum temperature gradient to the minimum temperature gradient in the eight neighborhoods. The cosine similarity calculation formula is used to calculate the feature similarity between different retained feature points based on the temperature description set of each retained feature point. A feature similarity threshold is preset, and the retained feature points with feature similarity greater than or equal to the feature similarity threshold are counted into the same feature merging set. The pixels in each feature merging set are merged to obtain merged features. All merged features constitute the weld features.

[0022] Furthermore, the formula for calculating the significant score is:

[0023] ;in, Represents the first feature point in the set The saliency score of pixels, represents the neighborhood selection window, represents the number of pixels in the neighborhood selection window, Represents the first feature point in the set The pixel temperature of each pixel, Represents the first The pixel temperature of each pixel, Represents the first The temperature gradient of each pixel;

[0024] The formula for feature merging is: ;in, represents the merged features, Represents the number of pixels in the feature merging set, represents the combined feature set, Representative feature merging set pixels and The feature similarity of pixels, Representative feature merging set The features of pixels, Representative feature merging set pixel features.

[0025] Furthermore, the weld defect detection model is constructed in the following manner:

[0026] The weld features and defect labels are used as the training sample set, the GAN model is used as the initial model, and the GAN model is trained using the training sample set. The weld features are used as the input data of the weld defect detection model, and the predicted defect categories are used as the output data of the weld defect detection model. Minimizing the error between the actual defect label and the predicted defect category is used as the training goal, and the recall rate function is used as the loss function of the weld defect detection model. When the loss function reaches convergence, the training is stopped to obtain the weld defect detection model.

[0027] The present invention provides a weld defect detection method based on infrared thermal imaging and deep learning, comprising:

[0028] S1. Collect weld thermal imaging data, perform enhancement preprocessing on the weld thermal imaging data, and obtain an enhanced thermal map;

[0029] S2. Adaptively segment the enhanced heat map to obtain a weld area map;

[0030] S3, performing sparse feature extraction on the weld area map to obtain weld features;

[0031] S4. Construct a weld defect detection model based on weld defect characteristics and defect labels, and perform defect detection on the weld based on the constructed weld defect detection model to obtain accurate detection results.

[0032] The technical effects and advantages of the weld defect detection system and method based on infrared thermal imaging and deep learning of the present invention are as follows:

[0033] The present invention significantly improves the clarity of the real defect area by performing enhanced preprocessing on the weld thermal imaging data, while maintaining the stability of the non-defect area, thereby effectively reducing the interference of pseudo defects, eliminating the boundary abruptness caused by the enhancement operation in the traditional method, and ensuring the overall visual continuity of the weld thermal image; by combining the dual threshold screening of pixel gradient and grayscale value, the high accuracy of the initial weld point set is ensured, and the pseudo weld area caused by high reflection or surface defects is effectively avoided; by dynamically controlling the value rules of the corrosion center and the expansion edge, the edge structure of the binary image is optimized, making the weld area segmentation result more accurate; by using a dynamic expansion window, the operation intensity can be adjusted according to the actual complexity of the weld area, thereby improving the adaptability to complex weld areas; the sparse sampling algorithm effectively reduces the computational complexity while retaining the key temperature feature points; combined with the significant scoring mechanism of four neighborhoods and eight neighborhoods, the representativeness of the feature points is ensured, the accuracy of the feature point selection is improved, and combined with the temperature gradient and statistical description set, multi-angle weld defect information is provided, thereby improving the discrimination ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the weld defect detection system based on infrared thermal imaging and deep learning of the present invention;

[0035] Figure 2 Schematic diagram of the weld defect detection method based on infrared thermal imaging and deep learning of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0037] Example 1

[0038] See also Figure 1 As shown, the weld defect detection system based on infrared thermal imaging and deep learning described in this embodiment includes:

[0039] Data acquisition and processing module: collects weld thermal imaging data, performs enhancement preprocessing on the weld thermal imaging data, and obtains enhanced thermal maps;

[0040] Adaptive segmentation module: adaptively segment the enhanced heat map to obtain the weld area map;

[0041] Defect feature extraction module: performs sparse feature extraction on the weld area map to obtain weld features;

[0042] Model building module: Build a weld defect detection model based on weld defect characteristics and defect labels, and perform defect detection on welds based on the built weld defect detection model to obtain accurate detection results;

[0043] Each module is connected via wired and / or wireless means to achieve data transmission between modules;

[0044] Weld thermal imaging data includes weld heat maps and defect labels. The weld heat maps are collected by high-sensitivity infrared thermal imaging sensors; defect labels are identifiers of defects formed during the welding process. Common defect labels include porosity, slag inclusions, incomplete penetration, lack of fusion, cracks, pits and undercuts.

[0045] Methods for preprocessing weld thermography data include:

[0046] The block size is preset based on the size of the weld heat map in the weld thermal imaging data. Common block sizes are Pixels, Pixels, the size of the block size is determined based on the resolution of the weld heat map, and the length and width of the block size both meet the divisibility condition, that is, the length of the block size can be divided by the long side of the resolution of the weld heat map, and the width of the block size can be divided by the wide side of the resolution of the weld heat map. The weld heat map is evenly divided into blocks based on the block size to obtain heat map sub-blocks. Each heat map sub-block is numbered incrementally from left to right and from top to bottom based on the corresponding position in the weld heat map to obtain the sub-block number of each heat map sub-block; the mean of the grayscale values ​​of the pixels in each heat map sub-block is used as the initial grayscale center, and the average attribution calculation is performed on each pixel in each heat map sub-block based on the initial grayscale center. The formula for the average attribution calculation is: ;in, represents the average degree of belonging, Represents the number of initial grayscale centers, Representative The initial gray center, represents the attribution width, which is used to control the sensitivity of the input data;

[0047] The mean of the average degree of belonging is used as the contrast threshold, and the standard deviation of the grayscale values ​​of the pixels in each heat map sub-block is used as the sub-block contrast. Based on the contrast threshold, the heat map sub-blocks with a sub-block contrast greater than or equal to the contrast threshold are recorded as high-definition sub-blocks, and the heat map sub-blocks with a sub-block contrast less than the contrast threshold are recorded as sub-blocks to be enhanced. Contrast enhancement is performed on the sub-blocks to be enhanced. The contrast enhancement method includes: performing histogram distribution statistics on each sub-block to be enhanced. The formula for performing histogram distribution statistics is: ;in, Representatives belong to The number of pixels with gray levels, Represents the first The gray value of a pixel, Represents the grayscale, and the grayscale meets the sub-block range condition: , Represents the grayscale value set of pixels appearing in the sub-block to be enhanced, Represents the number of pixels in the sub-block to be enhanced, Represents the indicator function. When the first The gray value of a pixel is equal to the gray level When the gray value of the sub-block to be enhanced is The gray value of the pixel is not equal to the gray level When the gray value is , the function value is 0; a gray histogram is constructed based on the histogram distribution statistics. The gray histogram is a columnar statistical graph with the gray value of the pixel in the sub-block to be enhanced as the horizontal coordinate and the number of pixels with the gray value corresponding to the horizontal coordinate as the vertical coordinate. The gray histogram shows the number of gray values ​​at each level and intuitively shows the quality of the image. If the histogram is concentrated at low gray levels, it means that the image is too dark, and if it is concentrated at high gray levels, it means that the image is too bright.

[0048] Based on the grayscale histogram, the grayscale value of each pixel in the enhanced sub-block is enhanced and mapped. The formula for the enhanced mapping is: ;in, represents the enhanced grayscale value, Represents the original grayscale value of the pixel, Represents the grayscale value corresponding to the minimum columnar value in the grayscale histogram, Represents the number of pixels in the sub-block to be enhanced, The minimum value of the column in the grayscale histogram, Represents the total number of gray levels, represents a rounding function. In this embodiment, the preferred rounding function is a rounding function. The sub-block to be enhanced after the enhancement mapping is recorded as an enhanced sub-block. The enhanced sub-block and the high-definition sub-block are sequentially spliced ​​based on the sub-block numbers to obtain an enhanced heat map. The pixels at the splicing edge of the enhanced sub-block and the high-definition sub-block are recorded as boundary pixels. The boundary pixels are smoothed to reduce the contrast difference at the boundary between the enhanced sub-block and the high-definition sub-block and make the boundary transition smooth. The formula for boundary smoothing is:

[0049] ;in, Representative The smoothed pixel values ​​of the boundary pixels, Representative The pixel values ​​of the boundary pixels, represents pi, Represents the smoothing factor, which is used to control the smoothness of the boundary pixels. Representative The pixel values ​​of the boundary pixels, represents the set of boundary pixels, Representative The boundary pixels and The distance of the boundary pixels, and Commonly used distance measurement formulas include the Euclidean distance formula and the Chebyshev distance formula. In weld inspection, pseudo defects are easily caused by factors such as noise or surface irregularities. By enhancing the pixels in the local area of ​​the weld heat map, the contrast of the real defects is improved, while the contrast of the non-defect area remains stable, making it easier to distinguish the real defects in the image and reducing the detection rate of pseudo defects.

[0050] Methods for adaptive segmentation of enhanced heatmaps include:

[0051] The lower left corner of the enhanced heat map is used as the origin of the pixel map coordinates, the horizontal index of the pixel in the enhanced heat map is used as the horizontal coordinate value of the pixel map coordinates, and the vertical index of the pixel in the enhanced heat map is used as the vertical coordinate value of the pixel map coordinates. Each pixel in the enhanced heat map is mapped to the pixel map coordinates; the pixel gradient is calculated for the pixels in the enhanced heat map based on the map coordinates. The formula for pixel gradient calculation is: ;in, Represents the horizontal axis value in the pixel coordinates , the vertical axis value is The pixel gradient value of the pixel at , Represents the derivative of the pixel on the horizontal axis in the pixel map coordinates, Represents the derivative of the pixel on the vertical axis in the pixel map coordinates; a weld point selection window is preset, and the weld point selection window is moved without covering based on the weld point selection window with the upper left corner of the enhanced heat map as the starting point (that is, the pixels in the weld point selection window after the move do not overlap with the pixels in the previous weld point selection window), and the pixels in the weld point selection window are recorded as window pixels. The gradient marking threshold and pixel marking threshold are calculated for the window pixels after each non-coverage movement. The calculation formula of the gradient marking threshold is:

[0052] ;in, represents the gradient calibration threshold, Represents the mean of the pixel gradient values ​​of the window pixels, Represents the standard deviation of the pixel gradient values ​​of the window pixels, Represents the gradient adjustment factor, which is used to control the sensitivity of the gradient marking threshold. The calculation formula of the pixel marking threshold is:

[0053] ;in, represents the pixel labeling threshold, Represents the mean grayscale value of the window pixels, Represents the standard deviation of the grayscale value of the window pixels, represents a pixel adjustment factor, which is used to control the sensitivity of the pixel marking threshold. An initial solder joint set is preset and initialized to be empty. Window pixels are double-screened based on the gradient marking threshold and the pixel marking threshold. Window pixels whose pixel gradient values ​​are greater than or equal to the gradient calibration threshold and whose pixel values ​​are greater than or equal to the pixel marking threshold are counted into the initial solder joint set. In this embodiment, the preferred gradient adjustment factor is 0.6, and the preferred pixel adjustment factor is 0.9.

[0054] Based on the pixels in the initial solder joint set, the pixel map coordinates are binary mapped, and the corresponding positions of the pixels in the initial solder joint set in the pixel map coordinates are marked as value one, and the remaining positions in the pixel map coordinates are marked as zero to obtain a binary map. An extended window is preset, and each pixel in the binary map is window eroded based on the extended window to obtain an eroded binary map. The window erosion method includes: for a selected pixel, based on the extended window, the selected pixel is used as the erosion center of the extended window, and the other pixels framed in the extended window are used as erosion edges. When the value of any erosion edge is zero, the value of the erosion center is zero, and when all erosion edges are zero, the value of the erosion center is zero. When the value of the edge is one, the value of the corrosion center is one; based on the expansion window, each pixel in the corrosion binary image is expanded to obtain an expanded binary image. The window expansion method includes: for a selected pixel, based on the expansion window, the selected pixel is used as the expansion center of the expansion window, and the other pixels framed in the expansion window are used as the expansion edges. When the value of any expansion edge is one, the value of the pixel center is one, and when the values ​​of all window edges are zero, the value of the pixel center is zero; based on the expansion binary image, the strengthening heat map is segmented according to the area with the most concentrated values ​​of one in the expansion binary image to obtain a weld area map; assuming the binary mapping image is: , the eroded binary image after window corrosion is: , the expanded binary image after window expansion is: .

[0055] Methods for extracting features from weld area maps include:

[0056] Perform linear temperature transformation on the weld area map to obtain the weld temperature map. Based on the weld temperature map, the temperature gradient of each pixel in the weld temperature map is calculated using a derivative formula. A sparse grid is preset. Based on the sparse grid, the pixel in the upper left corner of the weld temperature map is used as the starting feature point. The sparse grid is used as the selection span to select grid points. All selected grid points and the starting feature point constitute a feature point set. The four neighborhoods are used as the neighborhood selection window to calculate the significant score of each pixel in the feature point set. The formula for calculating the significant score is: ;in, Represents the first feature point in the set The saliency score of pixels, represents the neighborhood selection window, represents the number of pixels in the neighborhood selection window, Represents the first feature point in the set The pixel temperature of each pixel, Represents the first The pixel temperature of each pixel, Represents the first The temperature gradient of each pixel is calculated; a significant threshold is preset, and pixels in the feature point set whose significant scores are greater than or equal to the significant threshold are taken as retained feature points; a temperature description set is generated for each retained feature point based on the eight neighborhoods, and the temperature description set includes the average temperature in the eight neighborhoods, the mean of the temperature gradient in the eight neighborhoods, the standard deviation of the temperature gradient in the eight neighborhoods, and the ratio of the maximum temperature gradient to the minimum temperature gradient in the eight neighborhoods; based on the temperature description set of each retained feature point, the cosine similarity calculation formula is used to calculate the feature similarity between different retained feature points, a feature similarity threshold is preset, and the retained feature points with feature similarity greater than or equal to the feature similarity threshold are counted into the same feature merging set, and the pixels in each feature merging set are merged. The formula for feature merging is: ;in, represents the merged features, Represents the number of pixels in the feature merging set, represents the combined feature set, Representative feature merging set pixels and The feature similarity of pixels, Representative feature merging set The features of pixels, Representative feature merging set The features of each pixel are combined, and all the merged features constitute the weld features; the four neighborhoods are the pixels corresponding to the upper, lower, left and right positions, and the eight neighborhoods are the pixels corresponding to the upper, lower, left, right, upper left, lower left, upper right and lower right positions; the linear temperature conversion first obtains the pixel value range of the weld area map, which is usually 0 to 255 or larger. These pixel values ​​represent the brightness or color intensity of the image. According to the equipment calibration, the actual temperature range represented in the image is determined. The pixel feature is the temperature corresponding to the pixel in the weld temperature map.

[0057] The weld defect detection model is constructed in the following ways:

[0058] The weld features and defect labels are used as the training sample set, the GAN model is used as the initial model, and the GAN model is trained using the training sample set. The weld features are used as the input data of the weld defect detection model, and the predicted defect categories are used as the output data of the weld defect detection model. Minimizing the error between the actual defect label and the predicted defect category is used as the training goal, and the recall rate function is used as the loss function of the weld defect detection model. When the loss function reaches convergence, the training is stopped to obtain the weld defect detection model. Defect detection is performed on the weld heat map data based on the weld defect detection model to obtain accurate detection results.

[0059] This embodiment significantly improves the clarity of real defect areas while maintaining the stability of non-defect areas by performing enhanced preprocessing on weld thermal imaging data, thereby effectively reducing the interference of pseudo-defects, eliminating the boundary abruptness caused by the enhancement operation in traditional methods, and ensuring the overall visual continuity of the weld thermal image. By combining dual threshold screening of pixel gradient and grayscale value, the high accuracy of the initial weld point set is ensured, and pseudo-weld areas caused by high reflection or surface defects are effectively avoided. By dynamically controlling the value selection rules of the corrosion center and the expansion edge, the edge structure of the binary image is optimized, making the weld area segmentation result more accurate. The use of a dynamic expansion window can adjust the operation intensity according to the actual complexity of the weld area, improving the adaptability to complex weld areas. The sparse sampling algorithm effectively reduces the computational complexity while retaining key temperature feature points. The combination of the four-neighborhood and eight-neighborhood significance scoring mechanisms ensures the representativeness of the feature points and improves the accuracy of feature point selection. The combination of temperature gradient and statistical description set provides multi-angle weld defect information and enhances the discriminative ability of the model.

[0060] Example 2

[0061] See also Figure 2 As shown, for the parts not described in detail in this embodiment, please refer to the description of Example 1. A weld defect detection method based on infrared thermal imaging and deep learning is provided, including:

[0062] S1. Collect weld thermal imaging data, perform enhancement preprocessing on the weld thermal imaging data, and obtain an enhanced thermal map;

[0063] S2. Adaptively segment the enhanced heat map to obtain a weld area map;

[0064] S3, performing sparse feature extraction on the weld area map to obtain weld features;

[0065] S4. Construct a weld defect detection model based on weld defect characteristics and defect labels, and perform defect detection on the weld based on the constructed weld defect detection model to obtain accurate detection results.

[0066] Example 3

[0067] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation mode of the weld defect detection method based on infrared thermal imaging and deep learning provided above is implemented.

[0068] Since the electronic device introduced in this embodiment is an electronic device used to implement the weld defect detection method based on infrared thermal imaging and deep learning in the embodiment of this application, based on the weld defect detection method based on infrared thermal imaging and deep learning introduced in the embodiment of this application, those skilled in the art can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application will not be described in detail here. As long as those skilled in the art implement the electronic device used in the weld defect detection method based on infrared thermal imaging and deep learning in the embodiment of this application, it falls within the scope of protection of this application.

[0069] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters and thresholds in the formulas are set by technicians in this field according to actual conditions.

[0070] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for users of ordinary skill in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A weld defect detection method based on infrared thermal imaging and deep learning, characterized in that: include: S1. Collect weld thermal imaging data, perform enhancement preprocessing on the weld thermal imaging data, and obtain an enhanced thermal map; S2. Adaptively segment the enhanced heat map to obtain a weld area map; S3, performing sparse feature extraction on the weld area map to obtain weld features; S4. Constructing a weld defect detection model based on the weld defect characteristics and defect labels, and performing defect detection on the weld based on the constructed weld defect detection model to obtain accurate detection results; The method of adaptively segmenting the enhanced heat map includes: The lower left corner of the enhanced heat map is used as the origin of the pixel map coordinates, the horizontal index of the pixel in the enhanced heat map is used as the horizontal coordinate value of the pixel map coordinates, and the vertical index of the pixel in the enhanced heat map is used as the vertical coordinate value of the pixel map coordinates, and each pixel in the enhanced heat map is mapped to the pixel map coordinates; pixel gradients of the pixels in the enhanced heat map are calculated based on the map coordinates to obtain pixel gradient values; a weld point selection window is preset, and the weld point selection window is moved without covering based on the weld point selection window with the upper left corner of the enhanced heat map as the starting point, and the pixels in the weld point selection window are recorded as window pixels, and the gradient marking threshold and pixel marking threshold are calculated for the window pixels after each without covering movement; an initial weld point set is preset, and the initial weld point set is initialized to be empty. The window pixels are double-screened based on the gradient marking threshold and the pixel marking threshold, and the window pixels whose pixel gradient values ​​are greater than or equal to the gradient calibration threshold and whose pixel values ​​are greater than or equal to the pixel marking threshold are counted into the initial weld point set; Based on the pixels in the initial solder joint set, the pixel map coordinates are binary mapped, and the corresponding positions of the pixels in the initial solder joint set in the pixel map coordinates are marked as value one, and the remaining positions in the pixel map coordinates are marked as zero to obtain a binary mapping image. An extended window is preset, and each pixel in the binary mapping image is window eroded based on the extended window to obtain an eroded binary image. The window erosion method includes: for a selected pixel, based on the extended window, the selected pixel is used as the erosion center of the extended window, and the other pixels framed in the extended window are used as erosion edges. When the value of any erosion edge is zero, the value of the erosion center is zero. When When the values ​​of all corrosion edges are one, the value of the corrosion center is one; based on the expanded window, each pixel in the corrosion binary image is expanded to obtain an expanded binary image, and the window expansion method includes: for a selected pixel, based on the expanded window, the selected pixel is used as the expansion center of the expanded window, and other pixels framed in the expanded window are used as expansion edges. When the value of any expanded edge is one, the value of the pixel center is one, and when the values ​​of all window edges are zero, the value of the pixel center is zero; based on the expanded binary image, the enhanced heat map is segmented according to the area with the most concentrated values ​​of one in the expanded binary image to obtain a weld area map.

2. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 1 is characterized in that: The weld thermal imaging data includes a weld thermal map and defect labels.

3. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 2 is characterized in that: The method of performing enhanced preprocessing on the weld thermal imaging data includes: The block size is preset based on the size of the weld heat map in the weld thermal imaging data, and the weld heat map is evenly divided into blocks based on the block size to obtain heat map sub-blocks. Each heat map sub-block is numbered incrementally from left to right and from top to bottom based on the corresponding position in the weld heat map to obtain the sub-block number of each heat map sub-block; the mean of the grayscale values ​​of the pixels in each heat map sub-block is used as the initial grayscale center, and the average attribution calculation is performed on each pixel in each heat map sub-block based on the initial grayscale center to obtain the average attribution degree; the mean of the average attribution degrees is used as the comparison threshold , the standard deviation of the grayscale values ​​of the pixels in each heat map sub-block is used as the sub-block contrast. Based on the contrast threshold, the heat map sub-blocks with a sub-block contrast greater than or equal to the contrast threshold are recorded as high-definition sub-blocks, and the heat map sub-blocks with a sub-block contrast less than the contrast threshold are recorded as sub-blocks to be enhanced. The sub-blocks to be enhanced are contrast enhanced to obtain enhanced sub-blocks. The enhanced sub-blocks and the high-definition sub-blocks are sequentially spliced ​​based on the sub-block numbers to obtain an enhanced heat map. The pixels at the splicing edge of the enhanced sub-blocks and the high-definition sub-blocks are recorded as boundary pixels. The boundary pixels are smoothed. The formula for boundary smoothing is: ;in, Representative The smoothed pixel values ​​of the boundary pixels, Representative The pixel values ​​of the boundary pixels, represents pi, Represents the smoothing factor, which is used to control the smoothness of the boundary pixels. Representative The pixel values ​​of the boundary pixels, represents the set of boundary pixels, Representative The boundary pixels and The distance of the boundary pixels, and .

4. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 3 is characterized in that: The formula for calculating the average attribution is: ;in, represents the average degree of belonging, Represents the number of initial grayscale centers, Representative The initial gray center, Represents the attribution width.

5. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 4 is characterized in that: The method of performing contrast enhancement on the sub-block to be enhanced includes: Perform histogram distribution statistics on each sub-block to be enhanced. The formula for performing histogram distribution statistics is: ;in, Representatives belong to The number of pixels with gray levels, Represents the first The gray value of a pixel, Represents the grayscale, and the grayscale meets the sub-block range condition: , Represents the grayscale value set of pixels appearing in the sub-block to be enhanced, Represents the number of pixels in the sub-block to be enhanced, Represents the indicator function; a grayscale histogram is constructed based on the histogram distribution statistics, and the grayscale value of each pixel in the enhanced sub-block is enhanced based on the grayscale histogram. The formula for the enhancement mapping is: ;in, represents the enhanced grayscale value, Represents the original grayscale value of the pixel, Represents the grayscale value corresponding to the minimum columnar value in the grayscale histogram, Represents the number of pixels in the sub-block to be enhanced, The minimum value of the column in the grayscale histogram, Represents the total number of gray levels, represents the cut-off function, and the sub-block to be enhanced after the enhancement mapping is recorded as the enhanced sub-block.

6. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 5 is characterized in that: The method of performing sparse feature extraction on the weld area map includes: Perform linear temperature transformation on the weld area map to obtain the weld temperature map. Based on the weld temperature map, the temperature gradient of each pixel in the weld temperature map is calculated using a derivative formula. A sparse grid is preset. Based on the sparse grid, the pixel in the upper left corner of the weld temperature map is used as the starting feature point. The sparse grid is used as the selection span to select the grid points. All selected grid points and the starting feature point constitute a feature point set. The four neighborhoods are used as the neighborhood selection window to calculate the significance score of each pixel in the feature point set to obtain the significance score. A significance threshold is preset, and pixels in the feature point set whose significance score is greater than or equal to the significance threshold are retained as features. Points are generated for each retained feature point based on the eight neighborhoods. The temperature description set includes the average temperature in the eight neighborhoods, the mean of the temperature gradient in the eight neighborhoods, the standard deviation of the temperature gradient in the eight neighborhoods, and the ratio of the maximum temperature gradient to the minimum temperature gradient in the eight neighborhoods. The cosine similarity calculation formula is used to calculate the feature similarity between different retained feature points based on the temperature description set of each retained feature point. A feature similarity threshold is preset, and the retained feature points with feature similarity greater than or equal to the feature similarity threshold are counted into the same feature merging set. The pixels in each feature merging set are merged to obtain merged features. All merged features constitute the weld features.

7. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 6, characterized in that: The formula for calculating the significant score is: ;in, Represents the first feature point in the set The saliency score of pixels, represents the neighborhood selection window, represents the number of pixels in the neighborhood selection window, Represents the first feature point in the set The pixel temperature of each pixel, Represents the first The pixel temperature of each pixel, Represents the first The temperature gradient of each pixel; The formula for feature merging is: ;in, represents the merged features, Represents the number of pixels in the feature merging set, represents the combined feature set, Representative feature merging set pixels and The feature similarity of pixels, Representative feature merging set The features of pixels, Representative feature merging set pixel features.

8. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 7, characterized in that: The method of constructing the weld defect detection model includes: The weld features and defect labels are used as the training sample set, the GAN model is used as the initial model, and the GAN model is trained using the training sample set. The weld features are used as the input data of the weld defect detection model, and the predicted defect categories are used as the output data of the weld defect detection model. Minimizing the error between the actual defect label and the predicted defect category is used as the training goal, and the recall rate function is used as the loss function of the weld defect detection model. When the loss function reaches convergence, the training is stopped to obtain the weld defect detection model.

9. A weld defect detection system based on infrared thermal imaging and deep learning, which is used to implement the weld defect detection method based on infrared thermal imaging and deep learning according to any one of claims 1 to 8, characterized in that: include: Data acquisition and processing module: collects weld thermal imaging data, performs enhancement preprocessing on the weld thermal imaging data, and obtains enhanced thermal maps; Adaptive segmentation module: adaptively segment the enhanced heat map to obtain the weld area map; Defect feature extraction module: performs sparse feature extraction on the weld area map to obtain weld features; Model building module: Build a weld defect detection model based on weld defect characteristics and defect labels, and perform defect detection on the weld based on the built weld defect detection model to obtain accurate detection results.

Images