Weld defect detection system and method based on infrared thermal imaging and deep learning

Abstract

The invention belongs to the technical field of welding detection, and discloses a welding seam defect detection system and method based on infrared thermal imaging and deep learning, and the system comprises a data collection and processing module which collects welding seam thermal imaging data, carries out the enhancement preprocessing of the welding seam thermal imaging data, and obtains an enhanced heat map; the self-adaptive segmentation module is used for carrying out self-adaptive segmentation on the enhanced heat map to obtain a welding seam area map; the defect feature extraction module is used for carrying out sparse feature extraction on the welding seam region map to obtain welding seam features; the model building module is used for building a welding seam defect detection model based on the welding seam defect characteristics and the defect labels and carrying out defect detection on the welding seam based on the built welding seam defect detection model to obtain an accurate detection result; the high efficiency and the real-time performance of the detection process are ensured, and the welding seam defect detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of welding detection. More specifically, the present invention relates to a weld defect detection system and method based on infrared thermal imaging and deep learning. Background Art

[0002] In modern industry, welding is an indispensable process step in many manufacturing processes and is widely used in fields such as aerospace, automotive manufacturing, shipbuilding, petrochemical industry, etc. The quality of welding directly affects the strength, safety and service life of structural components. Therefore, the detection and evaluation of weld quality are crucial. During the welding process, various defects may occur in the weld due to factors such as materials, processes, equipment, and operations, such as cracks, pores, inclusions, lack of penetration, overburning, etc. If these defects are not detected and repaired in time, they may lead to the failure, damage or even major safety accidents of structural components.

[0003] Existing infrared thermal imaging systems mostly rely on static thermal images after welding for analysis. However, these images often fail to consider the instantaneous and dynamic nature of temperature changes during the welding process. This means that some transient and minute thermal changes during the welding process are easily masked by the heat diffusion effect, resulting in defects not being captured in time. Existing defect detection systems often cannot handle the complex interaction effects between multiple defects simultaneously. For example, the presence of pores may cause local overheating, while cracks may lead to uneven temperature distribution. These factors are superimposed on each other, generating "false defect" signals and resulting in misjudgment of defects. When there are current fluctuations or sudden changes in the welding speed during the welding process, the infrared thermal imaging images may present thermal patterns similar to defects, such as a sudden increase or decrease in surface temperature. This kind of change may lead to "false defects" and treat the "false defects" as real defects.

[0004] In view of this, the present invention proposes a weld defect detection system and method based on infrared thermal imaging and deep learning to solve the above problems. Summary of the Invention

[0005] In order to overcome the above defects of the prior art and to achieve the above object, the present invention provides the following technical solution: A weld defect detection system based on infrared thermal imaging and deep learning, comprising: A data acquisition and processing module: collecting weld thermal imaging data, performing enhanced preprocessing on the weld thermal imaging data to obtain an enhanced heat map; An adaptive segmentation module: adaptively segmenting the enhanced heat map to obtain a weld area map; A defect feature extraction module: performing sparse feature extraction on the weld area map to obtain weld features; Model construction module: Construct a weld defect detection model based on weld defect characteristics and defect labels, and perform defect detection on the weld based on the constructed weld defect detection model to obtain accurate detection results.

[0006] Further, the weld thermal imaging data includes a weld heat map and defect labels.

[0007] Further, the method for enhancing and preprocessing the weld thermal imaging data includes: Preset a block size based on the size of the weld heat map in the weld thermal imaging data, equally divide the weld heat map based on the block size to obtain heat map sub-blocks, and number each heat map sub-block incrementally in the order from left to right and top to bottom based on its corresponding position in the weld heat map to obtain the sub-block number of each heat map sub-block; use the mean of the gray values of the pixels in each heat map sub-block as the initial gray center, and perform average attribution calculation on each pixel in each heat map sub-block based on the initial gray center to obtain the average attribution degree; use the mean of the average attribution degrees as the comparison threshold, use the standard deviation of the gray values of the pixels in each heat map sub-block as the sub-block contrast, and based on the comparison threshold, mark the heat map sub-blocks with sub-block contrast greater than or equal to the comparison threshold as high-clarity sub-blocks, mark the heat map sub-blocks with sub-block contrast less than the comparison threshold as sub-blocks to be enhanced, perform contrast enhancement on the sub-blocks to be enhanced to obtain enhanced sub-blocks, splice the enhanced sub-blocks and high-clarity sub-blocks sequentially based on the sub-block numbers to obtain an enhanced heat map, mark the pixels at the splicing edges of the enhanced sub-blocks and high-clarity sub-blocks as boundary pixels, and perform boundary smoothing on the boundary pixels. The formula for boundary smoothing is: ; where represents the smoothed pixel value of the th boundary pixel, represents the pixel value of the th boundary pixel, represents pi, represents the smoothing factor used to control the smoothing degree of the boundary pixels, represents the pixel value of the th boundary pixel, represents the set of boundary pixels, represents the th boundary pixel and the distance between the th boundary pixel, and .

[0008] Further, the formula for performing average attribution calculation is: ; where represents the average attribution degree, represents the number of initial gray centers, represents the An initial grayscale center, Represents the attribution width.

[0009] Furthermore, the method for contrast enhancement of the to-be-enhanced sub-blocks includes: Perform histogram distribution statistics on each to-be-enhanced sub-block. The formula for histogram distribution statistics is: ; where Represents the number of pixels belonging to the th gray level, Represents the gray value of the th pixel in the to-be-enhanced sub-block, Represents the gray level, and the gray level satisfies the sub-block range condition: , Represents the set of gray values of the pixels that appear in the to-be-enhanced sub-block, Represents the number of pixels in the to-be-enhanced sub-block, Represents the indicator function; construct a gray histogram based on the histogram distribution statistics results, and perform enhanced mapping on the gray value of each pixel in the to-be-enhanced sub-block based on the gray histogram. The formula for enhanced mapping is: ; where Represents the enhanced gray value, Represents the original gray value of the pixel, Represents the gray value corresponding to the minimum value of the column in the gray histogram, Represents the number of pixels in the to-be-enhanced sub-block, The minimum value of the column in the gray histogram, Represents the total number of gray levels, Represents the selection function. Denote the to-be-enhanced sub-block after enhanced mapping as the enhanced sub-block.

[0010] Furthermore, the method for adaptively segmenting the enhanced heat map includes: Taking the lower left corner of the enhanced heat map as the origin of the pixel map coordinates, taking the horizontal index of the pixels in the enhanced heat map as the horizontal coordinate value of the pixel map coordinates, taking the vertical index of the pixels in the enhanced heat map as the vertical coordinate value of the pixel map coordinates, map each pixel in the enhanced heat map to the pixel map coordinates; calculate the pixel gradient of the pixels in the enhanced heat map based on the map coordinates to obtain the pixel gradient value; preset a weld point selection window, and based on the weld point selection window, move the weld point selection window without coverage starting from the upper left corner of the enhanced heat map. Denote the pixels within the weld point selection window as window pixels, and calculate the gradient marking threshold and the pixel marking threshold for the window pixels after each non-overlapping movement; preset an initial solder joint set, initialize the initial solder joint set to be empty, and perform double screening on the window pixels based on the gradient marking threshold and the pixel marking threshold. Include the window pixels whose pixel gradient value is greater than or equal to the gradient calibration threshold and whose pixel value is greater than or equal to the pixel marking threshold in the initial solder joint set; Perform binary mapping on the pixel map coordinates based on the pixels in the initial solder joint set, mark the positions corresponding to the pixels in the initial solder joint set in the pixel map coordinates as value one, and mark the remaining positions in the pixel map coordinates as zero to obtain a binary mapping map. Preset an expansion window, and perform window erosion on each pixel in the binary mapping map based on the expansion window to obtain an eroded binary map. The method of performing window erosion includes: for a selected pixel, based on the expansion window, taking the selected pixel as the erosion center of the expansion window, and the other pixels framed in the expansion window as the erosion edges. When the value of any erosion edge is zero, the value of the erosion center is zero; when the values of all erosion edges are one, the value of the erosion center is one; perform window dilation on each pixel in the eroded binary map based on the expansion window to obtain a dilated binary map. The method of performing window dilation includes: for a selected pixel, based on the expansion window, taking the selected pixel as the dilation center of the expansion window, and the other pixels framed in the expansion window as the dilation edges. When the value of any dilation edge is one, the value of the pixel center is one; when the values of all window edges are zero, the value of the pixel center is zero; based on the dilated binary map, segment the enhanced heat map with the area where the value is one most densely in the dilated binary map to obtain a weld area map.

[0011] Further, the method of extracting features from the weld area map includes: Perform a linear temperature conversion on the weld area diagram to obtain the weld temperature diagram. Based on the weld temperature diagram, use the derivative formula to calculate the temperature gradient of each pixel in the weld temperature diagram. Preset a sparse grid. Based on the sparse grid, use the pixel in the upper left corner of the weld temperature diagram as the starting feature point, and use the sparse grid as the selection span to select grid points. All the selected grid points and the starting feature point form a set of feature points. Use the four-neighborhood as the neighborhood selection window to calculate the saliency score for each pixel in the set of feature points to obtain the saliency score. Preset a saliency threshold, and use the pixels in the set of feature points whose saliency scores are greater than or equal to the saliency threshold as the retained feature points. Generate a temperature description set for each retained feature point based on the eight-neighborhood. The temperature description set includes the average temperature within the eight-neighborhood, the mean of the temperature gradients within the eight-neighborhood, the standard deviation of the temperature gradients within the eight-neighborhood, and the ratio of the maximum temperature gradient to the minimum temperature gradient within the eight-neighborhood. Calculate the feature similarity between different retained feature points using the cosine similarity calculation formula based on the temperature description set of each retained feature point. Preset a feature similarity threshold, and include the retained feature points whose feature similarities are greater than or equal to the feature similarity threshold in the same feature merging set. Merge the features of the pixels within each feature merging set to obtain the merged features. All the merged features form the weld features.

[0012] Further, the formula for calculating the saliency score is: ; where represents the saliency score of the th pixel in the set of feature points, represents the neighborhood selection window, represents the number of pixels in the neighborhood selection window, represents the pixel temperature of the th pixel in the set of feature points, represents the pixel temperature of the th pixel in the neighborhood selection window, represents the temperature gradient of the th pixel in the neighborhood selection window; The formula for feature merging is: ; where represents the merged feature, represents the number of pixels in the feature merging set, represents the feature merging set, represents the feature similarity between the th pixel and the th pixel in the feature merging set, represents the feature of the th pixel in the feature merging set, represents the feature of the th pixel in the feature merging set.

[0013] Furthermore, the method for constructing the weld defect detection model includes: Using weld features and defect labels as the training sample set, taking the GAN model as the initial model, training the GAN model with the training sample set, using the weld features as the input data of the weld defect detection model, and using the predicted defect category as the output data of the weld defect detection model; taking minimizing the error between the actual defect label and the predicted defect category as the training objective, taking the recall rate function as the loss function of the weld defect detection model, and stopping the training to obtain the weld defect detection model when the loss function converges.

[0014] The present invention provides a weld defect detection method based on infrared thermal imaging and deep learning, including: S1. Collecting weld thermal imaging data, performing enhanced preprocessing on the weld thermal imaging data to obtain an enhanced heat map; S2. Performing adaptive segmentation on the enhanced heat map to obtain a weld area map; S3. Extracting sparse features from the weld area map to obtain weld features; S4. Constructing a weld defect detection model based on weld defect features and defect labels, and performing defect detection on the weld based on the constructed weld defect detection model to obtain accurate detection results.

[0015] Technical effects and advantages of the weld defect detection system and method based on infrared thermal imaging and deep learning of the present invention: By performing enhanced preprocessing on the weld thermal imaging data, the present invention significantly improves the clarity of the real defect area while maintaining the stability of the non-defect area, thereby effectively reducing the interference of pseudo-defects, eliminating the boundary abruptness phenomenon caused by enhancement operations in traditional methods, and ensuring the overall visual continuity of the weld thermal map; by combining the dual-threshold screening of pixel gradient and gray value, the high accuracy of the initial solder joint set is ensured, effectively avoiding the pseudo-weld area caused by high reflection or surface defects; by dynamically controlling the value rules of the corrosion center and the expansion edge, the edge structure of the binary image is optimized, making the weld area segmentation result more accurate, and adopting a dynamic expansion window, which can adjust the operation intensity according to the actual complexity of the weld area, improving the adaptability to complex weld areas; by using the sparse sampling algorithm, the computational complexity is effectively reduced, while retaining the key temperature feature points, and combining the significant scoring mechanisms of the four-neighborhood and eight-neighborhood, the representativeness of the feature points is ensured, improving the accuracy of feature point selection, and combining the temperature gradient and the statistical description set, providing multi-angle weld defect information and enhancing the discrimination ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1Schematic diagram of the weld defect detection system based on infrared thermal imaging and deep learning of the present invention; Figure 2 Schematic diagram of the weld defect detection method based on infrared thermal imaging and deep learning of the present invention. Specific implementation manners

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0018] Embodiment 1 Please refer to Figure 1 As shown, the weld defect detection system based on infrared thermal imaging and deep learning in this embodiment includes: Data acquisition and processing module: Collect weld thermal imaging data, perform enhanced preprocessing on the weld thermal imaging data, and obtain an enhanced heat map; Adaptive segmentation module: Perform adaptive segmentation on the enhanced heat map to obtain a weld area map; Defect feature extraction module: Perform sparse feature extraction on the weld area map to obtain weld features; Model construction module: Construct a weld defect detection model based on weld defect features and defect labels, and perform defect detection on the weld based on the constructed weld defect detection model to obtain accurate detection results; Each module is connected by wired and / or wireless means to achieve data transmission between modules; The weld thermal imaging data includes a weld heat map and defect labels. The weld heat map is collected by a high-sensitivity infrared thermal imaging sensor; the defect label is an identifier of the defect formed during the welding process. Common defect labels include porosity, slag inclusion, incomplete penetration, lack of fusion, crack, pit, and undercut.

[0019] The ways to perform enhanced preprocessing on the weld thermal imaging data include: Preset the block size based on the size of the weld heat map in the weld thermal imaging data. Common block sizes are pixels, Pixels, the size of the divided block is determined based on the resolution of the weld heat map. Both the length and width of the divided block meet the divisibility condition, that is, the length of the divided block can be divided evenly by the long side of the resolution of the weld heat map, and the width of the divided block can be divided evenly by the short side of the resolution of the weld heat map. The weld heat map is evenly divided based on the divided block size to obtain heat map sub - blocks. Each heat map sub - block is incrementally numbered in the order from left to right and from top to bottom based on its corresponding position in the weld heat map to obtain the sub - block number of each heat map sub - block; taking the mean of the gray - scale values of the pixels in each heat map sub - block as the initial gray - scale center, the average attribution calculation is performed on each pixel in each heat map sub - block based on the initial gray - scale center. The formula for the average attribution calculation is: ; where, represents the average attribution degree, represents the number of initial gray - scale centers, represents the th initial gray - scale center, represents the attribution width, which is used to control the sensitivity of the input data; Taking the mean of the average attribution degrees as the comparison threshold, and taking the standard deviation of the gray - scale values of the pixels in each heat map sub - block as the sub - block contrast. Based on the comparison threshold, the heat map sub - blocks with sub - block contrast greater than or equal to the comparison threshold are marked as high - definition sub - blocks, and the heat map sub - blocks with sub - block contrast less than the comparison threshold are marked as sub - blocks to be enhanced. The contrast enhancement for the sub - blocks to be enhanced includes: performing a histogram distribution statistics on each sub - block to be enhanced. The formula for the histogram distribution statistics is: ; where, represents the number of pixels belonging to the th gray - scale level, represents the gray - scale value of the th pixel in the sub - block to be enhanced, represents the gray - scale level, and the gray - scale level satisfies the sub - block range condition: , represents the set of gray - scale values of the pixels that appear in the sub - block to be enhanced, represents the number of pixels in the sub - block to be enhanced, represents the indicator function. When the gray - scale value of the th pixel in the sub - block to be enhanced is equal to the gray - scale value of the gray - scale level , the function value is 1. When the gray - scale value of the th pixel in the sub - block to be enhanced is not equal to the gray - scale value of the gray - scale level When the gray value is 0, the function value is 0; a gray histogram is constructed based on the statistical result of the histogram distribution. The gray histogram is a columnar statistical chart, with the gray value of the pixels in the sub-block to be enhanced as the abscissa and the number of pixels with the corresponding gray value on the abscissa as the ordinate. The gray histogram shows the number of occurrences of gray values at each level and intuitively reflects the quality of the image. If the histogram is concentrated in the low gray level, it means the image is too dark; if it is concentrated in the high gray level, it means the image is too bright; Based on the gray histogram, the gray value of each pixel in the sub-block to be enhanced is subjected to enhanced mapping. The formula for enhanced mapping is: ; where represents the enhanced gray value, represents the original gray value of the pixel, represents the gray value corresponding to the minimum value of the column in the gray histogram, represents the number of pixels in the sub-block to be enhanced, the minimum value of the column in the gray histogram, represents the total number of gray levels, represents the selection function. In this embodiment, the selection function is preferably the rounding function; the sub-block to be enhanced after enhanced mapping is denoted as the enhanced sub-block; the enhanced sub-block and the high-definition sub-block are sequentially spliced based on the sub-block number to obtain the enhanced heat map. The pixels at the splicing edge of the enhanced sub-block and the high-definition sub-block are denoted as boundary pixels, and the boundary pixels are smoothed to reduce the contrast difference at the boundary between the enhanced sub-block and the high-definition sub-block, so that the boundary smoothly transitions; the formula for boundary smoothing is: ; where represents the smoothed pixel value of the th boundary pixel, represents the pixel value of the th boundary pixel, represents pi, represents the smoothing factor, which is used to control the smoothing degree of the boundary pixels, represents the th boundary pixel, represents the set of boundary pixels, represents the th boundary pixel and the th boundary pixel distance, and , and the commonly used distance measurement formulas include the Euclidean distance formula and the Chebyshev distance formula; in weld detection, due to factors such as noise or surface irregularities, pseudo-defects are likely to appear. By performing enhanced mapping on the pixels in the local area of the weld heat map, the contrast of real defects is increased, while the contrast of non-defect areas remains stable, so that real defects are more easily distinguishable in the image and the detection rate of pseudo-defects is reduced.

[0020] The methods for adaptively segmenting the enhanced heat map include: Taking the lower left corner of the enhanced heat map as the origin of the pixel map coordinates, taking the horizontal index of the pixels in the enhanced heat map as the horizontal coordinate value of the pixel map coordinates, taking the vertical index of the pixels in the enhanced heat map as the vertical coordinate value of the pixel map coordinates, and mapping each pixel in the enhanced heat map to the pixel map coordinates; calculating the pixel gradient of the pixels in the enhanced heat map based on the map coordinates, and the formula for calculating the pixel gradient is: ; where represents the pixel gradient value of the pixel at the horizontal axis value of and the vertical axis value of in the pixel map coordinates, represents the derivative of the pixel in the horizontal axis in the pixel map coordinates, represents the derivative of the pixel in the vertical axis in the pixel map coordinates; preset a weld point selection window, and move the weld point selection window without coverage (that is, the pixels in the moved weld point selection window do not overlap with the pixels in the previous weld point selection window) with the upper left corner of the enhanced heat map as the starting point based on the weld point selection window, record the pixels in the weld point selection window as window pixels, calculate the gradient marking threshold and the pixel marking threshold for the window pixels after each non-overlapping movement, and the formula for the gradient marking threshold is: ; where represents the gradient calibration threshold, represents the mean value of the pixel gradient values of the window pixels, represents the standard deviation of the pixel gradient values of the window pixels, represents the gradient adjustment factor used to control the sensitivity of the gradient marking threshold, and the formula for the pixel marking threshold is: ; where represents the pixel marking threshold, represents the mean value of the gray values of the window pixels, represents the standard deviation of the gray values of the window pixels, represents the pixel adjustment factor used to control the sensitivity of the pixel marking threshold; preset an initial solder joint set, initialize the initial solder joint set to be empty, perform double screening on the window pixels based on the gradient marking threshold and the pixel marking threshold, and include the window pixels whose pixel gradient values are greater than or equal to the gradient calibration threshold and whose pixel values are greater than or equal to the pixel marking threshold in the initial solder joint set; in this embodiment, the gradient adjustment factor is preferably 0.6, and the pixel adjustment factor is preferably 0.9; Perform binary mapping on the pixel map coordinates based on the pixels in the initial solder joint set. Mark the positions corresponding to the pixels in the initial solder joint set in the pixel map coordinates as value one, and mark the remaining positions in the pixel map coordinates as zero to obtain a binary mapping map. Preset an expansion window, and perform window erosion on each pixel in the binary mapping map based on the expansion window to obtain an eroded binary map. The method of performing window erosion includes: for a selected pixel, based on the expansion window, use the selected pixel as the erosion center of the expansion window, and use the other pixels framed in the expansion window as erosion edges. When the value of any erosion edge is zero, the value of the erosion center is zero; when the values of all erosion edges are one, the value of the erosion center is one. Perform window dilation on each pixel in the eroded binary map based on the expansion window to obtain a dilated binary map. The method of performing window dilation includes: for a selected pixel, based on the expansion window, use the selected pixel as the dilation center of the expansion window, and use the other pixels framed in the expansion window as dilation edges. When the value of any dilation edge is one, the value of the pixel center is one; when the values of all window edges are zero, the value of the pixel center is zero. Based on the dilated binary map, segment the enhanced heat map with the area where the value is one most densely in the dilated binary map to obtain a weld region map. Assume the binary mapping map is: , the eroded binary map after performing window erosion is: , the dilated binary map after performing window dilation is: .

[0021] The method of feature extraction for the weld region map includes: Perform linear temperature conversion on the weld region map to obtain a weld temperature map. Calculate the temperature gradient of each pixel in the weld temperature map based on the weld temperature map using a derivative formula. Preset a sparse grid. Based on the sparse grid, use the pixel in the upper left corner of the weld temperature map as the starting feature point, and use the sparse grid as the selection span to select grid points. All the selected grid points and the starting feature point form a feature point set. Calculate the significance score for each pixel in the feature point set using a four-neighborhood as the neighborhood selection window. The formula for calculating the significance score is: ; where, represents the significance score of the th pixel in the feature point set, represents the neighborhood selection window, represents the number of pixels in the neighborhood selection window, represents the pixel temperature of the th pixel in the feature point set, represents the pixel temperature of the th pixel in the neighborhood selection window, represents the The temperature gradient of a pixel; a preset significant threshold, and pixels with a significant score greater than or equal to the significant threshold in the set of feature points are used as retained feature points; based on the eight-neighborhood, a temperature description set is generated for each retained feature point, and the temperature description set includes the average temperature within the eight-neighborhood, the mean of the temperature gradients within the eight-neighborhood, the standard deviation of the temperature gradients within the eight-neighborhood, and the ratio of the maximum temperature gradient to the minimum temperature gradient within the eight-neighborhood; based on the temperature description sets of each retained feature point, the cosine similarity calculation formula is used to calculate the feature similarity between different retained feature points. A preset feature similarity threshold is set, and retained feature points with a feature similarity greater than or equal to the feature similarity threshold are included in the same feature merging set. Feature merging is performed on the pixels within each feature merging set, and the formula for feature merging is: ; where represents the merged feature, represents the number of pixels within the feature merging set, represents the feature merging set, represents the th pixel and the th pixel in the feature merging set, represents the feature of the th pixel in the feature merging set, represents the feature of the th pixel in the feature merging set. All merged features constitute the weld feature; the four-neighborhood refers to the pixels corresponding to the upper, lower, left, and right positions respectively, and the eight-neighborhood refers to the pixels corresponding to the upper, lower, left, right, upper-left, lower-left, upper-right, and lower-right positions respectively; for linear temperature conversion, first obtain the pixel value range of the weld area map, which is usually from 0 to 255 or a larger range. These pixel values represent the brightness or color intensity of the image. According to the device calibration, determine the actual temperature range represented in the image, and the feature of a pixel is the temperature corresponding to the pixel in the weld temperature map.

[0022] The construction method of the weld defect detection model includes: Using the weld feature and the defect label as the training sample set, using the GAN model as the initial model, training the GAN model with the training sample set, using the weld feature as the input data of the weld defect detection model, and using the predicted defect category as the output data of the weld defect detection model; using the minimization of the error between the actual defect label and the predicted defect category as the training objective, using the recall rate function as the loss function of the weld defect detection model. When the loss function converges, stop training to obtain the weld defect detection model, and perform defect detection on the weld heat map data based on the weld defect detection model to obtain accurate detection results.

[0023] In this embodiment, by performing enhanced preprocessing on the weld thermal imaging data, the clarity of the real defect area is significantly improved, while the stability of the non-defect area is maintained, thus effectively reducing the interference of pseudo-defects and eliminating the boundary abruptness phenomenon caused by the enhancement operation in the traditional method, ensuring the overall visual continuity of the weld thermal image; by combining the dual threshold screening of pixel gradient and gray value, the high accuracy of the initial solder joint set is ensured, effectively avoiding the pseudo-weld area caused by high reflection or surface defects; by dynamically controlling the value-taking rules of the corrosion center and the expansion edge, the edge structure of the binary image is optimized, making the weld area segmentation result more accurate. The dynamic expansion window can adjust the operation intensity according to the actual complexity of the weld area, improving the adaptability to complex weld areas; the sparse sampling algorithm effectively reduces the computational complexity while retaining the key temperature feature points. Combining the significant scoring mechanism of the four-neighborhood and eight-neighborhood ensures the representativeness of the feature points and improves the accuracy of feature point selection. Combining the temperature gradient and the statistical description set provides multi-angle weld defect information and improves the discrimination ability of the model.

[0024] Embodiment 2 Please refer to Figure 2 As shown, for the parts not described in detail in this embodiment, refer to the description content of Embodiment 1. A weld defect detection method based on infrared thermal imaging and deep learning is provided, including: S1. Collect the weld thermal imaging data and perform enhanced preprocessing on the weld thermal imaging data to obtain an enhanced heat map; S2. Perform adaptive segmentation on the enhanced heat map to obtain a weld area map; S3. Perform sparse feature extraction on the weld area map to obtain weld features; S4. Build a weld defect detection model based on the weld defect features and defect labels, and perform defect detection on the weld based on the built weld defect detection model to obtain accurate detection results.

[0025] Embodiment 3 This embodiment publicly provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it realizes the operation mode of the above-provided weld defect detection method based on infrared thermal imaging and deep learning.

[0026] Since the electronic device introduced in this embodiment is the electronic device used to implement the weld defect detection method based on infrared thermal imaging and deep learning in the embodiments of the present application, based on the weld defect detection method based on infrared thermal imaging and deep learning introduced in the embodiments of the present application, those skilled in the art can understand the specific implementation manners and various variations of the electronic device in this embodiment. Therefore, the specific implementation of how this electronic device implements the method in the embodiments of the present application will not be described in detail here. As long as those skilled in the art implement the electronic device used in the weld defect detection method based on infrared thermal imaging and deep learning in the embodiments of the present application, it falls within the scope of protection of the present application.

[0027] The above formulas are all dimensionless and take their numerical calculations. The formula is a formula obtained by collecting a large amount of data for software simulation to get the closest to the real situation. The preset parameters and threshold selection in the formula are set by those skilled in the art according to the actual situation.

[0028] The above is only the preferred implementation manner of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for ordinary technical users in the technical field, several improvements and refinements made without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for detecting weld defects based on infrared thermal imaging and deep learning, characterized in that Including: S1. Collect the thermal imaging data of the weld seam, perform enhanced preprocessing on the thermal imaging data of the weld seam, and obtain an enhanced heat map; S2. Perform adaptive segmentation on the enhanced heat map to obtain a weld seam area map; S3. Extract sparse features from the weld seam area map to obtain weld seam features; S4. Construct a weld seam defect detection model based on the weld seam defect features and defect labels, and perform defect detection on the weld seam based on the constructed weld seam defect detection model to obtain accurate detection results; The method for performing adaptive segmentation on the enhanced heat map includes: Taking the lower left corner of the enhanced heat map as the origin of the pixel map coordinates, taking the horizontal index of the pixels in the enhanced heat map as the horizontal coordinate value of the pixel map coordinates, taking the vertical index of the pixels in the enhanced heat map as the vertical coordinate value of the pixel map coordinates, and mapping each pixel in the enhanced heat map to the pixel map coordinates; calculating the pixel gradient of the pixels in the enhanced heat map based on the map coordinates to obtain the pixel gradient value; presetting a weld point selection window, and performing non-overlapping movement on the weld point selection window with the upper left corner of the enhanced heat map as the starting point based on the weld point selection window, and recording the pixels within the weld point selection window as window pixels, and calculating the gradient marking threshold and pixel marking threshold for the window pixels after each non-overlapping movement; presetting an initial weld point set, initializing the initial weld point set to be empty, and performing double screening on the window pixels based on the gradient marking threshold and pixel marking threshold, and including the window pixels whose pixel gradient value is greater than or equal to the gradient calibration threshold and whose pixel value is greater than or equal to the pixel marking threshold in the initial weld point set; Performing binary mapping on the pixel map coordinates based on the pixels in the initial weld point set, marking the positions corresponding to the pixels in the initial weld point set in the pixel map coordinates as value one, and marking the remaining positions in the pixel map coordinates as zero to obtain a binary mapping map, presetting an expansion window, and performing window erosion on each pixel in the binary mapping map based on the expansion window to obtain an eroded binary map. The method for performing window erosion includes: for the selected pixel, based on the expansion window, taking the selected pixel as the erosion center of the expansion window, and the other pixels framed in the expansion window as the erosion edges. When the value of any erosion edge is zero, the value of the erosion center is zero. When the values of all erosion edges are one, the value of the erosion center is one; performing window dilation on each pixel in the eroded binary map based on the expansion window to obtain a dilated binary map. The method for performing window dilation includes: for the selected pixel, based on the expansion window, taking the selected pixel as the dilation center of the expansion window, and the other pixels framed in the expansion window as the dilation edges. When the value of any dilation edge is one, the value of the pixel center is one. When the values of all window edges are zero, the value of the pixel center is zero; based on the dilated binary map, segmenting the enhanced heat map with the area where the value is one most densely in the dilated binary map to obtain a weld seam area map.

2. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 1, characterized in that The thermal imaging data of the weld seam includes a weld seam heat map and defect labels.

3. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 2, characterized in that The method for performing enhanced preprocessing on the thermal imaging data of the weld seam includes: Preset the block size based on the size of the weld heat map in the weld thermal imaging data. Equalize the weld heat map based on the block size to obtain heat map sub-blocks. Each heat map sub-block is incrementally numbered in the order from left to right and from top to bottom based on its corresponding position in the weld heat map to obtain the sub-block number of each heat map sub-block. Use the mean of the gray values of the pixels in each heat map sub-block as the initial gray center, and calculate the average attribution of each pixel in each heat map sub-block based on the initial gray center to obtain the average attribution degree. Use the mean of the average attribution degrees as the comparison threshold, and use the standard deviation of the gray values of the pixels in each heat map sub-block as the sub-block contrast. Based on the comparison threshold, mark the heat map sub-blocks with sub-block contrast greater than or equal to the comparison threshold as high-definition sub-blocks, and mark the heat map sub-blocks with sub-block contrast less than the comparison threshold as sub-blocks to be enhanced. Perform contrast enhancement on the sub-blocks to be enhanced to obtain enhanced sub-blocks. Sequentially splice the enhanced sub-blocks and the high-definition sub-blocks based on the sub-block numbers to obtain an enhanced heat map. Mark the pixels at the splicing edges of the enhanced sub-blocks and the high-definition sub-blocks as boundary pixels, and perform boundary smoothing on the boundary pixels. The formula for boundary smoothing is: ; where represents the smoothed pixel value of the th boundary pixel, represents the pixel value of the th boundary pixel, represents pi, represents the smoothing factor, used to control the smoothing degree of the boundary pixels, represents the pixel value of the th boundary pixel, represents the set of boundary pixels, represents the th boundary pixel and the distance between the th boundary pixel, and .

4. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 3, wherein The formula for performing average attribution calculation is: ; among them, represents the average membership degree, represents the number of initial gray centers, represents the th initial gray center, represents the membership width.

5. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 4, wherein The method for performing contrast enhancement on the sub-block to be enhanced includes: Histogram distribution statistics are performed on each sub-block to be enhanced, and the formula for histogram distribution statistics is as follows: ; wherein, represents the number of pixels belonging to the th gray level, represents the gray value of the th pixel in the sub-block to be enhanced, represents the gray level, and the gray level satisfies the sub-block range condition: , represents the set of gray values of the pixels appearing in the sub-block to be enhanced, represents the number of pixels in the sub-block to be enhanced, represents the indicator function; construct a gray histogram based on the histogram distribution statistics, and perform enhanced mapping on the gray value of each pixel in the sub-block to be enhanced. The formula for performing enhanced mapping is: ; wherein, represents the enhanced gray value, represents the original gray value of the pixel, represents the gray value corresponding to the minimum value of the column in the gray histogram, represents the number of pixels in the sub-block to be enhanced, the minimum value of the column in the gray histogram, represents the total number of gray levels, represents the selection function, and the sub-block to be enhanced after performing enhanced mapping is denoted as the enhanced sub-block.

6. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 5, wherein The method for extracting sparse features from the weld region image includes: Perform linear temperature conversion on the weld region image to obtain the weld temperature image. Based on the weld temperature image, use the derivative formula to calculate the temperature gradient of each pixel in the weld temperature image. Preset a sparse grid. Based on the sparse grid, use the pixel in the upper left corner of the weld temperature image as the starting feature point, and use the sparse grid as the selection span to select grid points. All the selected grid points and the starting feature point form a set of feature points. Use the four-neighborhood as the neighborhood selection window to calculate the saliency score for each pixel in the set of feature points to obtain the saliency score. Preset a saliency threshold, and use the pixels in the set of feature points whose saliency score is greater than or equal to the saliency threshold as the retained feature points. Generate a temperature descriptor set for each retained feature point based on the eight-neighborhood. The temperature descriptor set includes the average temperature within the eight-neighborhood, the mean of the temperature gradients within the eight-neighborhood, the standard deviation of the temperature gradients within the eight-neighborhood, and the ratio of the maximum temperature gradient to the minimum temperature gradient within the eight-neighborhood. Calculate the feature similarity between different retained feature points using the cosine similarity calculation formula based on the temperature descriptor set of each retained feature point. Preset a feature similarity threshold, and include the retained feature points whose feature similarity is greater than or equal to the feature similarity threshold in the same feature merging set. Merge the features of the pixels within each feature merging set to obtain the merged features. All the merged features form the weld features.

7. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 6, wherein The formula for calculating the saliency score is as follows: ; wherein, represents the significance score of the th pixel in the set of feature points, represents the neighborhood selection window, represents the number of pixels in the neighborhood selection window, represents the pixel temperature of the th pixel in the set of feature points, represents the pixel temperature of the th pixel in the neighborhood selection window, represents the temperature gradient of the th pixel in the neighborhood selection window; The formula for feature merging is as follows: ; where represents the merged feature, represents the number of pixels in the feature merging set, represents the feature merging set, represents the th pixel and the th pixel in the feature merging set, represents the feature similarity between the th pixel and the th pixel in the feature merging set, represents the feature of the th pixel in the feature merging set.

8. The weld defect detection method based on infrared thermal imaging and deep learning according to claim 7, characterized in that The method for constructing the weld defect detection model includes: Use the weld features and defect labels as the training sample set, use the GAN model as the initial model, and use the training sample set to train the GAN model. Use the weld features as the input data of the weld defect detection model, and use the predicted defect category as the output data of the weld defect detection model. Use minimizing the error between the actual defect label and the predicted defect category as the training objective, and use the recall rate function as the loss function of the weld defect detection model. When the loss function converges, stop training to obtain the weld defect detection model.

9. A weld defect detection system based on infrared thermal imaging and deep learning, which is used to implement the weld defect detection method based on infrared thermal imaging and deep learning described in any one of claims 1 to 8, characterized in that, It includes: Data acquisition and processing module: Collect weld thermal imaging data, perform enhanced preprocessing on the weld thermal imaging data to obtain an enhanced heat map; Adaptive segmentation module: Perform adaptive segmentation on the enhanced heat map to obtain the weld region image; Defect feature extraction module: Extract sparse features from the weld region image to obtain the weld features; Model construction module: Construct a weld defect detection model based on the weld defect features and defect labels, and perform defect detection on the weld based on the constructed weld defect detection model to obtain accurate detection results.

Images