Fin welding intelligent detection method and system
By combining vibrational thermal excitation and texture-enhanced attention network with a differentiable physical model, high-precision identification and physical quantification of welding defects in heat sinks are achieved, solving the problem of unreliable detection results in existing technologies and improving the accuracy and reliability of detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU JUNKAI POWER EQUIP CO LTD
- Filing Date
- 2026-03-18
- Publication Date
- 2026-06-16
AI Technical Summary
Existing infrared thermal imaging-based heat sink welding inspection methods are difficult to effectively identify minute defects in complex three-dimensional structures, have low signal-to-noise ratios, cannot achieve physical quantitative assessment of welding status, and have limited reliability and accuracy of inspection results.
By actively stimulating the thermal response of defects through vibration thermal excitation, and combining a texture-enhanced attention network and a differentiable physical model, a target detection network with a deep fusion texture attention mechanism is adopted to output physical quantification parameters such as weld rate, thus achieving a leap from image recognition to physical quantification.
It significantly improves the accuracy and robustness of defect detection, can accurately identify minute defects, and output welding status parameters with clear physical meaning, thereby improving the reliability of the detection results and their engineering guidance value.
Smart Images
Figure CN122222972A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of nondestructive testing and image processing technology, specifically relating to an intelligent inspection method and system for heat sink welding. Background Technology
[0002] In the manufacturing of power electronic equipment, the reliability of heat sinks is crucial, and its core lies in the welding quality between the substrate and the heat sink fins. Traditional inspection methods mainly rely on manual visual inspection, penetrant testing, or radiographic testing, which have limitations such as low efficiency, high subjectivity, inability to quantify internal defects, and hazards to personnel. With technological advancements, the path of non-destructive testing technology has gradually evolved: from relying on a single physical principle (such as ultrasound or eddy current) to the integration of multiple technologies; from manual interpretation to automated image analysis, especially with the introduction of infrared thermal imaging technology into welding inspection due to its advantages of non-contact and full-field observation.
[0003] However, existing infrared thermal imaging-based methods still face significant problems: for objects with complex three-dimensional structures, such as heat sink welds, passive imaging is easily affected by environmental and surface conditions, resulting in a low signal-to-noise ratio; it is insensitive to minor defects such as incomplete welds and internal lack of fusion; conventional image processing or traditional machine learning algorithms struggle to stably and accurately extract defect features from complex infrared images, and cannot achieve physical quantitative assessment of the weld status (such as weld ratio and thermal resistance), thus limiting the reliability, accuracy, and engineering guidance value of the detection results. Summary of the Invention
[0004] To address the aforementioned problems in the existing technology, this invention provides an intelligent detection method and system for heat sink welding. It actively excites the thermal response of defects through vibration thermal excitation, accurately identifies minute defects using a texture-enhanced attention network, and combines differentiable physical model inversion to output physical quantification parameters such as welding rate, thus achieving a leap from image recognition to physical quantification.
[0005] The objective of this invention can be achieved through the following technical solutions: The first aspect of this disclosure provides a smart inspection method for heat sink soldering, including the following steps: S1. Synchronous vibration thermal imaging signal acquisition: Through hardware-synchronized vibration excitation and infrared thermal imaging acquisition, the surface thermal sequence of the heat sink under continuous sinusoidal vibration is obtained, and multi-channel feature maps are generated through preprocessing and feature extraction. S2. Welding Defect Recognition: After standardizing and unifying the size of the multi-channel feature map, it is input into the target detection network with deep fusion texture attention mechanism, and the output is the defect area of the weld. S3. Quantitative output of welding status: Construct a differentiable forward heat conduction model as a physical information constraint layer, use inversion optimization to obtain physical estimates of defect parameters, and finally output welding status parameters; The architecture of the target detection network with the deep fusion texture attention mechanism includes, in sequence, an input layer, a feature extraction backbone network, a texture enhancement attention module, a multi-scale feature fusion neck network, an attention refinement prediction head, and an output layer. The texture enhancement attention module is connected to the end of the feature extraction backbone network. It is used to explicitly enhance the weak texture information related to defects in the feature map and adaptively allocate feature weights.
[0006] Furthermore, the synchronous vibration thermal imaging signal acquisition includes the following steps: S11. System setup: Fix the heat sink test piece to be tested on the vibration exciter, and set up an infrared thermal imager directly above or diagonally above the test piece. The vibration exciter and the infrared thermal imager are synchronously triggered by the hardware. S12. Thermal sequence acquisition: Control the vibration exciter to apply sinusoidal mechanical vibration of a predetermined frequency to the heat sink substrate, synchronously trigger the infrared thermal imager, set the sampling frame rate to continuously record the temperature field changes on the heat sink surface, and obtain thermal sequence data. S13. Feature map generation: The hot sequence data is preprocessed, and multi-channel feature maps are synthesized by calculating the statistical features of the time dimension of the preprocessed hot sequence data.
[0007] Furthermore, the preprocessing of the thermal sequence data includes time alignment and bad pixel correction, background subtraction to extract the pure temperature rise response, and signal enhancement using phase-locked loop processing to obtain a phase map; The multi-channel feature map includes a maximum temperature rise map, a arrival time map, a phase map, and a logarithmic decay rate map.
[0008] Furthermore, the input layer is used to receive the input tensor after the multi-channel feature map has been standardized and size normalized; the feature extraction backbone network is connected after the input layer and is used to perform deep spatial feature encoding on the input tensor; The texture enhancement attention module includes a texture extraction unit and a dual attention weight generation unit connected in sequence. The texture extraction unit uses a set of Gabor filter banks with different parameters and orientations to perform parallel convolution operations on the input feature map, extracting texture components in different directions. The dual attention weight generation unit receives the fusion result of the texture components and performs channel attention operations and spatial attention operations in parallel. The channel attention operation generates a channel-dimensional weight vector through a squeezing-excitation mechanism, and the spatial attention operation generates a spatial weight matrix by calculating the nonlocal correlations between spatial locations in the feature map. Finally, the channel-dimensional weight vector and the spatial weight matrix are combined to reweight the original feature map, outputting a texture-enhanced feature map.
[0009] Furthermore, the multi-scale feature fusion neck network, connected after the texture enhancement attention module, is used to fuse the semantic and detail information of the primary feature maps and texture enhancement feature maps from different levels of the feature extraction backbone network, and output the fused feature map. Attention-refined prediction head: includes multiple prediction branches that correspond one-to-one with the output scale of the neck network. Each prediction branch contains a convolutional block attention module and a detection head. The convolutional block attention module is used to perform channel attention and spatial attention operations on the input feature map in sequence to achieve feature selection before prediction. The detection head is connected after the convolutional block attention module and adopts a decoupled structure. It regresses the spatial location of the defect target, the confidence of the target's existence, and the defect category through independent convolutional layers. The output layer is used to summarize and output the results of all prediction branches, forming the final detection result containing bounding box coordinates, confidence scores, and categories.
[0010] Furthermore, the welding defect identification also includes the following steps: S23. Model Training: The network parameters are optimized using a composite loss function. The parameters of the feature extraction backbone network are initialized through a transfer learning strategy. During the training phase, data augmentation operations are applied to the input multi-channel feature maps. The optimization strategy uses the AdamW optimizer to perform iterative updates of the network parameters. The composite loss function consists of a weighted sum of bounding box regression loss, target confidence loss, and classification loss.
[0011] Furthermore, the quantitative output of the welding status includes the following steps: S31. Construct a physical information constraint layer: Establish a differentiable positive heat conduction model. The model input is the assumed defect parameters and the inherent thermal properties of the material. The output is the theoretical thermal response at the observation point on the surface of the heat sink, calculated under vibration excitation conditions. S32. Multi-source information alignment and fusion: Spatially, the coordinate systems of the infrared camera and ultrasonic probe are aligned to the coordinate system of the heat sink entity through pre-calibration; temporally, the thermal sequence data and ultrasonic scanning signal data are aligned to a unified time axis using hardware-synchronized timestamps. For each identified independent defect region, the average temperature change curve of the region during the entire vibration excitation process is extracted from the aligned thermal sequence data as the observed value of the actual thermal response; at the same time, the waveform feature vector reflecting the internal structural characteristics of the region is extracted from the aligned ultrasonic signal.
[0012] Furthermore, the quantitative output of the welding status also includes the following steps: S33. Inversion and Optimization Solution: By constructing an inversion loss function, a gradient-based optimization algorithm is used to automatically adjust the defect parameters to be inverted through iteration, continuously reducing the value of the inversion loss function until the theoretical thermal response and the actual thermal response reach the optimal match, thereby obtaining the optimal estimated value of the defect parameters. S34. Output Quantitative State Parameters: Based on the optimal estimate of the defect parameters, output state parameters, including local weld ratio, defect equivalent thermal resistance, and predicted defect depth or insufficient penetration.
[0013] The second aspect of this disclosure provides an intelligent inspection system for heat sink welding, applied to an intelligent inspection method for heat sink welding as described above, including a vibration thermal excitation and data acquisition module, a deep learning defect identification module, and a physical information fusion and quantization module; The vibration thermal excitation and data acquisition module is used to apply controlled mechanical vibration excitation to the heat sink specimen and simultaneously acquire infrared thermal radiation signals from the surface, generating multi-channel feature maps through signal processing.
[0014] As a preferred technical solution of the present invention, the deep learning defect recognition module is used to receive the multi-channel feature map generated by the vibration thermal excitation and data acquisition module, and use the target detection network with deep fusion texture attention mechanism to identify and locate the defects in the welding area of the heat sink, and output the boundary box, category and confidence of the defects. The physical information fusion and quantization module, guided by the defect region, combines the original thermal sequence data with other sensor data to perform inversion optimization under the constraints of a differentiable physical model, transforming the visual features of the defect into physical state parameters.
[0015] The beneficial effects of this invention are as follows: This invention firstly converts mechanical energy into heat energy at the defect location through hardware-synchronized vibration thermal excitation and infrared acquisition, achieving a leap from passive observation to active excitation. This significantly enhances the thermal contrast between the defect (especially internal defects) and the background, improving the signal-to-noise ratio and feature richness from the data source. Secondly, the target detection network model incorporates a texture enhancement attention module after the backbone network. It explicitly extracts multi-directional texture features through Gabor filter banks and uses adaptive weighting with channel and spatial dual attention mechanisms. This allows the network to focus on weak and low-contrast abnormal thermal texture patterns caused by defects, effectively solving the problem of small defect features being easily lost in deep networks, and greatly improving the accuracy and robustness of detection. Finally, by constructing a differentiable forward heat conduction model and establishing an inversion optimization framework, the defect region identified by the target detection network is combined with the physical response of the original thermal sequence, achieving a qualitative change from image recognition to physical parameter inversion. This outputs welding state parameters that are not only based on pixel statistics but also rooted in the physical laws of heat conduction, giving the detection results clear physical meaning and credibility. Attached Figure Description
[0016] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.
[0017] Figure 1 This is a schematic diagram illustrating the steps of an intelligent detection method for heat sink welding provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the steps of synchronous vibration thermal imaging signal acquisition provided in an embodiment of the present invention; Figure 3 A schematic diagram illustrating the steps for identifying welding defects provided in an embodiment of the present invention; Figure 4 A schematic diagram illustrating the steps for quantifying the welding status output in an embodiment of the present invention; Figure 5 This is a schematic diagram of a smart inspection system for heat sink welding provided in an embodiment of the present invention. Detailed Implementation
[0018] To further illustrate the technical means and effects of the present invention in achieving the intended purpose, the following detailed description of the specific implementation methods, structures, features and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.
[0019] Example 1 This embodiment provides an intelligent detection method for heat sink welding, such as Figure 1 As shown, it includes the following steps: S1. Synchronous Vibration Thermal Imaging Signal Acquisition: Through hardware-synchronized vibration excitation and infrared thermal imaging acquisition, the surface thermal sequence of the heat sink under continuous sinusoidal vibration is obtained, and after preprocessing and feature extraction, a multi-channel feature map is generated, such as... Figure 2 As shown, it includes the following steps: S11. System setup: Fix the heat sink test piece to be tested on a vibration exciter (such as a piezoelectric ceramic actuator or electromagnetic vibrator), and set up an infrared thermal imager directly above or diagonally above the test piece. The vibration exciter and the infrared thermal imager are synchronously triggered by hardware.
[0020] Understandably, hardware synchronous triggering involves sending the excitation waveform signal generated by the signal generator simultaneously to the power amplifier and as an external trigger signal to the infrared thermal imager. The power amplifier drives the vibration exciter, and the thermal imager begins acquiring data the instant it receives the edge of the trigger signal, ensuring that each frame of thermal image is aligned with a precise point on the excitation time axis, achieving microsecond-level time synchronization.
[0021] S12. Thermal sequence acquisition: Control the vibration exciter to apply low-power, continuous sinusoidal mechanical vibration at a predetermined frequency (e.g., 1-10kHz) to the heat sink substrate, synchronously trigger the infrared thermal imager, set the sampling frame rate (e.g., 50-100 Hz) to continuously record the temperature field changes on the heat sink surface, and obtain a set of time-series thermal images, i.e., thermal sequence data.
[0022] The sampling frame rate is set according to the Nyquist sampling theorem to be sufficient to capture the thermal response corresponding to the excitation frequency component (up to 10kHz).
[0023] Understandably, vibrational energy will propagate within the structure and be locally converted into heat energy at welding defects (such as incomplete welds or microcracks) due to friction and damping effects, forming an "internal heat source." For a predetermined frequency, the characteristic frequency most sensitive to a specific product can be determined through a pre-conducting frequency scanning experiment.
[0024] S13. Feature Map Generation: The thermal sequence data is preprocessed, including time alignment and bad pixel correction, background subtraction to extract the pure temperature rise response, and phase-locked loop (PLL) signal enhancement to obtain the phase map. From the preprocessed thermal sequence data, multi-channel feature maps (two-dimensional images) are synthesized by calculating statistical characteristics along the time dimension, including the maximum temperature rise map, arrival time map, phase map, and logarithmic decay rate. Figure 4 Aspect feature map.
[0025] It should be noted that background subtraction is performed by extracting the average frame before the excitation begins (e.g., the first 0-1 seconds) as the background reference image, and subtracting this background frame from each frame of the entire thermal sequence to obtain the temperature rise sequence. Phase-locked processing is used to enhance the signal, and phase-locked thermal imaging is performed on the temperature rise time curve of each pixel: the temperature rise time curve is correlated with the original excitation reference sinusoidal signal to extract the amplitude and phase maps with the same frequency as the excitation. Specifically, the maximum temperature rise map is visualized as a thermal response intensity distribution map, representing the maximum temperature rise at each point due to vibration excitation; the arrival time map is visualized as a thermal wave propagation isochronous plot, representing the time required for the heat wave to propagate from the excitation source to various points on the surface, which is sensitive to defect depth; the phase map is visualized as an internal structure perspective image, representing the phase delay of surface temperature change relative to the excitation source vibration, which is most sensitive to internal defects; and the logarithmic decay rate map is visualized as a heat dissipation rate map, representing the rate of temperature decay at that point after the excitation stops, reflecting local heat diffusion characteristics.
[0026] In this embodiment, the synchronous vibration thermal imaging signal acquisition introduces controlled vibration thermal excitation, causing the defect area to generate dynamic "hot spots" or "hot patches" that are distinct from the background. This active excitation method can significantly enhance the thermal contrast between the defect and the intact area, especially sensitive to unfused defects enclosed inside, effectively solving the problem of insufficient signal-to-noise ratio caused by relying on natural temperature differences. The acquired dynamic thermal sequence contains the thermal generation and diffusion process of the defect, and its spatiotemporal pattern contains richer defect feature information than a single frame image.
[0027] S2. Welding Defect Recognition: After standardizing and unifying the size of the multi-channel feature maps, they are input into a target detection network with a deep fusion texture attention mechanism, and the output is the defect region of the weld, such as... Figure 3 As shown, it includes the following steps: S21. Model Input Construction: Each independent feature channel of the multi-channel feature map is Z-score normalized channel by channel, and then stacked according to the channel dimension (3rd dimension) to form a [Batch] model. size The input tensor is [H, W, C=4].
[0028] H and W are the image space dimensions, which need to be uniformly adjusted to the network's preset input dimensions (e.g., 640x640 pixels). The adjustment process uses bilinear interpolation to preserve feature continuity.
[0029] S22. Model Architecture Design: The target detection network adopts a deep fusion texture attention mechanism, which includes, in sequence, an input layer, a feature extraction backbone network, a texture enhancement attention module, a multi-scale feature fusion neck network, an attention refinement prediction head, and an output layer. The input layer receives the input tensor after standardization and size normalization of the multi-channel feature maps. The feature extraction backbone network is connected after the input layer and is used to perform deep spatial feature encoding on the input tensor. Specifically, the CSPDarknet network with cross-stage partial connection structure is used as the backbone network, and its output is multiple primary feature maps with different receptive field scales.
[0030] Texture Enhancement Attention Module: Connected to the end of the feature extraction backbone network, this module explicitly enhances weak texture information related to defects in the feature maps and adaptively assigns feature weights. Specifically, this module includes sequentially connected texture extraction units and dual attention weight generation units.
[0031] The texture extraction unit uses a set of Gabor filters with different parameters and directions (such as 0°, 45°, 90°, 135°) to perform parallel convolution operations on the input feature map in order to extract texture components in different directions.
[0032] The dual attention weight generation unit receives the fusion result of the texture components and performs channel attention and spatial attention operations in parallel. The channel attention operation generates a channel-dimensional weight vector through a squeezing-excitation mechanism (i.e., global information aggregation through global average pooling and weight generation through a fully connected layer). The spatial attention operation generates a spatial weight matrix by calculating the nonlocal correlation between spatial locations of the feature map. Finally, the two weights are combined to reweight the original feature map and output a texture-enhanced feature map.
[0033] Multi-scale feature fusion neck network: connected after the texture enhancement attention module, it is used to efficiently fuse the semantic and detail information of primary feature maps and texture enhancement feature maps from different levels of the feature extraction backbone network.
[0034] Specifically, a bidirectional feature pyramid network (BiFPN) is used as the neck network. Through learnable weighted bidirectional connections, it repeatedly fuses multi-scale input features from top to bottom and bottom to top, outputting a series of fused feature maps with rich multi-scale contextual information.
[0035] Attention refinement prediction head: includes multiple prediction branches that correspond one-to-one with the output scale of the neck network (BiFPN). Each prediction branch contains a convolutional block attention module (CBAM) and a detection head.
[0036] The convolutional block attention module performs channel attention and spatial attention operations on the input feature map sequentially, enabling refined feature selection before prediction. The detection head, connected after CBAM, employs a decoupled structure, using independent convolutional layers to regress the spatial location of the defect target, predict the confidence level of its presence, and determine its defect category.
[0037] Output layer: Used to summarize and output the results of all prediction branches, forming the final detection result containing bounding box coordinates, confidence score, and category.
[0038] S23. Model Training: During training, a composite loss function is used to optimize the network parameters. This composite loss function consists of a weighted sum of the bounding box regression loss, the target confidence loss, and the classification loss. The expression is: L total = λ box L box + λ obj L obj + λ cls L cls ; In the formula, λ box , λ obj , λ cls These are the preset balance weight coefficients.
[0039] L box For bounding box regression loss, the Full Intersection over Union (CIU) loss function is used. This loss function comprehensively considers the overlap area between the predicted and ground truth bounding boxes, the center point distance, and aspect ratio consistency, resulting in more accurate bounding box regression. Its mathematical expression includes CIU terms, center point distance penalty terms, and aspect ratio consistency terms.
[0040] L obj For the target confidence loss, a binary cross-entropy loss function with a focus adjustment factor is adopted. This function automatically reduces the weight of easily classified background samples in the total loss by introducing an adjustable focus parameter, thereby solving the problem of the extreme imbalance in the number of pixels between defective targets and background areas in infrared thermal imaging, and making the training more focused on difficult samples.
[0041] L cls For classification loss, a multi-class cross-entropy loss function is used to measure the difference between the defect category distribution predicted by the network and the true category label.
[0042] The parameters of the feature extraction backbone network are initialized using a transfer learning strategy. Specifically, model weights pre-trained on a large public dataset (such as the COCO dataset) are loaded to initialize the feature extraction backbone network. This strategy helps the model converge quickly and improves its ability to generalize and identify welding defects in heat sinks with different characteristics.
[0043] During the training phase, a series of data augmentation operations are applied to the input multi-channel feature maps to improve the model's robustness and generalization performance. These augmentation operations are specifically designed for the characteristics of infrared thermal imaging data and include, but are not limited to: Mosaic stitching: Four training samples are randomly selected, scaled, and cropped before being stitched together into a composite image, thereby simulating multi-scale and multi-contextual training scenarios within a batch and enriching the distribution of training data.
[0044] Color space perturbation: In the HSV color space, the hue and saturation components that characterize temperature display are randomly and linearly transformed with small amplitude to simulate the temperature display changes caused by differences in environment and material surface conditions in actual testing.
[0045] Motion blur simulation: Applying linear motion blur filtering with random direction and intensity to the input image simulates the blurring effect caused by slight mechanical vibration during image acquisition, enhancing the model's robustness to such interference.
[0046] The optimization strategy uses the AdamW optimizer to perform iterative updates of network parameters. Building upon the standard Adam algorithm, the AdamW optimizer correctly decouples the weight decay term, which improves the model's generalization ability. Simultaneously, a cosine annealing strategy is employed to dynamically adjust the learning rate, causing it to decay from its initial value to its minimum value according to a cosine function curve during training. This strategy helps ensure smooth training and allows the model parameters to converge to a more optimal local minimum.
[0047] In this embodiment, welding defect recognition enhances the network's ability to perceive subtle and low-contrast thermal texture anomalies caused by defects in infrared thermal images through a pre-processed texture enhancement attention mechanism.
[0048] S3. Quantitative Output of Welding State: A differentiable forward heat conduction model is constructed as the physical information constraint layer. Inverse optimization is used to obtain physical estimates of defect parameters, and the final output welding state parameters are shown below. Figure 4 As shown, it includes the following steps: S31. Construct a physical information constraint layer: Establish a differentiable forward heat conduction model. The model input consists of assumed defect parameters (including the equivalent diameter of the defect, the burial depth, and the equivalent contact thermal resistance of the weld interface) and inherent thermal properties of the material (such as thermal conductivity, specific heat capacity, and density). The output is the theoretical temperature-time curve sequence (theoretical thermal response) at the observation point on the surface of the heat sink, calculated under vibration excitation conditions (such as excitation power, frequency, and duration).
[0049] It should be noted that, in order to achieve integration with deep learning frameworks, this forward heat conduction model must be constructed to be fully differentiable. That is, for the assumed defect parameters, it should not only be able to calculate the theoretical temperature-time curve sequence but also calculate the gradient through automatic differentiation techniques. This is typically achieved by employing differentiable numerical solutions (such as finite difference or spectral methods under the PyTorch or TensorFlow frameworks).
[0050] S32. Multi-source information alignment and fusion: Spatially, through pre-calibration, the coordinate systems of infrared cameras and ultrasonic probes (sensor data that can be collected synchronously, such as the preliminary shape of defects obtained by high-frequency ultrasonic scanning) are aligned to the coordinate system of the heat sink entity, ensuring that data from different sensors point to the same physical location; Temporally, using hardware-synchronized timestamps, thermal sequence data and ultrasonic scanning signal data are aligned to a unified time axis. For each identified independent defect region, the average temperature change curve of the region during the entire vibration excitation process is extracted from the aligned thermal sequence data as the observed value of the actual thermal response; at the same time, the waveform feature vector reflecting the internal structural characteristics of the region is extracted from the aligned ultrasonic signal.
[0051] S33. Inversion and Optimization Solution: Find a set of defect parameters that minimizes the difference between the theoretical thermal response and the actual thermal response output by the physical information constraint layer. By constructing an inversion loss function, a gradient-based optimization algorithm (such as the adaptive moment estimation algorithm or the confined memory quasi-Newton method) is used to automatically adjust the defect parameters to be inverted through iteration, continuously reducing the value of the inversion loss function until the theoretical thermal response and the actual thermal response reach the optimal match, thereby obtaining the optimal estimate of the defect parameters.
[0052] The inversion loss function includes a heat loss term that reflects the degree of fit between the thermal response curves and a difference term in data from other sensors (such as ultrasound). The weights of each part can be adjusted according to the reliability of the sensors.
[0053] It should be noted that when the optimization solution fails to converge or the physical parameters are incorrect and cannot fit the thermal sequence data, it indicates the presence of non-defect interference. In this case, an identification alarm is triggered, and manual review is required.
[0054] S34. Output Quantitative State Parameters: Based on the optimal estimate of the defect parameters, output state parameters, including local weld ratio, defect equivalent thermal resistance, and predicted defect depth or insufficient penetration.
[0055] Understandably, the local weld ratio is calculated as a percentage of the total area of the weld after subtracting the area of all defective regions. The defect area is determined by the identified curve region mask and supported by the inverted physical parameters (as a basis for cross-validation and confidence). The defect equivalent thermal resistance is given directly by the contact thermal resistance parameters obtained from the inversion, which quantitatively characterizes the obstruction caused by the defect to the heat dissipation performance. The predicted defect depth or insufficient penetration depth is given directly by the defect burial depth parameters obtained from the inversion, which is used to determine the severity and type of the defect.
[0056] In this embodiment, the welding state quantification output is achieved by establishing a differentiable model, allowing physical laws to serve as strong constraints to correct and interpret the output of the target detection network. This ensures that the final quantification indicators (such as weld success rate) are based not only on pixel statistics but also on thermal principles, thereby significantly improving their reliability and engineering guidance value. Furthermore, this framework allows for the integration of more types of sensors (multi-source), enhancing the system's scalability.
[0057] Example 2 This embodiment also provides an intelligent inspection system for heat sink welding, such as Figure 5 As shown, it includes a vibration thermal excitation and data acquisition module, a deep learning defect identification module, and a physical information fusion and quantization module.
[0058] The vibration thermal excitation and data acquisition module is used to apply controlled mechanical vibration excitation to the heat sink specimen and simultaneously acquire infrared thermal radiation signals from the surface, generating multi-channel feature maps through signal processing.
[0059] The generation of the multi-channel feature map includes the following steps: System setup and synchronous triggering: The heat sink test piece to be tested is rigidly fixed on a vibration exciter (such as a piezoelectric ceramic actuator). An infrared thermal imager is set up directly above the test piece. A sinusoidal signal of a predetermined frequency (such as 5kHz) is generated by a signal generator. One of the signals is sent to a power amplifier to drive the vibration exciter, and the other is sent to the infrared thermal imager as an external trigger signal, so as to achieve microsecond-level hardware synchronization between vibration excitation and thermal image acquisition. Thermal sequence data acquisition: Control the vibration exciter to apply continuous low-power sinusoidal vibration to the heat sink substrate, and simultaneously trigger the infrared thermal imager to continuously record the dynamic changes of the temperature field on the heat sink surface at a preset sampling frame rate (e.g., 100 Hz) to obtain the original time-series thermal image, i.e., thermal sequence data. Feature map generation: The thermal sequence data is preprocessed and temporal features are extracted. First, temporal alignment and bad pixel correction are performed. Then, the pure temperature rise response sequence is obtained by background subtraction (with the average frame before excitation as the background). Phase-locked processing is used to extract the phase map with the same frequency as the excitation. Finally, the statistical features of each pixel in the time dimension are calculated, and a multi-channel feature map containing four two-dimensional images is synthesized, specifically including: maximum temperature rise map, arrival time map, phase map and logarithmic decay rate map.
[0060] The deep learning defect identification module is used to receive multi-channel feature maps generated by the vibration thermal excitation and data acquisition module, and use a target detection network with deep fusion texture attention mechanism to identify and locate defects (such as cold solder joints and cracks) in the heat sink welding area, and output the boundary box, category and confidence level of the defects.
[0061] The process of identifying and locating defects in the soldering area of the heat sink includes the following steps: Input tensor construction: The four independent feature channels of the multi-channel feature map are respectively normalized by Z-Score for each channel, and then stacked along the channel dimension to form an input tensor of uniform size; Forward propagation and feature inference: The input tensor is sequentially fed into the following network substructures: Feature extraction backbone network: The CSPDarknet structure is used to encode deep spatial features of the input and output multi-scale primary feature maps; Texture enhancement attention module: First, a multi-directional Gabor filter bank is used to extract the texture components of the primary feature map. Then, parallel channel attention (squeeze-excitation) and spatial attention (non-local correlation) operations are used to generate fusion weights, which reweight the original features to highlight the weak textures related to defects. Multi-scale feature fusion neck network: A bidirectional feature pyramid network (BiFPN) is used to perform weighted bidirectional fusion of texture-enhanced features and shallow features of the backbone network to aggregate multi-scale contextual information; Attention-refined prediction head: After fusing features at various scales, a convolutional block attention module (CBAM) is connected for feature filtering. Finally, the defect bounding box, confidence score and category are regressed in parallel by decoupling the detection head. Output results: The results of all prediction branches are summarized and post-processed, and the final detection results list includes the spatial location (boundary box coordinates), category, and existence confidence level of each defect.
[0062] The physical information fusion and quantization module, guided by the defect region, combines the original thermal sequence data with other sensor data to perform inversion optimization under the constraints of a differentiable physical model, transforming the visual features of the defect into physical state parameters.
[0063] The process of converting the visual features of defects into physical state parameters includes the following steps: Physical information constraint layer construction: A differentiable forward heat conduction model is established. This model takes assumed defect parameters (equivalent diameter, burial depth, contact thermal resistance) and material thermal properties as input, and calculates the theoretical temperature-time curve sequence (theoretical thermal response) of the defect region on the surface of the heat sink under corresponding vibration excitation conditions using differentiable numerical solutions (such as finite difference based on PyTorch). Multi-source information alignment: Spatially, the coordinate systems of infrared thermal imagers and sensors such as ultrasonic probes are aligned with the workpiece coordinate system through calibration; temporally, the thermal sequence data is aligned with other sensor data to a unified time axis using hardware-synchronized timestamps, and the average temperature rise curve of each identified defect area is extracted as the actual thermal response observation value. Inversion optimization solution: Construct an inversion loss function, which includes the difference between the theoretical thermal response and the actual thermal response, as well as the difference with other sensor data; use a gradient-based optimization algorithm (such as Adam) to iteratively adjust the assumed defect parameters, minimize the loss function, until the theoretical response and the observed response are optimally matched, thereby obtaining the optimal estimate of the defect parameters; Quantitative state parameter output: Based on the optimal defect parameters obtained by inversion, the final quantitative indicators of welding state are calculated and output, mainly including: local weld rate (calculated from the total defect area and the preset total weld area), defect equivalent thermal resistance (directly given by the contact thermal resistance obtained by inversion), and predicted defect depth or insufficient penetration depth (directly given by the burial depth obtained by inversion).
[0064] This invention firstly converts mechanical energy into heat energy at the defect location through hardware-synchronized vibration thermal excitation and infrared acquisition, achieving a leap from passive observation to active excitation. This significantly enhances the thermal contrast between the defect (especially internal defects) and the background, improving the signal-to-noise ratio and feature richness from the data source. Secondly, the target detection network model incorporates a texture enhancement attention module after the backbone network. It explicitly extracts multi-directional texture features through Gabor filter banks and uses adaptive weighting with channel and spatial dual attention mechanisms. This allows the network to focus on weak and low-contrast abnormal thermal texture patterns caused by defects, effectively solving the problem of small defect features being easily lost in deep networks, and greatly improving the accuracy and robustness of detection. Finally, by constructing a differentiable forward heat conduction model and establishing an inversion optimization framework, the defect region identified by the target detection network is combined with the physical response of the original thermal sequence, achieving a qualitative change from image recognition to physical parameter inversion. This outputs welding state parameters that are not only based on pixel statistics but also rooted in the physical laws of heat conduction, giving the detection results clear physical meaning and credibility.
[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.
Claims
1. A smart inspection method for heat sink welding, characterized in that: Includes the following steps: Synchronous vibration thermal imaging signal acquisition: By synchronous vibration excitation and infrared thermal imaging acquisition through hardware, the surface thermal sequence of the heat sink under continuous sinusoidal vibration is obtained, and multi-channel feature map is generated through preprocessing and feature extraction. Welding defect identification: After standardizing and unifying the size of the multi-channel feature map, it is input into the target detection network with deep fusion texture attention mechanism, and the output is the defect area of the weld. Welding state quantification output: A differentiable forward heat conduction model is constructed as a physical information constraint layer. Inversion optimization is used to obtain physical estimates of defect parameters, and finally the welding state parameters are output. The architecture of the target detection network with the deep fusion texture attention mechanism includes, in sequence, an input layer, a feature extraction backbone network, a texture enhancement attention module, a multi-scale feature fusion neck network, an attention refinement prediction head, and an output layer. The texture enhancement attention module is connected to the end of the feature extraction backbone network. It is used to explicitly enhance the weak texture information related to defects in the feature map and adaptively allocate feature weights.
2. The intelligent detection method for heat sink welding according to claim 1, characterized in that: The synchronous vibration thermal imaging signal acquisition includes the following steps: System setup: Fix the heat sink test piece to be tested on the vibration exciter, and set up an infrared thermal imager directly above or diagonally above the test piece. The vibration exciter and the infrared thermal imager are synchronously triggered by hardware. Thermal sequence acquisition: The vibration exciter is controlled to apply sinusoidal mechanical vibration of a predetermined frequency to the heat sink substrate, which synchronously triggers the infrared thermal imager. The sampling frame rate is set to continuously record the temperature field changes on the surface of the heat sink to obtain thermal sequence data. Feature map generation: The hot sequence data is preprocessed, and multi-channel feature maps are synthesized by calculating the statistical features of the time dimension of the preprocessed hot sequence data.
3. The intelligent inspection method for heat sink welding according to claim 2, characterized in that: The preprocessing of thermal sequence data includes time alignment and bad pixel correction, background subtraction to extract pure temperature rise response, and signal enhancement using phase-locked loop processing to obtain a phase map; The multi-channel feature map includes a maximum temperature rise map, a arrival time map, a phase map, and a logarithmic decay rate map.
4. The intelligent detection method for heat sink welding according to claim 1, characterized in that: The input layer is used to receive the input tensor after the multi-channel feature map has been standardized and size normalized; the feature extraction backbone network is connected after the input layer and is used to perform deep spatial feature encoding on the input tensor. The texture enhancement attention module includes a texture extraction unit and a dual attention weight generation unit connected in sequence. The texture extraction unit uses a set of Gabor filter banks with different parameters and orientations to perform parallel convolution operations on the input feature map, extracting texture components in different directions. The dual attention weight generation unit receives the fusion result of the texture components and performs channel attention operations and spatial attention operations in parallel. The channel attention operation generates a channel-dimensional weight vector through a squeezing-excitation mechanism, and the spatial attention operation generates a spatial weight matrix by calculating the nonlocal correlations between spatial locations in the feature map. Finally, the channel-dimensional weight vector and the spatial weight matrix are combined to reweight the original feature map, outputting a texture-enhanced feature map.
5. The intelligent inspection method for heat sink welding according to claim 1, characterized in that: The multi-scale feature fusion neck network, connected after the texture enhancement attention module, is used to fuse the semantic and detail information of the primary feature maps and texture enhancement feature maps from different levels of the feature extraction backbone network, and output the fused feature map. Attention-refined prediction head: includes multiple prediction branches that correspond one-to-one with the output scale of the neck network. Each prediction branch contains a convolutional block attention module and a detection head. The convolutional block attention module is used to perform channel attention and spatial attention operations on the input feature map in sequence to achieve feature selection before prediction. The detection head is connected after the convolutional block attention module and adopts a decoupled structure. It regresses the spatial location of the defect target, the confidence of the target's existence, and the defect category through independent convolutional layers. The output layer is used to summarize and output the results of all prediction branches, forming the final detection result containing bounding box coordinates, confidence scores, and categories.
6. The intelligent detection method for heat sink welding according to claim 1, characterized in that: The welding defect identification also includes the following steps: Model training: The network parameters are optimized using a composite loss function. The parameters of the feature extraction backbone network are initialized using a transfer learning strategy. During the training phase, data augmentation operations are applied to the input multi-channel feature maps. The optimization strategy uses the AdamW optimizer to perform iterative updates of the network parameters. The composite loss function consists of a weighted sum of bounding box regression loss, target confidence loss, and classification loss.
7. The intelligent inspection method for heat sink welding according to claim 1, characterized in that: The quantitative output of the welding status includes the following steps: Constructing a physical information constraint layer: Establishing a differentiable forward heat conduction model, with the model input being assumed defect parameters and inherent thermal property parameters of the material, and the output being the theoretical thermal response at the observation point on the surface of the heat sink calculated under vibration excitation conditions; Multi-source information alignment and fusion: Spatially, the coordinate systems of the infrared camera and ultrasonic probe are aligned to the coordinate system of the heat sink entity through pre-calibration; temporally, the thermal sequence data and ultrasonic scanning signal data are aligned to a unified time axis using hardware-synchronized timestamps. For each identified independent defect region, the average temperature change curve of the region during the entire vibration excitation process is extracted from the aligned thermal sequence data as the observed value of the actual thermal response; at the same time, the waveform feature vector reflecting the internal structural characteristics of the region is extracted from the aligned ultrasonic signal.
8. The intelligent inspection method for heat sink welding according to claim 7, characterized in that: The quantitative output of the welding status also includes the following steps: Inversion and optimization solution: By constructing an inversion loss function, a gradient-based optimization algorithm is used to automatically adjust the defect parameters to be inverted through iteration, continuously reducing the value of the inversion loss function until the theoretical thermal response and the actual thermal response reach the optimal match, thereby obtaining the optimal estimated value of the defect parameters; Output quantified state parameters: Based on the optimal estimate of the defect parameters, output state parameters, including local weld ratio, defect equivalent thermal resistance, and predicted defect depth or insufficient penetration.
9. A smart inspection system for heat sink welding, comprising executing a smart inspection method for heat sink welding as described in any one of claims 1-8, characterized in that: It includes a vibration thermal excitation and data acquisition module, a deep learning defect identification module, and a physical information fusion and quantization module; The vibration thermal excitation and data acquisition module is used to apply controlled mechanical vibration excitation to the heat sink specimen and simultaneously acquire infrared thermal radiation signals from the surface, generating multi-channel feature maps through signal processing.
10. The intelligent inspection system for heat sink welding according to claim 9, characterized in that: The deep learning defect identification module is used to receive the multi-channel feature map generated by the vibration thermal excitation and data acquisition module, and use a target detection network with deep fusion texture attention mechanism to identify and locate defects in the welding area of the heat sink, and output the bounding box, category and confidence of the defect. The physical information fusion and quantization module, guided by the defect region, combines the original thermal sequence data with other sensor data to perform inversion optimization under the constraints of a differentiable physical model, transforming the visual features of the defect into physical state parameters.