A method and system for detecting the quality of a welded connection of a metal door and window

Abstract

This invention discloses a method and system for inspecting the quality of welded connections in metal doors and windows, belonging to the field of industrial quality inspection technology. The method for inspecting the quality of welded connections in metal doors and windows involves simultaneously acquiring visible light and infrared image sequences of the welded area after welding, performing registration processing, and then inputting the registered images into a pre-trained dual-modal temporal feature fusion analysis model to extract the welding quality fusion features of the welded area. Based on this, a welding connection quality score is analyzed. Based on the welding connection quality score, the welding connection quality level of the welded area is determined. This invention achieves cross-modal dynamic detection by simultaneously analyzing visible light and infrared image sequences, combining the surface morphology of the weld with its internal thermal behavior. It can accurately detect hidden welding defects that are difficult to observe with the naked eye, and by utilizing quantitative features and automatic scoring, it significantly improves the accuracy, objectivity, and efficiency of the inspection results.

Description

Technical Field

[0001] This invention relates to the field of industrial quality inspection technology, specifically to a method and system for inspecting the quality of welded connections in metal doors and windows. Background Technology

[0002] Metal doors and windows, as an important component of modern architecture, rely heavily on the quality of their welded joints for structural safety and long-term durability. In industrial production, ensuring that weld quality meets specifications is a crucial step.

[0003] In existing industrial quality inspection systems, various technical approaches have been developed for assessing welding quality. Visible light visual inspection is one of the more widely used methods. It involves capturing high-resolution images of the weld surface using industrial cameras and then employing image processing algorithms to identify macroscopic geometric defects such as weld formation, undercut, and surface cracks. This technology is characterized by its ease of implementation and high efficiency.

[0004] The limitations of existing technologies include at least the following problems. First, existing technologies are generally limited to using single-modal sensing data, such as relying solely on visible light vision or manual ultrasound for static and isolated defect identification, without conducting in-depth analysis from the perspective of the welding essence. This easily leads to the evaluation of welds remaining at the level of appearance and morphology, creating a blind spot in understanding the integrity of their internal structure and the stable state of the welding thermal process. This results in high costs for missed detections and misjudgments, and brings potential safety risks. This problem seriously restricts the accuracy of welding quality assessment and makes it difficult to provide a solid foundation for the long-term safety and durability of metal doors and windows in actual service. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for inspecting the quality of welded connections in metal doors and windows, solving the problems of missed detection of internal weld defects and misjudgment of quality caused by the reliance on a single detection mode and static analysis in existing technologies.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting the welding connection quality of metal doors and windows, comprising the following steps: after the welding of the metal doors and windows is completed, simultaneously acquiring visible light image sequences and infrared image sequences of the welding area, wherein the visible light image sequence consists of several frames of visible light images and the infrared image sequence consists of several frames of infrared images; performing registration processing on the visible light image sequences and infrared image sequences of the welding area of ​​the metal doors and windows to obtain corresponding visible light alignment image sequences and infrared alignment image sequences of the welding area, wherein the visible light alignment image sequences consist of several frames of visible light alignment images and the infrared alignment image sequences consist of several frames of infrared alignment images; inputting the visible light alignment image sequences and infrared alignment image sequences into a pre-trained dual-modal temporal feature fusion analysis model to extract welding quality fusion features of the welding area of ​​the metal doors and windows; analyzing the welding connection quality score of the welding area of ​​the metal doors and windows based on the welding quality fusion features; and determining the welding connection quality level of the welding area of ​​the metal doors and windows based on the welding connection quality score.

[0007] Further, the specific steps for obtaining the visible light alignment image sequence and infrared alignment image sequence of the corresponding welding area are as follows: Based on the preset image feature point extraction algorithm, the feature point set of the welding area is extracted from the visible light image sequence and infrared image sequence of the metal door and window welding area respectively; the extracted feature point set is matched based on the preset matching algorithm to obtain the image space transformation model; the visible light image sequence and infrared image sequence are geometrically transformed based on the image space transformation model to obtain the visible light alignment image sequence and infrared alignment image sequence of the corresponding welding area.

[0008] Furthermore, the visible light alignment image consists of several visible light pixels, and each visible light pixel corresponds to a light intensity value. The infrared alignment image consists of several infrared pixels, and each infrared pixel corresponds to an apparent temperature value. Each visible light pixel corresponds one-to-one with each infrared pixel.

[0009] Furthermore, the dual-modal temporal feature fusion analysis model includes a dual-modal input layer, a weld seam region segmentation layer, a region analysis layer, a key point sampling layer, and a feature analysis output layer.

[0010] Further, the specific steps for extracting the welding quality fusion features of the metal door and window welding area are as follows: In the dual-modal input layer of the dual-modal temporal feature fusion analysis model, the visible light alignment image sequence and infrared alignment image sequence of the metal door and window welding area are received, and channel normalization processing is performed to output a standardized dual-channel temporal data block; In the weld area segmentation layer of the dual-modal temporal feature fusion analysis model, the first frame of the visible light image in the output standardized dual-channel temporal data block is used as input to perform image segmentation processing to output a binarized mask region; In the region analysis layer of the dual-modal temporal feature fusion analysis model, the standardized dual-channel temporal data block and the binarized mask region are used as input to perform global pooling and mesh generation. The system processes and outputs the global width time-series curve, global temperature time-series curve, and temperature time-series curve set of the internal mesh region of the weld. In the key point sampling layer of the dual-modal time-series feature fusion analysis model, a binarized mask and a normalized dual-channel time-series data block are used as input to perform geometric feature point localization and data extraction processing, outputting the illumination intensity time-series curve and apparent temperature time-series curve of several key points. In the feature analysis output layer of the dual-modal time-series feature fusion analysis model, the global width time-series curve, global temperature time-series curve, mesh temperature time-series curve set, and illumination intensity time-series curve and apparent temperature time-series curve of several key points are used as input to perform fusion analysis and output the welding quality fusion features of the metal door and window welding area.

[0011] Furthermore, the welding quality fusion characteristics include deformation-thermal synchronization value, oxidation temperature correlation value, cooling morphology separation degree, internal thermal shock value, and color temperature phase difference.

[0012] Further, the specific steps for analyzing the welding connection quality score of the metal door and window welding area are as follows: read the cooling morphology separation degree of the metal door and window welding area and perform standardization processing; combine the standardized cooling morphology separation degree of the metal door and window welding area with the corresponding deformation-thermal-transformation synchronization value, oxidation temperature correlation value, internal thermal shock value, and color temperature phase difference for comprehensive analysis to obtain the welding connection quality score of the metal door and window welding area.

[0013] Furthermore, the specific steps for obtaining the welding connection quality score of the metal door and window welding area are as follows: the cooling morphology separation degree, deformation-thermal change synchronization value, oxidation temperature correlation value, internal thermal shock value, and color temperature phase difference of the metal door and window welding area are converted respectively; the converted cooling morphology separation degree, deformation-thermal change synchronization value, oxidation temperature correlation value, internal thermal shock value, and color temperature phase difference are input into the preset welding connection quality analysis model to analyze and obtain the welding connection quality score of the metal door and window welding area.

[0014] Furthermore, the specific steps for determining the welding connection quality level of the metal door and window welding area are as follows: the welding connection quality score of the metal door and window welding area is matched and analyzed with several preset quality score intervals, and each quality score interval corresponds to a quality level; the quality level corresponding to the welding connection quality score falling within a preset quality score interval is taken as the welding connection quality level of the metal door and window welding area.

[0015] A system for inspecting the welding connection quality of metal doors and windows includes: a synchronous image acquisition unit, used to synchronously acquire visible light image sequences and infrared image sequences of the welding area after the welding of the metal doors and windows is completed, wherein the visible light image sequence consists of several frames of visible light images and the infrared image sequence consists of several frames of infrared images; an image registration processing unit, used to register the visible light image sequences and infrared image sequences of the welding area of ​​the metal doors and windows to obtain corresponding visible light alignment image sequences and infrared alignment image sequences of the welding area, wherein the visible light alignment image sequences consist of several frames of visible light alignment images and the infrared alignment image sequences consist of several frames of infrared alignment images; a fusion feature extraction unit, used to input the visible light alignment image sequences and infrared alignment image sequences into a pre-trained dual-modal temporal feature fusion analysis model to extract welding quality fusion features of the welding area of ​​the metal doors and windows; a quality scoring analysis unit, used to analyze the welding connection quality score of the welding area of ​​the metal doors and windows based on the welding quality fusion features; and a quality level determination unit, used to determine the welding connection quality level of the welding area of ​​the metal doors and windows based on the welding connection quality score.

[0016] The present invention has the following beneficial effects: (1) The method for inspecting the welding connection quality of metal doors and windows can fully utilize the complementarity of the two modal images by simultaneously acquiring and registering visible light images and infrared images, and combining them with a dual-modal time-series feature fusion analysis model, thereby improving the accuracy of welding quality inspection. Traditional welding quality inspection usually relies on images of a single modality. For example, it is difficult to identify defects under the surface when only visible light images are used, while infrared images may have insufficient detection accuracy due to low resolution or background noise interference. This method ensures comprehensive and accurate analysis of the welding area within the same time window by simultaneously acquiring visible light images and infrared images, reducing the risk of image distortion or information loss due to time differences. In addition, the image sequence after registration further enhances the consistency between the two modalities, providing a more reliable foundation for subsequent feature extraction and quality analysis. This dual-modal fusion scheme can effectively improve the identification ability of welding defects, especially in some complex scenarios, it can accurately identify welding defects such as surface cracks and porosity, providing strong support for quality control on the production line.

[0017] (2) The method for quality inspection of welded joints in metal doors and windows, through the hierarchical structure design of the dual-modal temporal feature fusion analysis model, enables the model to extract key features more accurately and perform comprehensive analysis when processing images of the welding area. It provides higher flexibility and accuracy than traditional methods. In particular, during the extraction of welding quality fusion features, the model can not only extract multi-dimensional features such as deformation, thermal synchronization, and cooling morphology of the welding area, but also further optimize the positioning of defect areas through the regional analysis layer and the key point sampling layer. Through this hierarchical processing, the model can conduct in-depth analysis on different welding quality features and finally output a set of fusion features. These features can fully reflect the overall quality of the welded joint. Compared with the traditional single feature analysis method, this multi-level and multi-angle analysis mode has obvious advantages. Especially when dealing with complex defects in welded joints, it can provide more accurate and comprehensive quality scores. Finally, the welding quality grade is determined by the welding quality score, which not only improves the efficiency of automated inspection, but also makes the quality assessment more objective and quantitative, which helps to optimize production processes and quality control.

[0018] (3) The method for detecting the quality of welded connections in metal doors and windows adopts a strategy of analyzing the quality score by integrating welding quality features. Compared with the traditional method that relies on manual judgment, this method greatly improves the efficiency and reliability of welding quality assessment. By integrating the cooling morphology separation degree, deformation and thermal change synchronization value, oxidation temperature correlation value, internal thermal shock value and color temperature phase difference, the final quality score no longer relies on subjective human judgment, but is obtained through specific calculation. This quantitative assessment method not only eliminates the errors and inconsistencies in manual assessment, but also enables efficient and automated processing of welding quality detection. In addition, these integrated features themselves have high discrimination and can accurately reflect the surface and internal defect features during the welding process. Especially under some complex working conditions, potential defects that are easily overlooked by traditional methods can also be detected. Finally, by matching the welding connection quality score with the preset quality score range, the accurate judgment of the welding quality level is ensured.

[0019] (4) The metal door and window welding connection quality inspection system, through modular design, divides each processing step into an independent functional unit, which greatly improves the maintainability and scalability of the system. The various units of the system, such as the synchronous image acquisition unit, image registration processing unit, fusion feature extraction unit, quality scoring analysis unit, and quality level judgment unit, are independent yet closely cooperate with each other, ensuring the efficient operation of each link. In particular, during the image acquisition and registration process, the system can acquire two types of image data in real time and accurately align them through registration processing, so that visible light and infrared images can be efficiently fused, avoiding inaccurate quality assessment caused by time differences or image alignment errors. This modular design allows the entire system to be customized and upgraded according to actual needs. For example, by changing different deep learning models to adapt to new welding standards, or by optimizing image processing algorithms to improve detection accuracy. In addition, the high level of automation of the system can effectively reduce manual intervention, reduce human error and operational complexity, thereby improving production efficiency and the consistency of welding quality inspection, and providing a reliable quality monitoring tool for industrial production lines.

[0020] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0021] Figure 1 This is a flowchart of a method for inspecting the quality of welded connections in metal doors and windows according to the present invention.

[0022] Figure 2 This is a flowchart illustrating the specific steps involved in extracting welding quality fusion features from the welding area of ​​metal doors and windows using a method for detecting the welding connection quality of metal doors and windows according to the present invention.

[0023] Figure 3 This is a block diagram of a metal door and window welding connection quality inspection system according to the present invention. Detailed Implementation

[0024] Please see Figure 1This invention provides a technical solution: a method for detecting the welding connection quality of metal doors and windows, comprising the following steps: after the welding of the metal doors and windows is completed, simultaneously acquiring a visible light image sequence and an infrared image sequence of the welding area (e.g., within five seconds), wherein the visible light image sequence consists of several frames of visible light images and the infrared image sequence consists of several frames of infrared images; performing registration processing on the visible light image sequence and the infrared image sequence of the welding area of ​​the metal doors and windows to obtain a visible light alignment image sequence and an infrared alignment image sequence of the corresponding welding area, wherein the visible light alignment image sequence consists of several frames of visible light alignment images and the infrared alignment image sequence consists of several frames of infrared alignment images; inputting the visible light alignment image sequence and the infrared alignment image sequence into a pre-trained dual-modal temporal feature fusion analysis model to extract the welding quality fusion features of the welding area of ​​the metal doors and windows; analyzing the welding connection quality score of the welding area of ​​the metal doors and windows based on the welding quality fusion features; and determining the welding connection quality level of the welding area of ​​the metal doors and windows based on the welding connection quality score.

[0025] The visible light image consists of several visible light pixels, and each visible light pixel corresponds to a light intensity value. The infrared image consists of several infrared pixels, and each infrared pixel corresponds to an apparent temperature value.

[0026] The core purpose of simultaneously acquiring visible light and infrared image sequences is to capture the co-evolution of surface morphology and internal thermophysical state of the welded area during the cooling process.

[0027] Specifically, immediately after the welding process ends, the weld area accumulates a large amount of heat and enters a brief but characteristic natural cooling window (typically lasting several seconds to tens of seconds). During this time, internal defects (such as porosity and incomplete penetration) have the most significant effect on hindering or promoting heat conduction. This invention recognizes that not only is the internal thermal behavior dynamic, but certain surface effects it induces (such as the rate of formation of specific oxide colors and thermal stress deformation) are also dynamic. Therefore, it is necessary to use image sequences rather than single images to simultaneously record these two dynamic processes.

[0028] Due to the inherent differences in imaging principles and hardware construction between visible light and infrared, the directly acquired raw image sequences do not match in resolution, field of view, and spatial location. Therefore, this invention introduces a crucial registration processing step. The dual-modal sequences are mapped to a unified pixel coordinate system, generating visible light aligned image sequences and infrared aligned image sequences. This step is a necessary technical means to achieve the leap from two independent videos to a unified, pixel-level comparable dual-channel data cube.

[0029] Specifically, the steps to obtain the visible light alignment image sequence and infrared alignment image sequence of the corresponding welding area are as follows: Based on a preset image feature point extraction algorithm, feature point sets (including several feature points) of the welding area are extracted from the visible light image sequence and infrared image sequence of the metal door and window welding area, respectively. Specifically, the first frame of the visible light image and the first frame of the infrared image are preprocessed. For the visible light image, a Gaussian filter kernel (e.g., kernel size of 5×5 pixels, standard deviation σ=1.5) is used for convolution to suppress noise. For the infrared image, median filtering (e.g., filter window of 3×3 pixels) is used to effectively eliminate isolated noise points (thermal noise). Subsequently, the Scale Invariant Feature Transform (SIFT) algorithm was applied to the two preprocessed images for feature point detection. This algorithm constructs a Gaussian pyramid scale space and detects local extrema in the Difference of Gaussians (DoG) space as candidate feature points. Specifically, in areas with drastic grayscale changes, such as the edges of the welding area and the junction of the weld and the base material, the algorithm locates a series of stable key points. All detected candidate feature points are filtered based on their contrast (i.e., DoG response value) and edge response. Low-contrast points with a contrast below a preset threshold (e.g., 0.04) are removed, and unstable points located on edges are removed using the Hessian matrix. Finally, the most robust feature points are retained to form the feature point set. The extracted feature point set is matched based on a preset matching algorithm to obtain an image space transformation model, which is as follows: For each feature point, its feature descriptor is computed using the SIFT algorithm. This descriptor is a 128-dimensional vector formed by statistically analyzing the gradient direction histogram of pixels in the neighborhood of the feature point, and is highly invariant to changes in illumination, rotation, and scale.

[0030] The k-nearest neighbor (k-NN) matching algorithm is used to find the two most similar descriptors (i.e., closest in Euclidean distance) in the set of feature descriptors in the visible light image for each feature descriptor in the infrared image. Then, the nearest neighbor distance ratio (NNDR) test is applied: the best match is accepted only if the distance ratio between the best and second-best match is less than a preset threshold (e.g., 0.8); otherwise, it is considered an invalid match. This step effectively eliminates erroneous matches.

[0031] Using the correctly matched point pairs selected in the above steps, a perspective transformation model (Homography Matrix) from the infrared image coordinate system to the visible light image coordinate system is robustly estimated using the Random Sample Consensus (RANSAC) algorithm. This model is a 3x3 matrix that accurately describes the projection transformation relationship between two images, i.e., the image space transformation model. Geometric transformations are performed on visible light image sequences and infrared image sequences based on an image space transformation model to obtain visible light alignment image sequences and infrared alignment image sequences for the corresponding welding area. Specifically: The perspective transformation model is applied to the pixel coordinates of each frame in the infrared image sequence. For a pixel with coordinates (x_ir, y_ir) in the infrared image, its target homogeneous coordinates [x'_vis, y'_vis, w]^T in the visible light image coordinate system are calculated by multiplying the homogeneous form of the coordinates [x_ir, y_ir, 1]^T with the 3x3 perspective transformation matrix.

[0032] The homogeneous coordinates of the target obtained above are normalized to obtain the actual floating-point coordinates (x'_vis / w, y'_vis / w). Since the obtained coordinates are floating-point numbers, a bilinear interpolation algorithm is used to calculate the interpolated temperature value at the target location based on the apparent temperature values ​​of the four nearest neighbor pixels around the target point.

[0033] The above steps are repeated for each frame in the infrared image sequence, ultimately generating a new infrared image that is spatially perfectly pixel-aligned with the first frame of the visible light image sequence. All these new images constitute the infrared alignment image sequence in chronological order. In this embodiment, the visible light image sequence itself serves as the spatial reference; therefore, the visible light alignment image sequence is the original visible light sequence.

[0034] In this implementation scheme, precise image registration technology is used to achieve accurate alignment of visible light and infrared images, ensuring spatial consistency between the two different modalities and providing a reliable foundation for subsequent welding quality analysis. First, efficient preprocessing steps (such as Gaussian filtering and median filtering) are used to effectively remove noise and improve image quality. Then, the SIFT algorithm is used to extract feature points, and the k-NN and RANSAC algorithms are used to accurately match the feature points, ultimately obtaining a robust image spatial transformation model. This registration method ensures that even in complex welding areas, the matching of image features remains stable and reliable, eliminating errors caused by differences in image modalities. Through a perspective transformation model, geometric transformation is performed on each frame of the image, and combined with a bilinear interpolation algorithm, the visible light and infrared images are accurately aligned, ensuring their consistency in space and time. The resulting infrared aligned image sequence is perfectly matched with the visible light image sequence, thereby effectively improving detection accuracy and avoiding errors caused by image misalignment.

[0035] Specifically, the visible light alignment image consists of several visible light pixels, and each visible light pixel corresponds to a light intensity value. The infrared alignment image consists of several infrared pixels, and each infrared pixel corresponds to an apparent temperature value. Each visible light pixel corresponds one-to-one with each infrared pixel.

[0036] The welding quality fusion characteristics include deformation and thermal deformation synchronization value, oxidation temperature correlation value, cooling morphology separation degree, internal thermal shock value, and color temperature phase difference.

[0037] The dual-modal temporal feature fusion analysis model includes a dual-modal input layer, a weld seam region segmentation layer, a region analysis layer, a key point sampling layer, and a feature analysis output layer.

[0038] The pre-training steps for the dual-modal temporal feature fusion analysis model are as follows: A large number of metal door and window welding samples with known welding quality grades were collected. For each sample, visible light image sequences and infrared image sequences of its welding area were acquired simultaneously, and the same registration process was performed to obtain the visible light aligned image sequence and infrared aligned image sequence of that sample. These registered dual sequences will serve as the basic input data for the model.

[0039] Each sample is assigned a unique and accurate weld quality rating label. This label can be determined by combining two methods: 1. Conduct destructive experiments on the sample, such as cutting it to analyze its internal structure and performing mechanical property tests, to obtain objective data such as whether there are defects such as porosity and incomplete penetration, as well as tensile strength.

[0040] 2. Several experienced welding engineers will conduct independent evaluations and scoring based on the weld appearance standards and non-destructive testing results.

[0041] Finally, the above information is combined, and each sample is assigned a continuous value from zero to one hundred as a true quality score, where zero represents complete weld failure and one hundred represents perfect quality. This score will serve as the target that the model needs to approximate during training.

[0042] The prepared training samples (registered bimodal sequences) are input into the model. The data flows sequentially through the model's bimodal input layer, weld region segmentation layer, region analysis layer, key point sampling layer, and feature analysis output layer, ultimately outputting a five-dimensional weld quality fusion feature vector. Subsequently, this feature vector is fed into a weighted scoring formula to calculate the model's predicted quality score for this sample.

[0043] The core of model training lies in minimizing the difference between predicted and true values. To achieve this, a composite loss function is constructed, which incorporates two considerations simultaneously: Calculate the average of the squared differences between the predicted and actual scores for all samples. This section ensures that the model's predicted scores are as accurate as possible in numerical magnitude.

[0044] Ensure that the model's ranking of samples of different quality levels is completely consistent with the actual ranking. That is, if the actual quality of sample A is higher than that of sample B, then the model's predicted score for A must also be higher than that for B.

[0045] The final overall loss is a weighted sum of the two losses mentioned above. By adjusting the weights, the model's emphasis on numerical accuracy and ranking correctness can be balanced.

[0046] For deep learning segmentation networks (such as U-Net) in the model, their internal parameters are initialized using weights that have been trained on large general image datasets.

[0047] For the parameters of the custom feature analysis and scoring layers in the model (such as the weights of each fused feature and the sensitivity parameter of the Gaussian function), perform preset initialization. For example, initially set the weights of all features to be equal and set the sensitivity parameter to an empirical intermediate value.

[0048] Iterative optimization process: An adaptive learning rate optimization algorithm is adopted.

[0049] The training data is divided into multiple small batches and input into the model sequentially.

[0050] For each batch, forward propagation is performed to obtain the prediction result, and then the loss is calculated. Next, the gradient (i.e., adjustment direction and magnitude) of the loss function with respect to each trainable parameter in the model is calculated using the backpropagation algorithm.

[0051] The optimizer updates all parameters in the model synchronously based on the calculated gradient, including the weights of the deep learning network and the parameters in the custom formula.

[0052] Repeat the iterations across the entire dataset until the model's performance on the reserved validation dataset stabilizes and reaches its optimal level. At this point, the model is considered to have been trained and converged.

[0053] The model after training convergence is finally evaluated using a test set that was not involved in the training process. Evaluation criteria include the correlation strength between predicted scores and true scores, the average error magnitude, and the ranking accuracy.

[0054] A model is considered successfully trained and can be solidified and deployed for actual metal door and window welding connection quality inspection tasks only when it meets the preset stringent performance requirements on all evaluation criteria.

[0055] like Figure 2As shown, the specific steps for extracting the welding quality fusion features of the metal door and window welding area are as follows: In the dual-modal input layer of the dual-modal temporal feature fusion analysis model, the visible light alignment image sequence and infrared alignment image sequence of the metal door and window welding area are received, and channel normalization processing is performed to output a standardized dual-channel temporal data block, which is specifically as follows: The system receives a visible light alignment image sequence consisting of several frames, and an infrared alignment image sequence also consisting of several frames. For example, the sequence may consist of 50 frames acquired within 5 seconds after welding is completed, at a rate of 10 frames per second. The resolution of a single frame may be 640 pixels (width) × 480 pixels (height).

[0056] Each frame of a visible light image consists of several rows and columns of visible light pixels, and each visible light pixel stores a value representing its light intensity. For example, this value is usually an integer between 0 and 255, where 0 represents pure black and 255 represents the brightest.

[0057] Each frame of an infrared image consists of several rows and columns of infrared pixels, and each infrared pixel stores a numerical value representing its apparent temperature. For example, this value can be a floating-point number, such as representing a temperature in Celsius, like 156.8 degrees Celsius.

[0058] Furthermore, due to the registration process, visible light pixels and infrared pixels located in the same row and column represent the same physical location within the welding area. For example, a visible light pixel located in row 100 and column 200 reflects the reflectance intensity of a specific point on the weld surface; while an infrared pixel located in the same row 100 and column 200 reflects the surface temperature of that same physical point.

[0059] Subsequently, all pixel values ​​in both sequences are normalized. For the visible light sequence, the illumination intensity value of each pixel is linearly scaled to the range of zero to one. For example, if the original illumination intensity value is 200, the normalized value is approximately 200 / 255 ≈ 0.784.

[0060] For infrared sequences, the apparent temperature value of each pixel is linearly scaled to the range of zero to one, based on the highest and lowest temperatures in the entire sequence. For example, if the highest temperature in the entire sequence is 300℃ and the lowest temperature is 50℃, and the temperature of a certain pixel is 200℃, then its normalized value is (200-50) / (300-50)=0.6.

[0061] Finally, the normalized visible light sequence and infrared sequence are merged along the channel dimension to form a four-dimensional standardized dual-channel temporal data block. For example, for a 50-frame 640x480 image, the dimensions of this data block are 50×480×640×2.

[0062] In the weld region segmentation layer of the dual-modal temporal feature fusion analysis model, the first frame of visible light image in the output standardized dual-channel temporal data block is used as input for image segmentation processing, and a binarized mask region is output, specifically as follows: The visible light channel data of the first frame is extracted from the above standardized dual-channel time-series data block to obtain a normalized first frame visible light image.

[0063] The first visible light image is fed into a pre-trained deep learning segmentation network. For example, this network could use the U-Net architecture, which has been trained to convergence on a dataset containing thousands of images labeled with weld seam areas.

[0064] This network analyzes each pixel in the image and outputs a probability map of the same size as the input image. Each value in the probability map represents the confidence level that the corresponding pixel belongs to the weld area. For example, if a pixel has a probability value of 0.95, it means that there is a 95% chance that the pixel belongs to the weld area.

[0065] A fixed threshold is applied to the probability map for binarization decision-making. For example, the threshold is set to 0.5. Pixels with a confidence level higher than the threshold are identified as weld seam regions and assigned a value of one; pixels with a confidence level lower than the threshold are identified as background and assigned a value of zero. This results in a binarized mask image where all pixels with a value of one collectively constitute the weld seam region. For example, in the final mask image, the weld seam region might be a continuous strip-shaped region approximately 30 pixels wide.

[0066] In the region analysis layer of the dual-modal temporal feature fusion analysis model, standardized dual-channel temporal data blocks and binarized mask regions are used as inputs. Global pooling and mesh generation are performed, and the outputs are the global width temporal curve of the weld, the global temperature temporal curve, and the set of temperature temporal curves of the internal mesh region, specifically as follows: Calculating the global width temporal curve: For each frame in the temporal data block, within the weld region defined by the binarized mask, traverse each column of pixels horizontally. Find the topmost and bottommost pixels belonging to the weld region in each column; the difference in the number of rows between them is the estimated weld width for that column. Calculate the arithmetic mean of the width estimates across all columns for that frame to obtain the global average width for that frame. Repeat this operation for all frames to obtain a global width sequence arranged in chronological order, i.e., the global width temporal curve. For example, in frame 1, the calculated average width is 32.5 pixels; in frame 50, the average width becomes 28.1 pixels. This width curve shows an overall downward trend, reflecting the cooling and contraction process of the weld.

[0067] Calculating the global temperature time-series curve: For each frame in the time-series data block, within the weld area defined by the binarized mask, directly calculate the arithmetic mean of the apparent temperature values ​​of all infrared pixels to obtain the global average temperature for that frame; repeat this operation for all frames to obtain a global temperature sequence arranged in chronological order, i.e., the global temperature time-series curve. For example: in frame 1, the average temperature is 280℃; in frame 50, the average temperature drops to 85℃.

[0068] Calculating the time-series temperature profile set: First, the circumscribed rectangle of the weld region is divided into a fixed number of rows and columns, forming a regular grid. For example, it can be divided into a 3x5 grid, with a total of 15 cells. Then, for each cell in the grid, its actual intersection with the weld region is calculated. Next, for each frame of the time-series data block, the arithmetic mean of the apparent temperature values ​​of all infrared pixels within this intersection region is calculated to obtain the average temperature of that cell in that frame. This operation is repeated for all frames, generating a time-ordered average temperature sequence for each cell. All these temperature sequences together constitute the time-series temperature profile set of the grid. For example, the temperature profile of the cell located at grid (2,3) may show that its cooling rate is significantly slower than that of other cells, suggesting that there may be internal defects in this area.

[0069] In the key point sampling layer of the dual-modal temporal feature fusion analysis model, a binarized mask and a normalized dual-channel temporal data block are used as input to perform geometric feature point localization and data extraction processing, outputting the illumination intensity temporal curves and apparent temperature temporal curves of several key points, specifically as follows: Geometric feature point localization: Within the weld area defined by the binarized mask, the positions of key points are predefined based on its geometry. These key points include at least: the center point of the weld area, the center point of the upper edge of the weld, and the center point of the lower edge of the weld. For example: by calculating the geometric center of the weld area pixels, the center point coordinates are obtained, such as (240, 320); by finding the topmost contour point of the weld area, the center point coordinates of the upper edge are obtained, such as (235, 320).

[0070] Temporal data extraction: For each located keypoint, based on its row and column coordinates, the normalized illumination intensity values ​​at that location across all frames are extracted from the standardized dual-channel temporal data block to form the illumination intensity temporal curve for that keypoint. Simultaneously, the normalized apparent temperature values ​​at that location across all frames are extracted to form the apparent temperature temporal curve for that keypoint. For example, for the center point, the illumination intensity values ​​at 50 time points are extracted to form a sequence of length 50; simultaneously, the temperature values ​​at 50 time points are extracted to form another sequence of length 50.

[0071] In the feature analysis output layer of the dual-modal temporal feature fusion analysis model, the global width temporal curve, the global temperature temporal curve, the set of grid temperature temporal curves, and the illumination intensity temporal curves and apparent temperature temporal curves of several key points are used as inputs for fusion analysis. The fusion features of the welding quality of the metal door and window welding area are then output, specifically as follows: Calculate the deformation-thermal synchronization value: Calculate the Pearson product-moment correlation coefficient between the global width time series curve and the global temperature time series curve. This coefficient is the deformation-thermal synchronization value. For example, a strongly positive correlation curve is obtained with a correlation coefficient of +0.94. This high positive value indicates that the weld shrinkage and temperature drop are highly synchronized during the cooling process, which is a sign of good quality. If defects exist, this value may drop to +0.5 or even lower.

[0072] Calculate the oxidation temperature correlation value: For each keypoint output from the keypoint sampling layer, calculate the Pearson product-moment correlation coefficient between its illumination intensity temporal curve (a specific color channel, such as red) and its apparent temperature temporal curve; take the arithmetic mean of the absolute values ​​of the correlation coefficients calculated for all keypoints, and this average value is the oxidation temperature correlation value. For example, the correlation coefficients calculated for three keypoints are 0.88, 0.92, and 0.85, respectively, and their average value is 0.883.

[0073] Calculating Cooling Topography Separation: First, from the global temperature time series curve, calculate the reciprocal of the time required for the temperature to drop from the highest temperature to 63% of its maximum, as the average cooling rate; for example, if the temperature drops from 300℃ to 300*(1-0.63)=111℃ in 4 seconds, the cooling rate is approximately 0.25 seconds⁻¹. Then, from the global width time series curve, calculate the width difference between the first and last frames, as the total shrinkage; for example, if the width shrinks from 32.5 pixels to 28.1 pixels, the total shrinkage is 4.4 pixels. Finally, multiply the average cooling rate by the total shrinkage; the product is the cooling topography separation. For example, 0.25*4.4=1.1.

[0074] Calculate the internal thermal oscillation value: For all spatially adjacent cell pairs in the grid temperature time series curve set, calculate the dynamic time warping distance (DTW) between their temperature time series curves. For example, DTW is a method to measure the similarity between two time series of different lengths or shapes; the smaller the distance value, the more similar the two curve shapes. Then, calculate the arithmetic mean of the DTW distances of all these adjacent cell pairs; this average value is the internal thermal oscillation value. For example, after calculating the DTW distance of all adjacent cell pairs, the average value is 15.6. The higher this value, the more uneven the internal cooling process.

[0075] Calculating the color temperature phase difference: For each keypoint output from the keypoint sampling layer, align its illumination intensity time-series curve with its apparent temperature time-series curve, and calculate the square root of the mean of the sum of the squares of the differences between corresponding points on the two curves, i.e., the root mean square error (RMSE). Calculate the arithmetic mean of the RMS errors for all keypoints; this average value is the color temperature phase difference. For example, if the RMS errors calculated for three keypoints are 0.12, 0.09, and 0.14, the average is 0.117. This value reflects the degree of mismatch between color and temperature changes in detail.

[0076] Feature vector output: The five feature values ​​calculated above, namely deformation-thermal synchronization value, oxidation temperature correlation value, cooling morphology separation degree, internal thermal oscillation value, and color temperature phase difference, are combined into a five-dimensional vector in a preset order. This vector is the final output weld quality fusion feature. For example, the final output weld quality fusion feature vector is [0.94, 0.883, 1.1, 15.6, 0.117].

[0077] In this implementation scheme, by simultaneously acquiring visible light and infrared image sequences and combining them with advanced feature point extraction and registration algorithms, a high degree of spatial consistency between the two modal images is ensured. Then, a deep learning segmentation network is used to accurately locate the welding area and extract welding quality-related fusion features, such as weld width, temperature change, and thermal shock. These features can comprehensively reflect the quality status of the weld joint. In particular, when calculating the welding quality fusion features, this method avoids the errors that may be caused by a single feature through comprehensive analysis of multi-dimensional features such as deformation and thermal change synchronization value, oxidation temperature correlation value, cooling morphology separation degree, internal thermal shock value, and color temperature phase difference, ensuring the accuracy of quality assessment. These features can not only effectively detect welding defects but also provide a basis for optimizing the welding process. Through the final fusion feature vector output, the system can score and classify the welding quality, greatly improving detection efficiency and accuracy, and is suitable for real-time quality control and inspection on industrial automated production lines.

[0078] Specifically, the steps for analyzing the welding connection quality score of the metal door and window welding area are as follows: read the cooling morphology separation degree of the metal door and window welding area and perform standardization processing; combine the standardized cooling morphology separation degree of the metal door and window welding area with the corresponding deformation-thermal-transformation synchronization value, oxidation temperature correlation value, internal thermal shock value, and color temperature phase difference for comprehensive analysis to obtain the welding connection quality score of the metal door and window welding area.

[0079] The specific steps for obtaining the welding connection quality score of the metal door and window welding area are as follows: The cooling morphology separation degree, deformation-thermal change synchronization value, oxidation temperature correlation value, internal thermal shock value, and color temperature phase difference of the metal door and window welding area are converted respectively; the converted cooling morphology separation degree, deformation-thermal change synchronization value, oxidation temperature correlation value, internal thermal shock value, and color temperature phase difference are input into the preset welding connection quality analysis model to analyze and obtain the welding connection quality score of the metal door and window welding area.

[0080] The specific details of the preset welding connection quality analysis model are as follows: ;in, The quality of welded connections in the welded areas of metal doors and windows is scored. The values, in order, are: deformation-thermal-synchronous deformation value, oxidation temperature correlation value, cooling morphology separation degree, internal thermal shock value, and color temperature phase difference of the welded area of ​​the metal doors and windows after conversion treatment. The weighting coefficients are, in order, the deformation and thermal deformation weighting coefficient, the oxidation temperature correlation weighting coefficient, the cooling morphology separation weighting coefficient, the internal thermal oscillation weighting coefficient, and the color temperature phase difference weighting coefficient stored in the database. In this embodiment, the values ​​are all in the range of [0, 1], and the examples of values ​​are: 0.3, 0.2, 0.25, 0.15, and 0.10.

[0081] in, , This represents the deformation-thermal synchronization value of the welded area in metal doors and windows. Since its Pearson correlation coefficient ranges from [-1, 1], this transformation linearly maps it to [0, 1]. The closer the value is to 1, the better the synergy and the higher the quality.

[0082] , This is the oxidation temperature correlation value of the welded area of ​​metal doors and windows. The larger the value, the stronger the correlation between oxidation color change and temperature, which may indicate local overheating and poorer quality.

[0083] , This is a natural constant, and in this embodiment, it is taken as 2.71. To achieve the separation degree of cooling morphology in the welded area of ​​standardized metal doors and windows, The adjustment coefficient is stored in the database. An example value is 0.95. In the formula, "-0.5" makes it symmetrical about 0, changing the range to (-0.5, +0.5). "2 × absolute value of sum |...|" changes its range to [0,1]. "1-..." inverts the result, making... When =0, =1 (full marks); when The further away from 0, The closer it is to 0.

[0084] , This refers to the internal thermal shock value of the welded area in metal doors and windows. The preset maximum internal thermal oscillation value represents the maximum acceptable oscillation level. The larger the value, the more uneven the internal structure and the worse the quality.

[0085] , For the color temperature phase difference in the welding area of ​​metal doors and windows, The preset maximum phase difference represents the maximum acceptable phase difference. The higher the value, the less synchronized the color temperature changes, and the worse the quality.

[0086] In this implementation scheme, cooling morphology separation, deformation-thermal synchronization value, oxidation temperature correlation value, internal thermal oscillation value, and color temperature phase difference are transformed, processed, and standardized to uniformly quantify the impact of each feature on welding quality. These features cover aspects such as thermal deformation, cooling rate, and temperature correlation during the welding process, effectively reflecting the overall state of the weld joint. By combining preset weighting coefficients in the database, these features are weighted according to their importance to ultimately generate a welding quality score. Specifically, the deformation-thermal synchronization value reflects the coordination of thermal deformation and shrinkage during the welding process, the oxidation temperature correlation value indicates whether the weld area is overheated, the cooling morphology separation measures the consistency of cooling during the welding process, the internal thermal oscillation value reveals the non-uniformity during the cooling process, and the color temperature phase difference reflects the synchronicity between temperature changes and color temperature changes. Through this systematic analysis method, welding quality can be evaluated more comprehensively and accurately, avoiding the limitations of subjective judgment or single feature evaluation that may exist in traditional methods.

[0087] Specifically, the steps for determining the welding connection quality level of the metal door and window welding area are as follows: the welding connection quality score of the metal door and window welding area is matched and analyzed with several preset quality score intervals, and each quality score interval corresponds to a quality level; the quality level corresponding to the welding connection quality score falling within a preset quality score interval is taken as the welding connection quality level of the metal door and window welding area.

[0088] In this implementation scheme, by matching the weld connection quality score with a preset quality score interval, the advantage is that it transforms the complex weld quality analysis into a quantitative score and grade, avoiding errors from subjective human judgment and making the quality assessment more objective and standardized. By preset multiple quality score intervals and matching the scores with the corresponding quality levels of the intervals, the system can quickly and accurately classify the weld quality. This not only improves the efficiency of automated inspection but also effectively reduces the impact of human factors on quality judgment, ensuring the consistency and reliability of the inspection results. In addition, this method makes quality control more intuitive and easier to understand.

[0089] Please see Figure 3 This invention provides a technical solution: a system for detecting the welding connection quality of metal doors and windows, comprising: a synchronous image acquisition unit, used to synchronously acquire visible light image sequences and infrared image sequences of the welding area after the welding of the metal doors and windows is completed, wherein the visible light image sequence consists of several frames of visible light images and the infrared image sequence consists of several frames of infrared images; an image registration processing unit, used to register the visible light image sequences and infrared image sequences of the welding area of ​​the metal doors and windows to obtain visible light alignment image sequences and infrared alignment image sequences of the corresponding welding area, wherein the visible light alignment image sequences consist of several frames of visible light alignment images and the infrared alignment image sequences consist of several frames of infrared alignment images; a fusion feature extraction unit, used to input the visible light alignment image sequences and infrared alignment image sequences into a pre-trained dual-modal temporal feature fusion analysis model to extract the welding quality fusion features of the welding area of ​​the metal doors and windows; a quality scoring analysis unit, used to analyze the welding connection quality score of the welding area of ​​the metal doors and windows based on the welding quality fusion features; and a quality level determination unit, used to determine the welding connection quality level of the welding area of ​​the metal doors and windows based on the welding connection quality score.

[0090] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0091] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for inspecting the quality of welded connections in metal doors and windows, characterized in that, Includes the following steps: After the welding of metal doors and windows is completed, the visible light image sequence and infrared image sequence of the welding area are acquired simultaneously. The visible light image sequence consists of several frames of visible light images, and the infrared image sequence consists of several frames of infrared images. The visible light image sequence and infrared image sequence of the welding area of ​​metal doors and windows are registered to obtain the visible light alignment image sequence and infrared alignment image sequence of the corresponding welding area. The visible light alignment image sequence consists of several frames of visible light alignment images, and the infrared alignment image sequence consists of several frames of infrared alignment images. Visible light alignment image sequences and infrared alignment image sequences are input into a pre-trained dual-modal temporal feature fusion analysis model to extract welding quality fusion features of the welding area of ​​metal doors and windows; Based on the fusion characteristics of welding quality, the welding connection quality score of the welding area of ​​metal doors and windows is analyzed. Based on the welding connection quality score, the welding connection quality level of the welding area of ​​metal doors and windows is determined.

2. The method for inspecting the quality of welded connections in metal doors and windows according to claim 1, characterized in that, The specific steps to obtain the visible light alignment image sequence and infrared alignment image sequence of the corresponding welding area are as follows: Based on a preset image feature point extraction algorithm, feature point sets of the welding area are extracted from the visible light image sequence and infrared image sequence of the welding area of ​​metal doors and windows, respectively. The extracted feature point set is matched based on a preset matching algorithm to obtain an image space transformation model. Geometric transformations are performed on visible light image sequences and infrared image sequences based on an image space transformation model to obtain visible light alignment image sequences and infrared alignment image sequences for the corresponding welding area.

3. The method for inspecting the quality of welded connections in metal doors and windows according to claim 1, characterized in that, The visible light alignment image consists of several visible light pixels, and each visible light pixel corresponds to a light intensity value. The infrared alignment image consists of several infrared pixels, and each infrared pixel corresponds to an apparent temperature value. Each visible light pixel corresponds to each infrared pixel.

4. The method for inspecting the quality of welded connections in metal doors and windows according to claim 1, characterized in that, The dual-modal temporal feature fusion analysis model includes a dual-modal input layer, a weld seam region segmentation layer, a region analysis layer, a key point sampling layer, and a feature analysis output layer.

5. The method for inspecting the quality of welded connections in metal doors and windows according to claim 4, characterized in that, The specific steps for extracting the welding quality fusion characteristics of the welded area of ​​metal doors and windows are as follows: In the dual-modal input layer of the dual-modal temporal feature fusion analysis model, the visible light alignment image sequence and infrared alignment image sequence of the metal door and window welding area are received, and channel normalization processing is performed to output a standardized dual-channel temporal data block. In the weld region segmentation layer of the dual-modal temporal feature fusion analysis model, the first frame of visible light image in the output standardized dual-channel temporal data block is used as input for image segmentation processing, and the binarized mask region is output. In the regional analysis layer of the dual-modal temporal feature fusion analysis model, standardized dual-channel temporal data blocks and binarized mask regions are used as inputs. Global pooling and mesh generation are performed to output the global width temporal curve, global temperature temporal curve, and temperature temporal curve set of the internal mesh region of the weld. In the key point sampling layer of the dual-modal temporal feature fusion analysis model, a binary mask and a standardized dual-channel temporal data block are used as inputs to perform geometric feature point localization and data extraction processing, and output the illumination intensity temporal curves and apparent temperature temporal curves of several key points. In the feature analysis output layer of the dual-modal temporal feature fusion analysis model, the global width temporal curve, the global temperature temporal curve, the set of grid temperature temporal curves, and the light intensity temporal curves and apparent temperature temporal curves of several key points are used as inputs to perform fusion analysis and output the welding quality fusion features of the metal door and window welding area.

6. The method for inspecting the quality of welded connections in metal doors and windows according to claim 1, characterized in that, The welding quality fusion characteristics include deformation and thermal deformation synchronization value, oxidation temperature correlation value, cooling morphology separation degree, internal thermal shock value, and color temperature phase difference.

7. The method for inspecting the quality of welded connections in metal doors and windows according to claim 6, characterized in that, The specific steps for analyzing the welding connection quality score of the welding area of ​​metal doors and windows are as follows: The cooling morphology separation degree of the welded area of ​​metal doors and windows is read and standardized. The cooling morphology separation degree of the welded area of ​​the standardized metal doors and windows is combined with the corresponding deformation-thermal change synchronization value, oxidation temperature correlation value, internal thermal shock value, and color temperature phase difference for comprehensive analysis to obtain the welding connection quality score of the welded area of ​​the metal doors and windows.

8. The method for inspecting the quality of welded connections in metal doors and windows according to claim 7, characterized in that, The specific steps for obtaining a welding connection quality score for the welded area of ​​metal doors and windows are as follows: The cooling morphology separation degree, deformation-thermal synchronization value, oxidation temperature correlation value, internal thermal shock value, and color temperature phase difference of the welding area of ​​metal doors and windows are converted and processed respectively. The transformed cooling morphology separation degree, deformation-thermal synchronization value, oxidation temperature correlation value, internal thermal shock value, and color temperature phase difference are input into the preset welding connection quality analysis model to obtain the welding connection quality score of the metal door and window welding area.

9. The method for inspecting the quality of welded connections in metal doors and windows according to claim 1, characterized in that, The specific steps for determining the weld connection quality level in the welded area of ​​metal doors and windows are as follows: The welding connection quality score of the metal door and window welding area is matched and analyzed with several preset quality score intervals, and each quality score interval corresponds to a quality level. The quality level corresponding to the welded connection quality score falling within a preset quality score range is used as the welded connection quality level of the metal door and window welding area.

10. A system for inspecting the quality of welded connections in metal doors and windows, employing the method for inspecting the quality of welded connections in metal doors and windows as described in any one of claims 1-9, characterized in that, include: The synchronous image acquisition unit is used to synchronously acquire the visible light image sequence and infrared image sequence of the welded area after the metal door and window welding is completed. The visible light image sequence consists of several frames of visible light images, and the infrared image sequence consists of several frames of infrared images. The image registration processing unit is used to register the visible light image sequence and the infrared image sequence of the welding area of ​​metal doors and windows to obtain the visible light alignment image sequence and the infrared alignment image sequence of the corresponding welding area. The visible light alignment image sequence consists of several frames of visible light alignment images, and the infrared alignment image sequence consists of several frames of infrared alignment images. The fusion feature extraction unit is used to input the visible light alignment image sequence and the infrared alignment image sequence into the pre-trained dual-modal temporal feature fusion analysis model to extract the welding quality fusion features of the welding area of ​​metal doors and windows; The quality scoring analysis unit is used to analyze the welding connection quality score of the welding area of ​​metal doors and windows based on the welding quality fusion characteristics. The quality grade determination unit is used to determine the welding connection quality grade of the welding area of ​​metal doors and windows based on the welding connection quality score.

Images