Weld defect detection method and system

Abstract

The invention discloses a weld defect detection method and system, and relates to the technical field of defect detection.The method comprises the steps that a visible light image sequence, an infrared image sequence and a pixel point cloud sequence are collected, windows are moved in geometric coding results of pixel point cloud of each frame and a visible light image through window attention, and the windows are extracted and sorted into visual feature maps; extracting a temperature feature map from each frame of infrared image through thermal gradient convolution and cavity convolution, and arranging the temperature feature map into a geometric feature sequence and a defect feature sequence based on cross attention and decoupling head mapping; capturing the time sequence dependence of the defect feature sequence through a Transform encoder, carrying out potential space modeling, and generating a defect probability vector by utilizing defect priori knowledge in combination with a Bayesian network; and a Gaussian regression model and a logarithmic probability equation are adopted to map the geometric feature sequence into a probability threshold of each defect type, and the probability threshold is compared with a defect probability vector to determine the defect, so that high-precision defect detection fusing geometry, temperature, time sequence and priori knowledge is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a weld defect detection method and system. Background Art

[0002] Weld quality is directly related to the safety and reliability of products in key areas. Weld defect detection can ensure production safety, optimize processes, and reduce accident risks and economic losses.

[0003] Existing technologies mostly rely on the coordination of single-modal detection or multi-modal detection results, and it is difficult to integrate multi-dimensional information such as geometric structure and temperature field. There is a lack of effective modeling of data time series dependence and probabilistic reasoning driven by prior knowledge of defects, resulting in insufficient defect detection accuracy and a high risk of missed detection. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention proposes a weld defect detection method, which improves the detection accuracy of weld defects by fusing the visual features and temperature features of visible light, infrared, and pixel point cloud multimodal sequences, mining temporal associations and Bayesian probabilistic reasoning.

[0005] The technical solutions for achieving the purpose of the present invention are:

[0006] The weld defect detection method comprises the following steps:

[0007] Collect and process the generated Visible light image sequence of frames , infrared image sequence and pixel point cloud sequence , 、 and Respectively Visible light image, infrared image and pixel point cloud of the frame;

[0008] The three collected sequences are input into the feature decoupler, and the window attention is used to Pixel point cloud of the frame The geometric coding results of Visible light map of the frame Move the window and extract the Visual feature map of the frame , through thermal gradient convolution and hole convolution from the Infrared image of the frame Extract the Temperature profile of the frame , based on cross attention and decoupled head mapping to generate Frame geometry and defect characteristics , sort out and obtain the geometric feature sequence and defect feature sequence ;

[0009] Defect feature sequence Input probability reasoning network, capture defect feature sequence through Transformer encoder The temporal dependency of the defect is modeled and the potential space is modeled. The defect probability vector is generated by combining the defect prior knowledge with the Bayesian network. ;

[0010] The geometric feature sequence is transformed into Mapped to The probability threshold of the defect type and the defect probability vector Compare to identify any defects.

[0011] Specifically, Visible light map of the frame and infrared images Acquired by dual-spectral camera, Pixel point cloud of the frame The first Frame of the world point cloud The camera rotation matrix obtained after calibration , camera translation matrix and the camera distortion matrix Transformation generation, dual-spectral camera and visible light radar based on IEEE 1588 precision clock protocol Synchronous acquisition of frames to obtain visible light image sequence , infrared image sequence and pixel point cloud sequence .

[0012] Furthermore, the processing steps of the feature decoupler include:

[0013] The window encoder will Pixel point cloud of the frame The pixel coordinates in the image are converted into position vectors through geometric coding. Visible light map of the frame The sliding window is used to obtain the tiles and add the index. The window attention splices the position vector in each window with the flattened tile. The visual feature value of each window is generated by self-attention weighting and pooling, and the first Visual feature map of the frame ;

[0014] The progressive convolutional network uses the Sobel operator to calculate the Infrared image of the frame Comprehensive thermal gradient map And perform dilated convolution with dilated convolution kernels of different expansion rates, and stack the dilated convolution results to generate the first Temperature profile of the frame ;

[0015] Using cross attention fusion Visual feature map of the frame and temperature characteristic diagram , generating the Frame fusion feature map ;

[0016] The decoupling head will Frame fusion feature map After nonlinear mapping, the first Frame geometry and defect characteristics , organized and constructed into a geometric feature sequence and defect feature sequence .

[0017] Furthermore, the processing steps of the window encoder include:

[0018] Splicing Pixel point cloud of the frame The pixel horizontal coordinate of each point in The product of the pixel vertical coordinate and The sine and cosine of the product generate the position vector of each point;

[0019] Will The window is placed in the Visible light map of the frame The upper left corner of the and the index is defined as , according to the row priority principle, each time the window is moved to the right Column and add 1 to the second index until the window moves right to the Visible light map of the frame The rightmost column of the window will jump to the first column according to the same column starting principle. Visible light map of the frame The leftmost column and move down At the same time, the first bit of the index is incremented by 1, and the window continues to move right according to the row priority principle until the window traverses the first bit. Visible light map of the frame Stop when

[0020] The first Visible light map of the frame The tiles in each window are flattened and concatenated with all position vectors in the window to generate the window feature vector and perform window attention. The self-attention mechanism is used to weight the reconstruction of the window feature vector and generate the visual feature value of each window through average pooling. The first Visual feature map of the frame .

[0021] Furthermore, the progressive convolutional network fuses the Sobel operator with the first Infrared image of the frame The convolution result generates the Comprehensive thermal gradient map of the frame ,use The dilated convolution kernels with different dilation rates and the Comprehensive thermal gradient map of the frame Perform dilated convolution to extract the Comprehensive thermal gradient map of the frame Zhongzai Temperature feature subgraphs of different scales are stacked according to the scale to generate the number of channels No. Temperature profile of the frame , where the hole convolution kernel is passed through The number of holes inserted into the traditional convolution kernel is equal to the dilation rate minus 1.

[0022] Specifically, cross attention transforms the first Visual feature map of the frame and temperature characteristic diagram Converted into Visual query matrix for a frame , temperature bond matrix Matrix with temperature values , will Visual query matrix for a frame with temperature bond matrix The product of divided by Temperature key matrix of the frame Number of channels And transformed into the first Cross-weight map of frames , and the Matrix of temperature values ​​for the frame Multiply to generate Frame fusion feature map .

[0023] Furthermore, the processing steps of the probabilistic reasoning network include:

[0024] The defect feature sequence is encoded by the Transformer Perform multi-head attention mechanism to capture the Frame defect characteristics The temporal correlation with the defect features of the remaining frames is used to generate the first The defect reconstruction features of the frame are spliced ​​and the first Frame defect indication , the mean of the defect representations of all frames is normalized by layer to generate a global defect representation ;

[0025] The global defect Generate defect distribution mean through different nonlinear mappings and the logarithmic variance of the defect distribution And correct it to the standard potential vector that conforms to the standard normal distribution, generating the potential vector ;

[0026] Obtain defect prior probability vector by statistical historical data , the Bayesian network transforms the latent vector Nonlinear transformation and defect prior logarithmic probability vector The sum is mapped to the defect probability vector .

[0027] Furthermore, the Gaussian regression model and log-odds equation are used to transform the geometric feature sequence The collaborative mapping is The probability threshold of the defect type is determined by the following steps:

[0028] The geometric features of each frame are input into the Gaussian regression model to obtain the The score of each defect type is The average score of the defect type in different frames is taken as the Comprehensive score of defect types , combined to construct a comprehensive score vector ;

[0029] Based on Bayesian reasoning, using Comprehensive score of defect types The hyperbolic tangent of The amplitude modulation hyperparameters of the defect types The product of the correction Initial threshold parameters for defect types , generating the Threshold parameters for defect types ;

[0030] Based on the reverse derivation of the logarithmic probability equation, Probability threshold of defect type Equal to Threshold parameters for defect types The standard logistic function value (Logistic).

[0031] Furthermore, the Gaussian regression model needs to be established in advance, and the Historical samples of frames and their corresponding defect marker vectors , historical samples include visible light images, infrared images and pixel point clouds, and feature decouplers are used to extract historical geometric features from historical samples of each frame and construct historical geometric feature sets , radial basis function is used to calculate the historical geometric feature set The covariance matrix of , construct the log-marginal likelihood function To describe the historical geometric feature set Defect marking vector under certain conditions The log probability of occurrence, to maximize the log marginal likelihood function Construct an optimization problem for the target and update the optimal width parameter through the optimization algorithm and the optimal noise parameters The optimal radial basis function is determined, and 0 and the optimal radial basis function are used as the mean and variance to construct a Gaussian regression model.

[0032] A weld defect detection system for executing the weld defect detection method, comprising a synchronous acquisition module, a feature decoupling module, a defect reasoning module, and a dynamic decision module;

[0033] The synchronous acquisition module synchronously controls the dual-spectrum camera and visible light radar based on the IEEE 1588 precision clock protocol. Frame collection and processing to generate visible light image sequence , infrared image sequence and pixel point cloud sequence ;

[0034] The feature decoupling module uses the feature decoupler to process the three sequences generated by the synchronous acquisition module, and uses the window attention Pixel point cloud of the frame The geometric coding results of Visible light map of the frame Move the window and extract the Visual feature map of the frame , through thermal gradient convolution and hole convolution from the Infrared image of the frame Extract the Temperature profile of the frame , using cross attention and decoupling head to output the Frame geometry and defect characteristics And organized into geometric feature sequences and defect feature sequence ;

[0035] The defect reasoning module converts the defect feature sequence Input probability reasoning network, capture defect feature sequence through Transformer encoder The long-term temporal dependency of the latent space is modeled and input into the Bayesian network together with the prior knowledge of defects to generate the defect probability vector ;

[0036] The dynamic decision module combines the Gaussian regression model and the logarithmic probability equation to transform the geometric feature sequence Convert to The probability threshold of the defect type and the defect probability vector Compare to identify any defects.

[0037] Compared with the prior art, the feature decoupler of the present invention uses window attention to move the window in the geometric encoding results of the pixel point cloud of each frame and the visible light image of each frame to extract and organize them into the visual feature map of each frame, and extracts the temperature feature map of each frame from the infrared image of each frame through thermal gradient convolution and void convolution. The geometric features and defect features of each frame are generated based on cross attention and decoupling head mapping, and are organized into geometric feature sequences and defect feature sequences, realizing cross-modal fusion of visual information and temperature information, and the defect feature sequence is input into the probabilistic inference network. The temporal dependency of the defect feature sequence is captured through the Transformer encoder and latent space modeling is performed. The defect prior knowledge is combined with the Bayesian network to generate a defect probability vector, realizing causal reasoning based on temporal dependency and prior knowledge; the Gaussian regression model and logarithmic probability equation are used to map the geometric feature sequence to the probability threshold of each defect type, and compare it with the defect probability vector to determine the defect, realizing high-precision defect detection that integrates geometry, temperature, temporal dependency and prior knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the weld defect detection method;

[0039] Figure 2 This is a schematic diagram of the window movement rule;

[0040] Figure 3 This is a progressive convolutional network model diagram;

[0041] Figure 4Schematic diagram of weld defect detection system. DETAILED DESCRIPTION

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0043] Example 1

[0044] like Figure 1 As shown, a specific embodiment of the present invention discloses a weld defect detection method, comprising the following steps:

[0045] Synchronous control of dual-spectral camera and visible light radar to fix frame length Collect target objects and process them to generate visible light image sequences , infrared image sequence and pixel point cloud sequence , 、 and Respectively Frame visible light image, infrared image and pixel point cloud, is the total number of frames;

[0046] Visible light image sequence , infrared image sequence and pixel point cloud sequence Common input feature decoupler, window encoder uses window attention in the first Pixel point cloud of the frame The geometric coding results of Visible light map of the frame Move the window and extract the Visual feature map of the frame , will Infrared image of the frame Input progressive convolutional network, through thermal gradient convolution and hole convolution, extract and generate the first Temperature profile of the frame , using cross attention to achieve cross-modal fusion and output the first Frame geometry and defect characteristics , sort out and obtain the geometric feature sequence and defect feature sequence ;

[0047] Defect feature sequence Input probability reasoning network, capture defect feature sequence through Transformer encoder middle The long time series dependency of the frame is used to fuse the time series information and probability distribution information of the defect features using latent space modeling, and the defect probability vector is generated using the Bayesian network in combination with the prior knowledge of the defect. , defect probability vector Record The probability of occurrence of a defect type;

[0048] Using Gaussian regression model to establish geometric feature sequence and Association of defect types to obtain geometric feature sequence The score of each defect type is converted into the corresponding probability threshold based on the logarithmic probability equation, and the defect probability vector is converted into The probability of occurrence of each defect type is compared with the corresponding probability threshold to determine the existence of defects, and the defect type with an occurrence probability greater than or equal to the corresponding probability threshold is considered to be confirmed to exist.

[0049] Furthermore, the dual-spectral camera and visible light radar are based on the IEEE 1588 precision clock protocol and use a fixed frame length in the same clock domain. Synchronous triggering to Frame collection, Frame acquisition and processing includes:

[0050] The dual-spectral camera collects the reflected visible light of the target object through the lens and detects the infrared energy radiated by the target object at the same time, and generates a two-dimensional first Visible light map of the frame and the two-dimensional first Infrared image of the frame ;

[0051] Visible light radar emits visible light pulses to the target object, captures the reflected light through the detector array and uses the distance measurement algorithm to calculate the distance to generate the first Frame of the world point cloud , No. Frame of the world point cloud include The world coordinates of a point in the world coordinate system, The dimension of the detector array, the distance measurement algorithm includes the time of flight method (ToF) and the phase difference method (FMCW), based on the camera rotation matrix obtained by pre-calibration of the dual-spectral camera , camera translation matrix and the camera distortion matrix , the three-dimensional Frame of the world point cloud Convert to the two-dimensional pixel coordinate system Pixel point cloud of the frame , making the Pixel point cloud of the frame The pixel coordinates of each point in Visible light map of the frame Hedi Infrared image of the frame Keeping the space unified, the transformation process is as follows:

[0052] ,

[0053] Among them, the camera rotation matrix and the camera translation matrix Collectively referred to as the camera extrinsic matrix, the spatial alignment of the world coordinate system and the camera coordinate system is achieved through rotation and translation. The camera extrinsic matrix is ​​directly related to the acquisition pose of the bispectral camera, and the camera distortion matrix Also known as the camera intrinsic parameter matrix, it reduces the dimensionality of the three-dimensional camera coordinates in the camera coordinate system and maps them to the imaging plane of the bispectral camera, generating two-dimensional pixel coordinates in the pixel coordinate system. The camera intrinsic parameter matrix is ​​only related to the specific model of the bispectral camera. After the posture of the selected specific model of bispectral camera is fixed, the camera extrinsic parameter matrix and camera intrinsic parameter matrix can be pre-acquired through the Zhang Zhengyou calibration method.

[0054] Furthermore, the processing steps of the feature decoupler include:

[0055] From the visible light image sequence , infrared image sequence and pixel point cloud sequence Extract the Visible light map of the frame , infrared image and pixel point cloud ;

[0056] The window encoder will Pixel point cloud of the frame The pixel coordinates of each point in the image are converted into a position vector through geometric coding, using a size of The window in Visible light map of the frame Middle step length Slide and add indexes, perform window attention in each window, concatenate the position vector in the window and the flattening of the tiles in the window, generate the visual feature value of the window based on self-attention weighting and pooling, and satisfy , so that there are overlapping areas in adjacent windows to ensure the continuity of visual feature values, and the visual feature values ​​of each window are arranged according to the index of the window to generate the first Visual feature map of the frame ;

[0057] The first Infrared image of the frame Input progressive convolutional network, use Sobel operator to calculate the first Infrared image of the frame The thermal gradient in the horizontal and vertical directions of the pixel is constructed. Comprehensive thermal gradient map of the frame , using different dilation rates of the dilated convolution kernel and the Comprehensive thermal gradient map of the frame Perform parallel dilated convolution to extract temperature feature sub-graphs of different scales and stack them to generate the first Temperature profile of the frame ;

[0058] Use cross attention to Visual feature map of the frame and temperature characteristic diagram Cross-modal fusion, generating the Frame fusion feature map ;

[0059] The first Frame fusion feature map Input decoupling head, which is based on a multi-layer perceptron with branch structure, and processes the first Frame fusion feature map To enhance the nonlinear expression ability, and to coordinate the input of static branch and dynamic branch, static branch and dynamic branch use different linear transformation and the same ReLU function to map the output of Frame geometry and defect characteristics , where the geometric features include the weld width, reinforcement and groove angle of the weld, and the defect features include the size and quantity changes of the pore circular hot plate and the position and amplitude changes of the crack linear temperature gradient;

[0060] Sort them out separately Frame geometry and defect characteristics , generating a sequence of geometric features and defect feature sequence .

[0061] Furthermore, the processing steps of the window encoder include:

[0062] For the Pixel point cloud of the frame Middle The pixel coordinates of the point , will Pixel point cloud of the frame Middle The pixel horizontal coordinate of the point and pixel ordinates Multiply Converted into transverse angular frequency and longitudinal angular frequency , calculate the sine and cosine values ​​respectively and integrate them into the first The position vector of the point , ;

[0063] like Figure 2 As shown, the size is The window is placed in the Visible light map of the frame The upper left corner and add the index , according to the row priority principle, the window is gradually moved to the right, and the step length of each right shift is Column, let the second index of the window before right shift plus 1 as the index of the window after right shift, until the rightmost column of the window reaches Visible light map of the frame The rightmost column of the window is jumped according to the same column starting principle, so that the top row and leftmost column of the window after jumping are respectively the bottom row of the top row of the window before jumping. Row and Visible light map of the frame The leftmost column of the window is overlapped, and the first bit of the index of the window before the jump is increased by 1 to be the index of the window after the jump. According to the row priority principle, it continues to move right gradually until the bottom row and rightmost column of the window overlap with the leftmost column of the window respectively. Visible light map of the frame Stop when the bottom row coincides with the rightmost column. Specifically, the index is The window covers Visible light map of the frame Middle Go to Row, No. Column to Tiles made of columns;

[0064] For each window, the image blocks in the window are flattened into the window visible light vector by linear transformation, and the Pixel point cloud of the frame The position vectors of all points within the window are concatenated by columns to generate a window feature vector and perform window attention. The window feature vector is weighted and reconstructed through the self-attention mechanism to strengthen the position association of local features within the window. Average pooling is used to reduce the dimension and map it to the visual feature value of the window to reduce the computational complexity.

[0065] According to the index of the window, the visual feature values ​​of all windows are arranged in a matrix form to generate the first Visual feature map of the frame , No. Visual feature map of the frame Effectively reflects obvious edge information, including the edge of the target object or the edge of the weld defect.

[0066] like Figure 3 As shown, further, the processing steps of the progressive convolutional network include:

[0067] Using Sobel operator and Infrared image of the frame Convolution is performed to obtain the pixel horizontal axis direction and the pixel vertical axis direction in the first Horizontal thermal gradient map of the frame and longitudinal thermal gradient map , the Sobel operator includes a size of The horizontal convolution kernel and vertical convolution kernel , the specific formula is as follows:

[0068] ,

[0069] ,

[0070] in, Represents the convolution operation, Horizontal thermal gradient map of the frame and longitudinal thermal gradient map According to vector operation fusion, get the Comprehensive thermal gradient map of the frame , No. Comprehensive thermal gradient map of the frame Can effectively reflect the trend and amplitude of temperature changes on the target object;

[0071] use The dilated convolution kernels with different dilation rates and the Comprehensive thermal gradient map of the frame Perform parallel dilated convolution, the dilated convolution kernel is Insert holes into the traditional convolution kernel to expand the receptive field. Comprehensive thermal gradient map of the frame exist Feature extraction is performed at different scales to obtain the corresponding temperature feature subgraphs. The expansion rate is always equal to the number of inserted holes plus 1. The dilation rate of the dilated convolution kernel is , as the dilation rate increases, the dilated convolution kernel can capture the Comprehensive thermal gradient map of the frame The temperature gradient changes in a larger range will The temperature feature sub-graphs of different scales are stacked according to the scale to generate the multi-channel Temperature profile of the frame , No. Temperature profile of the frame Number of channels .

[0072] Specifically, cross attention will Visual feature map of the frame Through linear transformation into Visual query matrix for a frame , will Temperature profile of the frame Through two different linear transformations, they are transformed into Temperature key matrix of the frame Matrix with temperature values , and the first Temperature key matrix of the frame Number of channels Equal to Matrix of temperature values ​​for the frame Number of channels , less than Temperature profile of the frame Number of channels , will Visual query matrix for a frame with temperature bond matrix The product of is transformed into the first Cross-weight map of frames And act on the Matrix of temperature values ​​for the frame , cross-modal fusion generates Frame fusion feature map , the cross attention is as follows:

[0073] ,

[0074] Among them, cross attention is essentially to observe temperature features from a visual perspective to dynamically focus on important temperature features, thereby achieving an effective combination of edge information and temperature information.

[0075] Furthermore, the processing steps of the probabilistic reasoning network include:

[0076] Processing defect feature sequences through Transformer encoder , capturing the first Frame defect characteristics Defect feature sequence The temporal correlation of the remaining defect features in the Frame defect characteristics They are mapped to different subspaces and converted into attention weight vectors in different subspaces through Softmax function. The attention weight vector in each subspace is weighted summed with the remaining defect features to obtain the first defect vector in different subspaces. The defect reconstruction features of the frame are divided into The defect reconstruction features of the frame are concatenated by column and mapped into the first Frame defect indication ,Will The defect representations of the frames are averaged and layer normalized (LN) to generate a global defect representation. , global defect representation Condensed defect feature sequence Core defect characteristics and timing information in;

[0077] Based on latent space modeling, the global defect is represented as Through two different linear transformations and ReLU functions, they are mapped to the mean of the defect distribution. and the logarithmic variance of the defect distribution , randomly sampled from the standard normal distribution to generate a standard latent vector, and the defect distribution mean and defect distribution variance Corrected calculation of latent vector , latent space modeling transforms defect feature sequences The core defect features and timing information in the model are compressed and mapped to the latent space to avoid overfitting and enhance generalization ability. For the natural constant Logarithmic function with base ;

[0078] Obtaining defect prior probability vectors through historical statistics , and the latent vector The Bayesian network adjusts the potential vector by linear transformation. The dimension is And use the ReLU function to enhance the nonlinear expression ability, and the defect prior logarithmic probability vector After addition, the defect probability vector is generated through the Softmax function , defect prior probability vector Recording historical data The probability of occurrence of a defect type, the Bayesian network is essentially a combination of the prior information of historical data to predict the potential vector Conditional probability reasoning is performed based on the defect distribution reflected.

[0079] Furthermore, the Gaussian regression model is used to establish the geometric feature sequence and The correlation between the defect types and the corresponding probability thresholds are obtained by converting them, including the following steps:

[0080] The first Frame geometry Enter the Gaussian regression model and get the Frame Scores for different defect types, Frame 1 to frame 2 of the defect type The mean score of the frames is taken as Comprehensive score of defect types To reduce the impact of single frame noise, a comprehensive score vector is generated based on the order of defect types. ;

[0081] Based on Bayesian reasoning, Comprehensive score of defect types Through the hyperbolic tangent function Introducing Initial threshold parameters for defect types Make adaptive adjustments to generate the Threshold parameters for defect types ,in, For the The amplitude modulation hyperparameters of the first defect type are Initial threshold parameters for defect types Predefined based on historical data;

[0082] Based on the log-odds equation, Probability threshold of defect type Complementary probability threshold The logarithm of the ratio is equal to Threshold parameters for defect types ,Right now , reverse deduction to obtain the Probability threshold of defect type , For the natural constant An exponential function with base .

[0083] Furthermore, the Gaussian regression model needs to be established in advance and pre-collected The historical samples of each frame are marked with the corresponding defect marking vector according to the defect type that appears in the historical samples of each frame. , defect marker vector Record The presence or absence of a defect type, 1 and 0 respectively indicate presence and absence. The historical samples of each frame include visible light images, infrared images and pixel point clouds. The historical geometric features of the historical samples of each frame are extracted by the feature decoupler and the historical geometric feature set is constructed. , radial basis function is selected as the kernel function to calculate Covariance matrix of historical geometric features of the frame , where the covariance matrix No. Rank Covariance of columns The specific calculation method is as follows:

[0084] ,

[0085] in, and Respectively Frame and The historical geometric features corresponding to the historical samples of the frame, is the width parameter of the radial basis function, and the logarithmic marginal likelihood function is constructed To describe the historical geometric feature set Defect marking vector under certain conditions Conditional probability of occurrence The logarithmic form of the log marginal likelihood function The details are as follows:

[0086] ,

[0087] in, Defect mark vector The transpose of is the noise parameter of the radial basis function, is the covariance matrix Identity matrix of the same dimension, log marginal likelihood function The first item is used to measure the defect marking vector The degree of match with the predicted value of the Gaussian regression model. The second term is used to control the complexity of the Gaussian regression model to maximize the log marginal likelihood function Construct an optimization problem for the target and update the optimal width parameter through the optimization algorithm and the optimal noise parameters The optimal radial basis function is determined, and 0 and the optimal radial basis function are used as the mean and variance respectively to construct a Gaussian regression model. The optimization algorithms include conjugate gradient method, Newton method, quasi-Newton method, stochastic gradient descent algorithm and expectation maximization algorithm.

[0088] Example 2

[0089] like Figure 4 As shown, a specific embodiment of the present invention discloses a weld defect detection system for executing the weld defect detection method, comprising a synchronous acquisition module, a feature decoupling module, a defect reasoning module and a dynamic decision module;

[0090] The synchronous acquisition module is based on the IEEE 1588 precision clock protocol and uses a fixed frame length in the same clock domain. Synchronous control of dual-spectral camera and visible light radar Frame collection and processing to generate visible light image sequence , infrared image sequence and pixel point cloud sequence ;

[0091] The feature decoupling module uses feature decouplers to collaboratively process visible light image sequences , infrared image sequence and pixel point cloud sequence , using window attention in the first Pixel point cloud of the frame The geometric coding results of Visible light map of the frame Move the window and extract the Visual feature map of the frame , for Infrared image of the frame Perform thermal gradient convolution and dilated convolution to extract the first Temperature profile of the frame , using cross attention fusion feature map and outputting the first Frame geometry and defect characteristics , sort out and obtain the geometric feature sequence and defect feature sequence ;

[0092] The defect reasoning module converts the defect feature sequence Input probability reasoning network, capture defect feature sequence through Transformer encoder The long-term temporal dependency of the latent space is modeled and input into the Bayesian network together with the prior knowledge of defects to generate the defect probability vector ;

[0093] The dynamic decision module uses Gaussian regression model to establish geometric feature sequence and The association of defect types generates a score for each defect type and converts it into a corresponding probability threshold based on the logarithmic probability equation. The occurrence probability of each defect type is compared with the corresponding probability threshold to determine the existence of defects.

[0094] The present invention discloses a weld defect detection method and system, which inputs the collected visible light image sequence, infrared image sequence and pixel point cloud sequence into a feature decoupler, uses window attention to move the window in the geometric encoding result of the pixel point cloud of each frame and the visible light image of each frame, and extracts and organizes them into a visual feature map of each frame, extracts the temperature feature map of each frame from the infrared image of each frame through thermal gradient convolution and void convolution, generates the geometric features and defect features of each frame based on cross attention and decoupling head mapping, and organizes them into a geometric feature sequence and a defect feature sequence; inputs the defect feature sequence into a probabilistic inference network, captures the temporal dependency of the defect feature sequence through a Transformer encoder and performs latent space modeling, and generates a defect probability vector by combining defect prior knowledge with a Bayesian network; uses a Gaussian regression model and a logarithmic probability equation to map the geometric feature sequence to a probability threshold for each defect type and compares it with the defect probability vector to determine the defect, thereby realizing high-precision defect detection that integrates geometry, temperature, temporal dependency and prior knowledge.

[0095] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A weld defect detection method, characterized in that: The following steps are involved: The collected visible light image sequence, infrared image sequence, and pixel point cloud sequence are input into the feature decoupler. Window attention is used to move the window in the geometric encoding results of the pixel point cloud of each frame and the visible light image of each frame to extract and organize them into visual feature maps of each frame. Thermal gradient convolution and hole convolution are used to extract the temperature feature map of each frame from the infrared image of each frame. Based on cross attention and decoupling head mapping, the geometric features and defect features of each frame are generated and organized into geometric feature sequences and defect feature sequences. The defect feature sequence is input into the probabilistic inference network. The temporal dependency of the defect feature sequence is captured through the Transformer encoder and latent space modeling is performed. The defect probability vector is generated by combining the defect prior knowledge with the Bayesian network. The Gaussian regression model and log-odds equation are used to map the geometric feature sequence into the probability threshold of each defect type and compare it with the defect probability vector to determine the defects.

2. The weld defect detection method according to claim 1, wherein: The method of mapping the geometric feature sequence to the probability threshold of each defect type using the Gaussian regression model and the logarithmic probability equation includes the following steps: A Gaussian regression model is used to convert the geometric features of each frame into scores for all defect types in each frame. The mean of the scores of each defect type in different frames is regarded as the comprehensive score of each defect type, and the comprehensive score vector is constructed by combining them. Based on Bayesian reasoning, the initial threshold parameter of the corresponding defect type is modified by multiplying the hyperbolic tangent value of the comprehensive score of each defect type by the amplitude modulation hyperparameter of the corresponding defect type to generate the threshold parameter of the corresponding defect type; Based on the log-odds equation, the probability threshold of each defect type is obtained to be equal to the standard logistic function value of the threshold parameter of each defect type.

3. The weld defect detection method according to claim 1 or 2, characterized in that: The Gaussian regression model is pre-established and includes the following steps: Collect multiple frames of historical samples and annotate the corresponding defect marker vectors. The historical samples include visible light images, infrared images, and pixel point clouds. The feature decoupler is used to extract historical geometric features from the historical samples of each frame and construct a historical geometric feature set; The radial basis function is used to calculate the covariance matrix of the historical geometric feature set and construct the logarithmic marginal likelihood function; An optimization problem is constructed with the goal of maximizing the log-marginal likelihood function, and an optimization algorithm is used to determine the optimal radial basis function to construct a Gaussian regression model.

4. The weld defect detection method according to claim 1, wherein: The processing steps of the feature decoupler include: The window encoder converts the pixel coordinates in each frame's pixel point cloud into position vectors through geometric encoding. It then slides a window across the visible light image of each frame to obtain tiles and add indexes. The window attention concatenates the position vectors within each window with the flattened tiles. It then generates visual feature values ​​for each window through self-attention weighting and pooling, and constructs a visual feature map for each frame based on the indexes. The progressive convolutional network uses the Sobel operator to calculate the comprehensive thermal gradient map of the infrared image of each frame and simultaneously performs dilated convolutions with different expansion rates, stacking them to generate the temperature feature map of each frame; Use cross attention to fuse the visual feature map and temperature feature map of each frame to generate a fused feature map of each frame; The decoupling head processes the fused feature map of each frame through nonlinear mapping, and then generates the geometric features and defect features of each frame through nonlinear mapping of different branches, and organizes and constructs them into geometric feature sequences and defect feature sequences.

5. The weld defect detection method according to claim 4, wherein: The processing steps of the window encoder include: The position vector of each point is generated by concatenating the sine and cosine values ​​of the multiples of the pixel horizontal coordinate and the pixel vertical coordinate of each point in the pixel point cloud of each frame; Place the window at the upper left corner of the visible light map of each frame and define the index as , each time the window is moved right by a fixed step size and the second bit of the index is incremented by 1 until the window is moved right to the rightmost column of the visible light image. The window is then directly jumped to the leftmost column of the visible light image and moved down by a fixed step size. At the same time, the first bit of the index is incremented by 1. The right shift is continued until the window traverses the visible light image of each frame. The visible light map of each frame is flattened in each window and concatenated with all position vectors in the window to generate a window feature vector and perform window attention. The visual feature value of each window is generated through the self-attention mechanism and pooling, and arranged into a visual feature map of each frame according to the index.

6. The weld defect detection method according to claim 1, wherein: The processing steps of the probabilistic inference network include: The Transformer encoder performs a multi-head attention mechanism on the defect feature sequence to capture the temporal correlation between the defect features of each frame and the defect features of other frames in different subspaces. The defect reconstruction features of each frame in different subspaces are generated and concatenated. The defect representation of each frame is generated through nonlinear mapping. The mean of the defect representations of all frames is normalized through layers to generate a global defect representation. The global defect representation is passed through different nonlinear mappings to generate the defect distribution mean and defect distribution logarithmic variance, and then corrected to a standard latent vector that conforms to the standard normal distribution to generate a latent vector; The defect prior probability vector is obtained by statistically analyzing historical data. The Bayesian network uses the Softmax function to map the nonlinear transformation of the latent vector and the sum of the defect prior logarithmic probability vector into a defect probability vector.

7. The weld defect detection method according to claim 4, wherein: The progressive convolutional network generates a comprehensive thermal gradient map for each frame by fusing the convolution result of the Sobel operator with the infrared image of each frame based on vector operations. Dilated convolution kernels with different expansion rates are used for dilated convolution to extract temperature feature submaps of different scales from the comprehensive thermal gradient map of each frame and stack them to generate a temperature feature map for each frame. The dilated convolution kernel is generated by inserting a number of holes equal to the expansion rate minus 1 into the traditional convolution kernel.

8. The weld defect detection method according to claim 1 or 4, characterized in that: Cross attention transforms the visual feature map and temperature feature map of each frame into the visual query matrix, temperature key matrix and temperature value matrix of each frame through different linear transformations. The product of the visual query matrix and the temperature key matrix of each frame is divided by the number of channels of the temperature key matrix of each frame and transformed into the cross weight map of each frame through the Softmax function, which is multiplied with the temperature value matrix of each frame to generate the fusion feature map of each frame.

9. The weld defect detection method according to claim 1 or 4, characterized in that: The collection of visible light image sequences, infrared image sequences, and pixel point cloud sequences includes: The visible light image and infrared image of each frame are acquired by shooting with a dual-spectral camera and combined to construct a visible light image sequence and an infrared image sequence; The visible light radar collects the world point cloud of each frame and transforms it through the camera rotation matrix, camera translation matrix and camera distortion matrix to generate the pixel point cloud of each frame, and construct a pixel point cloud sequence.

10. Weld defect detection system, characterized in that, It includes synchronous acquisition module, feature decoupling module, defect reasoning module and dynamic decision module; The synchronous acquisition module synchronously acquires multiple frames and processes them to generate a visible light image sequence, an infrared image sequence and a pixel point cloud sequence; The feature decoupling module uses window attention to extract the corresponding visual feature map from the geometric encoding results of the pixel point cloud of each frame and the part of the visible light image of each frame covered by the window. It extracts the corresponding temperature feature map from the infrared image of each frame through thermal gradient convolution and void convolution. Based on cross attention and decoupling head mapping, it generates geometric features and defect features of each frame and organizes them into geometric feature sequences and defect feature sequences. The defect reasoning module captures the temporal dependency of the defect feature sequence through the Transformer encoder and performs latent space modeling, and inputs the defect prior knowledge into the Bayesian network to generate a defect probability vector; The dynamic decision module combines the Gaussian regression model with the logarithmic probability equation to convert the geometric feature sequence into a probability threshold for each defect type, and compares it with the defect probability vector to determine the defect.

Images