A weld surface quality detection method, system and storage medium

Abstract

This invention provides a method, system, and storage medium for weld surface quality inspection. The method involves acquiring a weld surface image, extracting shallow feature maps via a backbone network, performing discrete wavelet transform on the shallow feature maps, generating a frequency domain guiding map based on the energy distribution of high-frequency components, and weighting the shallow feature map channels to obtain shallow enhanced features. In the upsampling path, adjacent high- and low-level feature maps are used as queries and keys, respectively. Pixel-level correlation is calculated to generate cross-scale gating weights, which are then weighted and fused with higher-level feature maps to obtain multi-scale fused features. These multi-scale fused features are processed using deformable convolution with dual amplitude and direction constraints. The amplitude constraint uses the feature information entropy to set the upper bound of the sampling point offset, and the direction constraint uses the local gradient direction map to guide the offset direction. The features are upsampled to the original image size, and a defect probability map is obtained through a pixel-level classifier. After thresholding, the location and contour of weld surface defects are determined.

Description

Technical Field

[0001] This application belongs to the field of image processing, and in particular relates to a method, system and storage medium for inspecting the surface quality of welds. Background Technology

[0002] Current weld quality inspection mainly relies on manual visual inspection, which makes it difficult to guarantee the stability and consistency of the inspection. Manual feature extraction operators, such as edge detection and texture analysis, are sensitive to changes in lighting and background noise, and have poor generalization ability. Deep learning-based detection methods mostly employ an encoder-decoder architecture, extracting high-level semantic features through layer-by-layer downsampling, and then restoring resolution by combining low-level spatial details through upsampling paths, achieving pixel-level defect segmentation. However, during the encoder's extraction of shallow features, convolutional operations struggle to distinguish high-frequency details related to minute defects, easily losing crucial clues during downsampling. In the upsampling path, features at different levels are spliced ​​or added through skip connections, which can easily lead to conflicts and information confusion between high-level abstract semantics and low-level textures, affecting defect boundary localization.

[0003] In deformable convolution, the offset of sampling points is learned by the network from input features, lacking geometric constraints and prior knowledge guidance. This can lead to two problems: first, the offset amplitude is uncontrollable; when the learned offset is too large, sampling points may deviate from the target and fall into irrelevant background areas, thus utilizing noise interference and affecting feature representation; second, the offset direction is unclear. For directional linear crack defects, unconstrained offset learning may fail to form a sampling pattern along the defect direction, resulting in insufficient feature extraction capability for the defect. Therefore, how to constrain the sampling behavior of deformable convolution so that it can adapt to the defect morphology while focusing on the target region and perceiving the geometric structure is a technical challenge that urgently needs to be solved to improve the performance of weld defect detection. Summary of the Invention

[0004] This invention proposes a weld surface quality inspection method to address the problem that existing technologies fail to constrain the sampling behavior of deformable convolution, enabling it to adapt to defect morphology while focusing on the target region and sensing geometric structure. The method includes:

[0005] A weld surface image is acquired and input into a backbone network to extract shallow feature maps. Discrete wavelet transform is performed on the shallow feature maps, and a frequency domain guide map is generated based on the energy distribution of high-frequency components. The shallow feature maps are then channel-weighted using the frequency domain guide map to obtain shallow enhancement features.

[0006] In the upsampling path, for high-level feature maps and low-level feature maps at adjacent scales, the high-level feature map is used as the query and the low-level feature map is used as the key and value to calculate the pixel-level correlation between the two to generate cross-scale gating weights; the low-level feature map is weighted using the weights, and the weighted low-level feature map is fused with the high-level feature map to obtain multi-scale fused features;

[0007] The multi-scale fusion features obtained above are processed by deformable convolution with dual constraints to obtain the features.

[0008] The features obtained above are upsampled to the original image size, and a defect probability map is obtained through a pixel-level classifier. The defect probability map is then thresholded to determine the location and outline of the defects on the weld surface.

[0009] In a second aspect, the present invention proposes a weld surface quality inspection system, comprising the following modules:

[0010] A generation module is used to acquire a weld surface image, input it into a backbone network to extract a shallow feature map; perform discrete wavelet transform on the shallow feature map, generate a frequency domain guide map based on the energy distribution of high-frequency components, and use the frequency domain guide map to perform channel weighting on the shallow feature map to obtain shallow enhancement features;

[0011] The fusion module is used to calculate the pixel-level correlation between high-level and low-level feature maps at adjacent scales in the upsampling path, using the high-level feature map as the query and the low-level feature map as the key and value, to generate cross-scale gating weights; the low-level feature map is weighted using the weights, and the weighted low-level feature map is fused with the high-level feature map to obtain multi-scale fused features;

[0012] The computation module is used to process the multi-scale fusion features obtained above through deformable convolution with dual constraints to obtain features;

[0013] The determination module is used to upsample the features obtained above to the original image size, obtain a defect probability map through a pixel-level classifier, and perform threshold processing on the defect probability map to determine the location and contour of the weld surface defects.

[0014] In a third aspect, the present invention provides a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the method described in the first aspect.

[0015] This invention generates a frequency domain guiding map using discrete wavelet transform and applies channel weighting to shallow feature maps, enabling the acquisition of fine-grained texture details of weld defects. In the upsampling path, cross-scale gating weights are calculated, and the semantic information of high-level features guides the fusion of low-level features, achieving alignment and integration of features at different scales. Furthermore, a deformable convolution with dual constraints is used. Feature information entropy is used to limit the amplitude of sampling point offset, avoiding sampling from irrelevant background regions; simultaneously, local gradients provide directional guidance for sampling point offset, ensuring the deformable convolution conforms to the actual geometry of the defect. This improves the model's ability to model weld defects of varying shapes and achieves segmentation of defect location and contour. Attached Figure Description

[0016] Figure 1 A flowchart of the first embodiment;

[0017] Figure 2 A schematic diagram of the shallow feature enhancement module for weld seam images;

[0018] Figure 3 A schematic diagram for calculating multi-scale fusion features;

[0019] Figure 4 This is a schematic diagram of the grayscale histogram. Detailed Implementation

[0020] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0021] In the first embodiment, the present invention proposes a method for inspecting the surface quality of welds, see [link to relevant documentation]. Figure 1 ,include:

[0022] S1. Acquire a weld surface image and input it into the backbone network to extract shallow feature maps; perform discrete wavelet transform on the shallow feature maps, generate a frequency domain guide map based on the energy distribution of high-frequency components, and use the frequency domain guide map to perform channel weighting on the shallow feature maps to obtain shallow enhancement features;

[0023] Two-dimensional grayscale images of the weld are continuously acquired by camera scanning and input into the first few convolutional layers of a pre-trained ResNet50 network as the backbone network. The feature map output from the first residual block is extracted as the shallow feature map. Haar wavelet transform is applied to each channel of the shallow feature map, decomposing it into four components: LL, LH, HL, and HH. For each channel, the sum of squared energy values ​​of the three high-frequency components (LH, HL, and HH) is calculated. The energy values ​​of all channels are combined into a vector, which is then used to generate a channel attention vector, i.e., a frequency domain guiding map, through a fully connected layer and a sigmoid activation function. The frequency domain guiding map is multiplied channel-by-channel with the original shallow feature map to obtain the shallow enhanced features, such as... Figure 2 Defects on the weld surface, such as cracks, porosity, and incomplete penetration, typically appear as edges, abrupt texture changes, or high-contrast areas in images, all of which constitute high-frequency information in signal processing. By performing discrete wavelet transform on the shallow feature map, the features can be decomposed into low-frequency components representing the contour and background, and high-frequency components representing edges and details. A guide map is generated based on the energy distribution of the high-frequency components, identifying which feature channels respond more strongly to details highly correlated with defects.

[0024] Optionally, the shallow feature map has dimensions [B,C,H,W], while the frequency domain guide map it generates undergoes an averaging operation along the channel dimension, thus changing its dimensions to [B,1,H,W]. During channel-weighted multiplication (i.e., element-wise multiplication), the guide map with dimensions [B,1,H,W] is automatically broadcast, copying its unique channel C times to expand it into the shape [B,C,H,W], and then multiplying it with the shallow feature map.

[0025] To utilize high-frequency information in the feature map to locate potential defect edges or texture details, in an optional embodiment, performing a discrete wavelet transform on the shallow feature map to generate a frequency domain guiding map based on the energy distribution of the high-frequency components includes:

[0026] A second-order discrete wavelet transform is performed on each channel of the shallow feature map to obtain multiple high-frequency sub-bands;

[0027] For each channel, the absolute values ​​of each high-frequency sub-band obtained by the second-order discrete wavelet transform are summed point by point to obtain the energy response map of the channel. , where c is the channel index;

[0028] The initial energy map is obtained by averaging the energy response maps of all channels along the channel dimension. ;

[0029] The initial energy map is normalized using an activation function to generate the frequency domain guide map.

[0030] In one example, a shallow feature map with 64 channels and a size of 128×128 pixels is input. A two-level discrete wavelet transform is performed on each channel of the feature map, decomposing the feature map into sub-bands of different frequencies. The two-level transform produces six high-frequency sub-bands: a first-level horizontal, vertical, and diagonal sub-band, and a second-level horizontal, vertical, and diagonal sub-band, each with a size of 32×32 pixels. These high-frequency sub-bands primarily detect edge and detail information in the image.

[0031] For any of the 64 channels, the absolute values ​​of the corresponding six high-frequency sub-bands are taken and accumulated pixel-level at the same spatial location. This process generates a 32×32 energy response map for each channel. The 64 energy response maps are averaged along the channel dimension. That is, for each pixel location, the energy values ​​of the 64 channels are added together and then divided by 64, thus fusing them into a single-channel initial energy map. The initial energy map is processed using the Sigmoid activation function, smoothly mapping the values ​​to between 0 and 1 to generate a frequency domain guide map. Regions with values ​​close to 1 in the map indicate concentrated high-frequency energy, potentially indicating defects; conversely, regions with values ​​closer to 1 represent background areas.

[0032] S2, In the upsampling path, for high-level feature maps and low-level feature maps of adjacent scales, the high-level feature map is used as the query and the low-level feature map is used as the key and value to calculate the pixel-level correlation between the two to generate cross-scale gating weights; the low-level feature map is weighted using the weights, and the weighted low-level feature map is fused with the high-level feature map to obtain multi-scale fused features.

[0033] In the upsampling decoding path, the feature map upsampled by bilinear interpolation in the decoder is used as the high-level feature map, and the feature map of the corresponding spatial size in the encoder path, passed through skip connections, is used as the low-level feature map. The low-level feature map comes from the encoder in the first half of the network, i.e., the downsampling path; the high-level feature map comes from the decoder in the second half of the network, i.e., the upsampling path. The high-level feature map is flattened in the spatial dimension as the query Q, and the flattened low-level feature map is used as the key K and value V, respectively. The dot product of query Q and the transpose of key K is calculated, the result is scaled by dividing by the square root of the feature dimension, and then the attention score matrix, i.e., the cross-scale gating weights, is calculated using the Softmax function. The weight matrix is ​​multiplied by the value V, for example, by multiplying the weight map element-wise with the low-level feature map used as the value, to obtain the weighted low-level feature map. The weighted low-level feature map and the high-level feature map are concatenated in the channel dimension, and then information fusion and channel dimensionality reduction are performed through a 1×1 convolutional layer to obtain multi-scale fused features, such as... Figure 3.

[0034] More specifically, the step of using the high-level feature map as a query and the low-level feature map as a key and value to calculate the pixel-level correlation between the two and generate cross-scale gating weights includes:

[0035] The high-level feature map and the low-level feature map are transformed in dimension to obtain the query matrix Q and the key matrix K.

[0036] Transpose of the query matrix Q and the key matrix K Perform matrix multiplication and scaling to obtain a pixel-level correlation matrix. ,in The dimension of the key vector;

[0037] The cross-scale gating weights are obtained by normalizing along the key dimension of the correlation matrix using the Softmax function.

[0038] Before calculating the correlation, a 1×1 convolutional layer is used to project both the high-level and low-level feature maps onto an intermediate feature space of the same dimension. For example, the high-level feature map has 256 channels and is 32×32 pixels; the low-level feature map has 128 channels and is 64×64 pixels, preserving spatial details. Their channel dimensions are then adjusted using a 1×1 convolution, assuming both are adjusted to 128 dimensions. A dimensional transformation is performed, flattening the high-level feature map into a 1024×128 query matrix Q, where 1024 represents the total number of pixels (32×32); and flattening the low-level feature map into a 4096×28 key matrix K, where 4096 represents the total number of pixels (64×64).

[0039] Calculate the relevance between the query and the key by transposing the query matrix Q and the key matrix K. Matrix multiplication is performed to obtain a 1024×4096 pixel-level correlation matrix. Each element in this matrix represents the similarity between a specific pixel in a high-level feature map and a specific pixel in a low-level feature map. To ensure the stability of the gradient during backpropagation, each element of the correlation matrix is ​​divided by a scaling factor. For each row of the correlation matrix, i.e., along the key dimension, the Softmax function is applied. This transforms the 4096 correlation scores in each row into a probability distribution that sums to 1, resulting in the cross-scale gating weights. These weights determine which pixels in the lower-level feature maps contribute most to the cross-scale gating weights for each pixel in the higher-level feature maps.

[0040] S3, The multi-scale fusion features obtained above are processed by deformable convolution with dual constraints to obtain features;

[0041] Fixed-shape convolutional kernels, such as 3x3 rectangles, struggle to adapt to changing geometric features. Deformable convolutions learn additional offsets, allowing their sampling points to dynamically adjust their positions. This enables the receptive field of the kernel to adaptively fit the actual shape of the defect, improving feature extraction capabilities for irregular targets. Dual constraints address potential instability issues during training of deformable convolutions. Unconstrained offset learning can cause sampling points to deviate too far from the target, falling into irrelevant background regions and introducing noise. Dual constraints, such as limiting the range or direction of the offsets, regulate the offset learning process, ensuring that sampling points are always near meaningful feature regions. In one embodiment, for the input multi-scale fused feature map, a deformable convolutional layer is connected in parallel with a standard convolutional layer to predict the offset field. The size of the offset field is the same as the input feature map, but the number of channels is twice the number of sampling points in the convolutional kernel, with each point corresponding to an offset in both the x and y directions. The sampling point positions in the input feature map are dynamically adjusted based on the offsets. The dual constraints are added as regularization terms to the loss function, thereby guiding the learning of the offsets in a more reasonable and focused direction when the model backpropagates to update the parameters.

[0042] In some embodiments, the dual constraints include: magnitude constraint: calculating the feature information entropy of the multi-scale fused feature at each spatial location to generate an upper bound for limiting the magnitude of the deformable convolution sampling point offset; and orientation constraint: calculating the local gradient direction map of the multi-scale fused feature to provide directional guidance for the sampling point offset of the deformable convolution. The Sobel operator is applied to each channel of the multi-scale fused feature map to calculate the gradients in the x and y directions. and Through the arctangent function The principal gradient direction of each pixel is calculated to obtain the gradient direction map. When training the offset of the deformable convolution, an additional loss function is used to minimize the angle between the learned offset vector and the corresponding principal gradient direction on the gradient direction map, thereby guiding the sampling points to offset along the texture direction of the feature.

[0043] In an optional embodiment, calculating the feature information entropy of the multi-scale fused features at each spatial location to generate an upper bound for limiting the offset magnitude of deformable convolution sampling points includes:

[0044] For any spatial location (i,j) on the multi-scale fused feature map, extract the channel feature vector. ;

[0045] After taking the absolute value of each element of the channel feature vector, L1 normalization is performed to obtain the probability distribution vector. ;

[0046] Calculate the feature information entropy of the location based on the probability distribution vector. Where C is the number of channels, Let c be the c-th component of the probability distribution vector;

[0047] The upper bound is set based on the feature information entropy. .

[0048] The receptive field of deformable convolution is adjusted based on the local information complexity of the feature map. Assume there is a 256-channel, 64×64-pixel multi-scale fused feature map. For any spatial location on the map, such as coordinates (10, 20), a feature vector of length 256 is extracted. The vector represents the response intensity of the location on different feature channels.

[0049] The absolute values ​​of all 256 elements of the vector are taken, and the sum of all absolute values ​​is calculated. The absolute value of each element is then divided by this sum; this process is called L1 normalization. The original feature vector is then transformed into a probability distribution vector with a sum of 1. Using the formula for calculating information entropy, the feature information entropy is calculated on the probability distribution vector. If the feature response is evenly distributed across all channels, the information entropy will be high, indicating that the feature uncertainty in the region is large; if the response is concentrated on a few channels, the information entropy will be low, indicating that the feature certainty is high. Performing this calculation on all 4096 locations on the feature map yields a 64×64 information entropy map. The information entropy map is then converted into an upper bound map U through a linear mapping, for example, mapping the entropy range from 0 to 8 to an upper bound of 1 to 5 pixels for the offset amplitude. Regions with high entropy values ​​correspond to larger upper bounds, allowing sampling points to offset further and explore a broader context, while conversely, limiting the offset to focus on a local area. Optionally, the feature information entropy can be normalized to obtain... At the same time, obtain the upper and lower bounds of the offset. , The upper bound of the feature information entropy is obtained through linear interpolation. .

[0050] In an optional embodiment, calculating the local gradient direction map of the multi-scale fused features to provide directional guidance for the sampling point offset of the deformable convolution includes:

[0051] The average value of the multi-scale fused features along the channel dimension is used to obtain a single-channel feature map;

[0052] The gradient operator is used to calculate the gradient of the single-channel feature map in the horizontal and vertical directions. and ;

[0053] Calculate the gradient angle of each pixel based on the gradients in the horizontal and vertical directions. This forms the local gradient direction pattern.

[0054] Structural orientation information is extracted from the feature map to guide the sampling points of deformable convolutions to shift along the direction of feature change. Taking a 256-channel, 64×64-pixel multi-scale fused feature map as an example, the multi-scale fused feature map is compressed into a single-channel map. By calculating the average of the 256 channel values ​​at each pixel location, a 64×64 single-channel grayscale feature map can be obtained, which integrates the feature responses of all channels.

[0055] The single-channel feature map is processed using the Sobel gradient operator, which is implemented using two 3×3 convolutional kernels. One kernel is used to detect horizontal edges and intensity changes, generating a horizontal gradient map. Another convolutional kernel is used to detect changes in the vertical direction, generating a vertical gradient map. Both gradient maps are 64×64 in size. For each pixel position i,j in the map, the vertical gradient value of that point is obtained by using the arctangent function atan2. and horizontal gradient value Calculate the gradient angle of the point. The angle represents the direction of change in the feature intensity of the point. Combining the gradient angles of all pixels yields a 64×64 local gradient direction map.

[0056] In an optional embodiment, the processing via deformable convolution with dual constraints to obtain features includes:

[0057] Predicting the initial 2D offset from the convolutional layer ;

[0058] The initial offset is adjusted according to the amplitude constraint to obtain the amplitude-constrained offset. Where U is the upper bound, To prevent division by zero of small constants;

[0059] During network training, the offset is minimized after the magnitude constraint. The gradient unit vector derived from the local gradient direction pattern The cosine distance loss between them guides the learning direction of the offset.

[0060] In a 3×3 deformable convolution, offsets need to be predicted for 9 sampling points. A standard convolutional layer is used to predict these offsets, outputting 18 channels, corresponding to 9 two-dimensional offset vectors. Assume an initial offset is predicted at a certain position. The vector is (4,3).

[0061] The upper bound value for the specified position is retrieved from the previously calculated upper bound graph U, assumed to be 3. The magnitude of the initial offset, i.e., the L2 norm, is 5. Since the magnitude of 5 is greater than the upper bound of 3, the offset needs to be scaled. A scaling factor of 0.6 is calculated. The initial offset is multiplied by this factor to obtain the offset after magnitude constraint. The vector is (2.4, 1.8) with an amplitude of exactly 3. The constrained offset is used for the forward propagation calculation of deformable convolution, that is, sampling is performed on the input feature map at a position offset from the center point by 2.4, 1.8.

[0062] From the local gradient direction pattern The gradient angle at the specified location is queried, assumed to be 30°. A unit gradient vector is then constructed from this. For vectors ( ) Calculate the offset after amplitude constraint. With gradient unit vector Cosine distance loss between The smaller the value of the loss function, the closer the directions of the two vectors are. By adding this loss term to the network's total loss function, and through backpropagation and gradient descent, the network is motivated to learn offsets that are not only of reasonable magnitude but also aligned with the direction of the feature gradient, thereby detecting structural features of defects.

[0063] The neural network is a deep convolutional neural network for weld surface defect segmentation, typically employing an encoder-decoder structure. The encoder part of the neural network can use a pre-trained ResNet50 network to extract feature maps at different levels, such as shallow and high-level feature maps. The decoder part progressively upsamples and fuses features from the corresponding encoder levels, employing the aforementioned cross-scale attention mechanism and a dual-constraint deformable convolutional module. The input to the neural network is a 512×512 pixel image of the weld surface. The output of the network is a single-channel defect probability map of the same size as the input. The training set consists of weld images and corresponding pixel-level labeled masks, with the mask indicating the location and shape of the defects.

[0064] S4. Upsample the features obtained above to the original image size, obtain a defect probability map through a pixel-level classifier, and perform threshold processing on the defect probability map to determine the location and contour of the weld surface defects.

[0065] The features processed by double-constrained deformable convolution are then subjected to a series of bilinear interpolation operations to enlarge the spatial resolution to be identical to the input image. A 1×1 convolutional layer is used to reduce the number of feature channels to 1, and then a sigmoid activation function is applied to map the output value of each pixel to a range of 0 to 1, generating a defect probability map. A global threshold, such as 0.5, is set, and each pixel in the probability map is traversed. Pixels with values ​​greater than 0.5 are classified as defect points, and those less than or equal to 0.5 are classified as background points, thereby generating a binary mask image that marks the location and outline of all weld defects.

[0066] In an optional embodiment, the step of thresholding the defect probability map to determine the location and contour of the weld surface defect includes:

[0067] The defect probability map is analyzed using the maximum inter-class variance method to calculate a segmentation threshold that maximizes the inter-class variance between defective and non-defective pixels.

[0068] The probability map is binarized according to the segmentation threshold to distinguish between the defect region and the background region, thereby generating a defect segmentation map.

[0069] The network outputs a single-channel defect probability map, with the same size as the input image, for example, 512×512 pixels. Each pixel in the map has a grayscale value between 0 and 1, representing the probability that the pixel belongs to a defect. Determining which pixels truly belong to defects requires an objective segmentation threshold.

[0070] The threshold is determined using the Otsu's algorithm, also known as the maximum inter-class variance method. The gray-level histogram of the statistical probability graph obtained from Otsu's algorithm is shown below. Figure 4 The process iterates through all possible thresholds, for example, from 0.01 to 0.99, with a step size of 0.01. For each candidate threshold, all pixels in the image are divided into two classes: background with a probability value below the threshold and foreground or defect with a probability value above the threshold. The inter-class variance (AVV) between the two classes is calculated, which measures the difference in grayscale distribution between the two classes of pixels. A threshold that maximizes the AVV is found. For example, calculations show that the AVV between the foreground and background is maximized when the threshold is 0.52, so 0.52 is selected as the segmentation threshold. The probability map is binarized based on this threshold, setting all pixels with values ​​greater than 0.52 to 1 (white), representing defects, and setting pixels with values ​​less than or equal to 0 to 0 (black), representing the background, thereby generating a binary map of defect location and contour.

[0071] In a second embodiment, the present invention also proposes a weld surface quality inspection system, comprising the following modules:

[0072] A generation module is used to acquire a weld surface image, input it into a backbone network to extract a shallow feature map; perform discrete wavelet transform on the shallow feature map, generate a frequency domain guide map based on the energy distribution of high-frequency components, and use the frequency domain guide map to perform channel weighting on the shallow feature map to obtain shallow enhancement features;

[0073] The fusion module is used to calculate the pixel-level correlation between high-level and low-level feature maps at adjacent scales in the upsampling path, using the high-level feature map as the query and the low-level feature map as the key and value, to generate cross-scale gating weights; the low-level feature map is weighted using the weights, and the weighted low-level feature map is fused with the high-level feature map to obtain multi-scale fused features;

[0074] The calculation module is used to process the multi-scale fusion features obtained above through deformable convolution with dual constraints to obtain features. The dual constraints include: amplitude constraint: calculating the feature information entropy of the multi-scale fusion features at each spatial location to generate an upper bound for limiting the offset amplitude of the deformable convolution sampling points; orientation constraint: calculating the local gradient direction map of the multi-scale fusion features to provide directional guidance for the offset of the sampling points of the deformable convolution.

[0075] The determination module is used to upsample the features obtained above to the original image size, obtain a defect probability map through a pixel-level classifier, and perform threshold processing on the defect probability map to determine the location and contour of the weld surface defects.

[0076] The exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0077] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for inspecting the surface quality of welds, characterized in that, Includes the following steps: A weld surface image is acquired and input into a backbone network to extract shallow feature maps. Discrete wavelet transform is performed on the shallow feature maps, and a frequency domain guide map is generated based on the energy distribution of high-frequency components. The shallow feature maps are then channel-weighted using the frequency domain guide map to obtain shallow enhancement features. In the upsampling path, for high-level feature maps and low-level feature maps at adjacent scales, the high-level feature map is used as the query and the low-level feature map is used as the key and value to calculate the pixel-level correlation between the two and generate cross-scale gating weights. The weights are used to weight the bottom-level feature map, and the weighted bottom-level feature map is fused with the high-level feature map to obtain multi-scale fused features; The multi-scale fusion features obtained above are processed by deformable convolution with dual constraints to obtain the features. The features obtained above are upsampled to the original image size, and a defect probability map is obtained through a pixel-level classifier. The defect probability map is then thresholded to determine the location and outline of the defects on the weld surface.

2. The method according to claim 1, characterized in that, The step of performing discrete wavelet transform on the shallow feature map to generate a frequency domain guiding map based on the energy distribution of high-frequency components includes: A second-order discrete wavelet transform is performed on each channel of the shallow feature map to obtain multiple high-frequency sub-bands; For each channel, the absolute values ​​of each high-frequency sub-band obtained by the second-order discrete wavelet transform are added point by point to obtain the energy response map of the channel; The initial energy map is obtained by averaging the energy response maps of all channels along the channel dimension. The initial energy map is normalized using an activation function to generate the frequency domain guide map.

3. The method according to claim 1, characterized in that, The step of using the high-level feature map as the query and the low-level feature map as the key and value to calculate the pixel-level correlation between the two and generate cross-scale gating weights includes: The high-level feature map and the low-level feature map are transformed in dimension to obtain the query matrix Q and the key matrix K. Transpose of the query matrix Q and the key matrix K Perform matrix multiplication and scaling to obtain a pixel-level correlation matrix; The cross-scale gating weights are obtained by normalizing along the key dimension of the correlation matrix using the Softmax function.

4. The method according to claim 1, characterized in that, The dual constraints include: amplitude constraint: calculating the feature information entropy of the multi-scale fusion feature at each spatial location to generate an upper bound for limiting the offset amplitude of the deformable convolution sampling points; and orientation constraint: calculating the local gradient direction map of the multi-scale fusion feature to provide directional guidance for the offset of the sampling points of the deformable convolution.

5. The method according to claim 4, characterized in that, The calculation of the feature information entropy of the multi-scale fused features at each spatial location, and the generation of an upper bound for limiting the offset amplitude of deformable convolution sampling points, includes: For any spatial location (i,j) on the multi-scale fused feature map, extract the channel feature vector; The absolute value of each element of the channel feature vector is taken and then L1 normalized to obtain the probability distribution vector. Calculate the feature information entropy of the location based on the probability distribution vector; The upper bound is set based on the feature information entropy.

6. The method according to claim 4, characterized in that, The calculation of the local gradient direction map of the multi-scale fused features provides directional guidance for the sampling point offset of deformable convolution, including: The average value of the multi-scale fused features along the channel dimension is used to obtain a single-channel feature map; The gradient operator is used to calculate the gradient of the single-channel feature map in the horizontal and vertical directions; The gradient angle of each pixel is calculated based on the gradients in the horizontal and vertical directions to obtain the local gradient direction map.

7. The method according to claim 1, characterized in that, The process of obtaining features through deformable convolution with dual constraints includes: Predicting the initial 2D offset from the convolutional layer ; The initial offset is adjusted according to the amplitude constraint to obtain the amplitude-constrained offset. Where U is the upper bound, To prevent division by zero of small constants; During network training, the offset is minimized after the magnitude constraint. The gradient unit vector derived from the local gradient direction pattern The cosine distance loss between them guides the learning direction of the offset.

8. The method according to claim 1, characterized in that, The step of thresholding the defect probability map to determine the location and contour of the weld surface defects includes: The defect probability map is analyzed using the maximum inter-class variance method to calculate a segmentation threshold that maximizes the inter-class variance between defective and non-defective pixels. The probability map is binarized according to the segmentation threshold to distinguish between the defect region and the background region, thereby generating a defect segmentation map.

9. A weld surface quality inspection system, characterized in that, Includes the following modules: A generation module is used to acquire a weld surface image, input it into a backbone network to extract a shallow feature map; perform discrete wavelet transform on the shallow feature map, generate a frequency domain guide map based on the energy distribution of high-frequency components, and use the frequency domain guide map to perform channel weighting on the shallow feature map to obtain shallow enhancement features; The fusion module is used to calculate the pixel-level correlation between high-level feature maps and low-level feature maps of adjacent scales in the upsampling path, using the high-level feature map as the query and the low-level feature map as the key and value, to generate cross-scale gating weights. The weights are used to weight the bottom-level feature map, and the weighted bottom-level feature map is fused with the high-level feature map to obtain multi-scale fused features; The computation module is used to process the multi-scale fusion features obtained above through deformable convolution with dual constraints to obtain features; The determination module is used to upsample the features obtained above to the original image size, obtain a defect probability map through a pixel-level classifier, and perform threshold processing on the defect probability map to determine the location and contour of the weld surface defects.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-8.

Images