A pump shell weld quality detection method based on image segmentation

Abstract

The application discloses a pump shell weld quality detection method based on image segmentation, generates a steady-state pump shell weld image stream under the driving of motion compensation; obtains a domain-adaptive DINOv2 visual embedding feature map; generates a semantic morphological segmentation map through adaptive pyramid fusion and cross-scale attention operation on the domain-adaptive DINOv2 visual embedding feature map; generates a semantic-texture fusion mask; combines a pixel-level confidence map to implement uncertainty weighted optimization on the semantic-texture fusion mask to obtain a weld defect instance map; generates an interpretable texture anomaly heat map; and through multi-view supplementary shooting or manual review, supplementary pump shell weld image data is obtained and the steady-state pump shell weld image stream is updated. The application enables the system to continuously maintain accurate positioning of the pixel-level segmentation boundary in a weakly-labeled or even unlabeled migration scene, effectively avoiding boundary drift and area missed detection of the segmentation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pump shell, in particular to a pump shell weld quality detection method based on image segmentation. BACKGROUND

[0002] With the development of intelligent manufacturing and industrial internet, online quality detection of pump shell weld components puts forward higher requirements on production efficiency and quality traceability. The existing pump shell weld quality segmentation detection method mainly depends on traditional supervised semantic segmentation network or image processing algorithm based on feature engineering. The existing method usually requires a large number of high-quality manual annotation samples, and lacks adaptability to working condition changes, light disturbance and material surface treatment differences, resulting in serious segmentation boundary drift, region missing detection and false detection problems in cross-batch and cross-equipment deployment, which is difficult to stably support production line automation detection and dynamic quality judgment.

[0003] The existing industrial segmentation detection method often only relies on global or single scale semantic features when dealing with complex surface textures of metal welds, ignoring the effective differentiation of small pores, slag inclusions and fine crack fine texture defects. The conventional segmentation network is difficult to obtain robust representation in weak annotation or small sample environment, lacks adaptive identification and processing mechanism for high uncertainty or low confidence segmentation region, and limits the traceability of segmentation results and the automatic judgment ability of quality level. SUMMARY

[0004] One object of the present application is to provide a pump shell weld quality detection method based on image segmentation. The present application enables the system to continuously maintain the accurate positioning of the pixel-level segmentation boundary in the weak annotation or even no annotation migration scene, effectively avoiding the boundary drift and region missing detection of the segmentation result.

[0005] According to the pump shell weld quality detection method based on image segmentation of the present application, the method comprises the following steps:

[0006] Collecting pump shell weld image data, generating a steady-state pump shell weld image stream under the driving of motion compensation;

[0007] Inputting the steady-state pump shell weld image stream into the improved DINOv2 visual backbone network to obtain a domain-adaptive DINOv2 visual embedding feature map;

[0008] Constructing a semantic morphology branch to generate a semantic morphology segmentation map by performing adaptive pyramid fusion and cross-scale attention operation on the domain-adaptive DINOv2 visual embedding feature map;

[0009] Constructing a deep texture contrast branch to generate a structured contrast sample by applying directional filtering and bandpass frequency domain disturbance to the steady-state pump shell weld image stream, and outputting a deep texture contrast embedding map through contrast learning;

[0010] The semantic morphological segmentation graph is input into a post-fusion segmentation module in combination with a deep texture contrast embedding graph to generate a semantic-texture fusion mask;

[0011] A pixel-level confidence graph is generated using Monte Carlo dropout inference according to the semantic-texture fusion mask, and the semantic-texture fusion mask is subjected to uncertainty weighted optimization in combination with the pixel-level confidence graph to obtain a weld defect instance graph;

[0012] Based on the weld defect instance graph and the pixel-level confidence graph, a defect area ratio index and a texture deviation index are calculated, and a quality level result is obtained according to a preset threshold to generate an interpretable texture abnormality heat map;

[0013] If the confidence of the pump shell weld image data in the quality level result is lower than the preset threshold, a re-inspection process is triggered, and supplementary pump shell weld image data is obtained through multi-angle retakes or manual review, and the steady-state pump shell weld image stream is updated.

[0014] Optionally, the generation of the steady-state pump shell weld image stream under the motion compensation driving includes:

[0015] An industrial vision camera is used to collect images of the pump shell weld area to form a set of pump shell weld original images;

[0016] Frame-to-frame stability evaluation and registration compensation are performed on the set of pump shell weld original images, and a local motion estimation quantity is calculated for the pump shell weld images between each frame.

[0017] A stable mapping function is constructed based on the local motion estimation quantity, the stable mapping function aligns the pump shell weld original image frames to a reference frame to obtain aligned pump shell weld image frames, and a steady-state pump shell weld image stream is constructed for all aligned pump shell weld image frames.

[0018] Optionally, the improved DINOv2 visual backbone network includes:

[0019] The morphological structure guided geometric region coding is performed on each pump shell weld image frame in the steady-state pump shell weld image stream, the pump shell weld image frame is input into an edge response guided module to generate a weld structure guided graph;

[0020] The pump shell weld image frame is divided into a plurality of patches through patch segmentation, and is mapped into a patch embedding sequence, the weld structure guided graph is divided in the same way as the patch segmentation, and a structure weight is matched for each patch, and all structure weights are combined into a structure weight sequence according to the patch order;

[0021] The structural weight sequence is integrated into the multi-head attention module of the DINOv2 as a weight modulation term, and a patch embedding sequence is combined to perform an attention function on each group of query vectors, key vectors and value vectors to construct a structure-guided attention function;

[0022] A difference between a pixel gray value of each pump shell weld image frame and a global gray mean value is calculated as an image brightness deviation metric, and brightness deviation metrics of all pixels are normalized to obtain a disturbance guide map;

[0023] In each visual coding layer, a local mean and a local standard deviation of the patch embedding vector are calculated, and the patch embedding vector is normalized and linearly adjusted using a value of the disturbance guide map corresponding to the patch position;

[0024] In the output of the improved DINOv2 visual backbone network, a region patch embedding vector related to the weld structure is selected to form a weld embedding feature subgraph, and a self-distillation method is used to align the full-image patch embedding vector with the weld embedding feature subgraph, and a distance between the non-weld region patch embedding vector and the mean value of the weld embedding feature subgraph is calculated as an embedding realignment loss;

[0025] Each pump shell weld image frame is input into the improved DINOv2 visual backbone network, and a domain adaptation DINOv2 visual embedding feature map is output.

[0026] Optionally, the semantic morphology branch comprises:

[0027] The domain adaptation DINOv2 visual embedding feature map is input into the semantic morphology branch module, and a feature map set of different scales is obtained for each preset scale through downsampling operation;

[0028] For the feature map set of different scales, each scale of feature map is restored to the original resolution through upsampling processing by adaptive feature fusion operation, an adaptive fusion weight is assigned to each scale, and the upsampling feature maps of all scales are multiplied by the corresponding adaptive fusion weights and then summed element by element to obtain a fused feature map.

[0029] The fused feature map is subjected to position enhancement and channel enhancement by a cross-scale attention module to generate an attention-enhanced feature map.

[0030] The attention-enhanced feature map is subjected to feature transformation by a decoder module composed of a plurality of convolution layers, normalization layers and nonlinear activation functions stacked alternately to map into a segmentation prediction map.

[0031] A Softmax operation is performed on the segmentation prediction map at each pixel position along the channel direction to obtain normalized probability values of each pixel belonging to each category;

[0032] The structural category with the maximum normalized probability value at each pixel position is selected as the predicted category label of the current pixel, and the category labels of all pixels are combined to form a semantic morphological segmentation map.

[0033] Optionally, the depth-texture contrast branch comprises:

[0034] For each pump shell weld image frame in the pump shell weld image stream, a Gabor kernel of a direction set is used to perform convolution operation to obtain a corresponding direction response map in all specified directions;

[0035] A discrete Fourier transform is performed on each pump shell weld image frame to obtain a frequency spectrum map of the corresponding pump shell weld image frame, a band-pass mask is constructed according to a preset lower radius and an upper radius of a band-pass, the band-pass mask and the frequency spectrum map are multiplied element by element at a corresponding frequency domain position to retain a middle-high frequency component and suppress low and high frequency noise, and an inverse transform operation is performed on the screened frequency spectrum map to obtain a band-pass frequency domain disturbance map.

[0036] The pump shell weld image frame is taken as an anchor view, the direction response map thereof is selected as a positive sample view, and a result obtained by randomly phase disturbing the band-pass frequency domain disturbance map is selected as a difficult negative sample view, and the three are combined into a structured contrast sample triplet.

[0037] Each view in the structured contrast sample triplet is divided into local patches of the same size, and each patch feature vector is extracted by inputting the local patches into a texture encoder respectively to obtain an anchor patch feature vector, a positive patch feature vector and a negative patch feature vector.

[0038] Based on the anchor patch feature vector, the positive patch feature vector and the negative patch feature vector of each triplet, a contrast learning loss is calculated, and an average value of the contrast learning losses calculated in all time frames and in all directions is taken as an overall contrast learning loss.

[0039] After the contrast learning optimization is completed, the patch feature vectors are recombined in the original spatial order to obtain a depth-texture contrast embedding map.

[0040] Optionally, the generating of the semantic-texture fusion mask comprises:

[0041] The semantic morphological segmentation graph and the depth texture contrast embedding graph are input into a post-fusion segmentation module, the post-fusion segmentation module respectively applies linear transformation on the channel dimension of the semantic feature graph and the texture embedding graph to obtain semantic feature representation and texture feature representation with consistent channel numbers;

[0042] The semantic feature representation and the texture feature representation are spliced in the channel dimension, and the spliced feature graph is input into a group of one-by-one convolution kernels to output a single-channel attention weight graph;

[0043] For each spatial position, the semantic feature representation and the texture feature representation are weighted and fused according to the value of the spatial position in the attention weight graph to obtain an attention-guided fusion feature graph;

[0044] A morphological consistency constraint module is constructed to measure the boundary alignment degree of the edge response of the semantic morphological segmentation graph and the attention-guided fusion feature graph of the semantic feature branch to obtain a boundary difference loss;

[0045] The attention-guided fusion feature graph is input into a fusion mask generator, and a fusion mask is output after multi-level feature transformation;

[0046] The fusion mask is subjected to Softmax normalization in each spatial position along the structure category direction to obtain the normalized probability of each spatial position belonging to each structure category, the structure category with the maximum normalized probability value is selected as the final category label of the spatial position, and the final category labels of all spatial positions form a frame-level fusion prediction graph. All frame-level fusion prediction graphs are arranged in sequence to generate a semantic-texture fusion mask.

[0047] Optionally, the post-fusion segmentation module into which the semantic morphological segmentation graph and the depth texture contrast embedding graph are input comprises:

[0048] The semantic-texture fusion mask is input into an inference model containing Monte Carlo dropout inference, the semantic-texture fusion mask is subjected to multiple independent forward propagation in the test stage, different random dropout masks are used in each forward propagation, and a specified number of semantic-texture fusion mask samples are obtained;

[0049] For all semantic-texture fusion mask samples, the variance of each spatial position and each structure category is calculated, and the variances are averaged to obtain a pixel-level confidence map;

[0050] The normalized probability graph of each spatial position in the semantic-texture fusion mask sample and the pixel-level confidence map are point-by-point weighted and fused to obtain an uncertainty weighted optimization mask;

[0051] For each spatial position of the uncertainty-weighted optimization mask, all structure categories are traversed, and the structure category with the largest numerical value is selected as the final segmentation output category label of the corresponding spatial position, and the final segmentation output category labels of all spatial positions are combined into a frame-level weld defect instance map.

[0052] Optionally, the quality level result includes:

[0053] For each frame, all spatial positions are traversed in combination with the weld defect instance map and the pixel-level confidence map, spatial positions with a category label of defect in the weld defect instance map are filtered out, and pixels with a pixel-level confidence map confidence higher than a confidence threshold in the spatial positions are further filtered out to determine effective defect regions, a defect area ratio index is calculated according to a ratio of the number of spatial positions of the effective defect regions to the total number of all spatial positions of the current frame;

[0054] The depth texture contrast embedding vectors corresponding to the spatial positions of all effective defect regions are extracted, the mean of the depth texture contrast embedding vectors is calculated as a texture center vector of the effective defect region, and a texture deviation index is calculated according to the average of the Euclidean distances between the depth texture contrast embedding vectors of all spatial positions in the effective defect region and the texture center vector;

[0055] A preset quality level evaluation rule is set, and the current frame is divided into different quality level results according to the defect area ratio index and the texture deviation index.

[0056] Optionally, the quality level result includes:

[0057] The excellent level is that the defect area ratio index is not greater than a first ratio index and the texture deviation index is not greater than a first texture deviation threshold;

[0058] The good level is that the defect area ratio index is greater than the first ratio index and not greater than a second ratio index or the texture deviation index is greater than the first texture deviation threshold and not greater than a second texture deviation threshold;

[0059] The poor level is that the defect area ratio index is greater than the second ratio index or the texture deviation index is greater than the second texture deviation threshold.

[0060] The beneficial effects of the present application are:

[0061] (1) The application introduces structure-guided attention, disturbance-guided dynamic normalization and local embedded self-distillation mechanism in the improved DINOv2 visual backbone network, realizes the spatial dynamic perception of the importance of the weld body, the weld toe, the spatter area and the base material background, combines multi-angle light disturbance adaptive normalization, improves the stability and segmentation precision of the model under cross-device, cross-batch, different surface treatment and complex light, breaks through the problem of sensitivity to labeled scarcity and domain change of traditional segmentation algorithm, enables the system to continuously maintain the accurate positioning of the pixel-level segmentation boundary in the weak labeling or even no labeling migration scene, and effectively avoids the boundary drift and area missing of the segmentation result.

[0062] (2) The application effectively improves the micro-defect resolution capability through deep texture contrast coding and structured triplet contrast learning, constructs a structured contrast sample generation mechanism jointly driven by directional Gabor filtering and band-pass frequency domain disturbance, designs a deep texture contrast coding branch for the defects such as fine-grained pores, slag inclusion and cracks on the surface of the pump shell weld, and adopts anchor-positive-negative triplet contrast loss for end-to-end texture separation training, which can actively enlarge the distribution interval of normal and abnormal texture features, greatly improve the detection sensitivity of the system to low contrast and early micro-defects, and significantly reduce the missing rate.

[0063] (3) In the post-fusion segmentation stage, the application performs pixel-level weighted fusion on the semantic branch and the texture branch through the attention gate mechanism, introduces morphological consistency constraint and Monte Carlo dropout inference, outputs a pixel-level confidence map, and forms a structured quality evaluation index based on the segmentation instance and the confidence result. All indexes are automatically mapped to multi-level quality results according to the preset threshold, and the re-inspection suggestion or heat map visualization is triggered for the high-uncertainty area. BRIEF DESCRIPTION OF DRAWINGS

[0064] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, which together with the embodiments of the application, are used to explain the application, and do not constitute a limitation of the application. In the drawings:

[0065] Fig. 1 A flowchart of a pump shell weld quality detection method based on image segmentation is provided for the application;

[0066] Fig. 2 A principle diagram of an improved DINOv2 visual backbone network visual embedding feature extraction and structure weight modulation attention mechanism in a pump shell weld quality detection method based on image segmentation is provided for the application. DETAILED DESCRIPTION

[0067] Example 1: Reference Figs. 1-2 A pump shell weld quality detection method based on image segmentation, comprising:

[0068] collecting pump shell weld seam image data, generating a steady-state pump shell weld seam image stream under motion compensation driving;

[0069] In this embodiment, the steady-state pump shell weld seam image stream generated under motion compensation driving includes:

[0070] An industrial vision camera is used to collect images of the pump shell weld seam area to form a set of pump shell weld seam original images.

[0071] The camera pose and exposure parameters are adjusted in real time during the collection process to ensure spatial alignment and brightness consistency between the collected frames. The set of pump shell weld seam original images is composed of pump shell weld seam image frames collected at multiple time points. Each pump shell weld seam image frame represents a pump shell weld seam image collected at a specific time. The number of collected frames is a preset number of frames.

[0072] Frame stability evaluation and registration compensation are performed on the set of pump shell weld seam original images to calculate the local motion estimate of the pump shell weld seam image between each frame.

[0073] The frame stability evaluation and registration compensation specifically includes: taking the first frame or a specified frame in the sequence of the set of pump shell weld seam original images as a reference frame, for each subsequent pump shell weld seam image frame, using a point cloud matching algorithm to extract key points and calculate the lateral inter-frame displacement and the longitudinal inter-frame displacement between the key points and the reference frame. The lateral inter-frame displacement and the longitudinal inter-frame displacement together constitute the local motion estimate, which quantifies the spatial displacement change of each frame relative to the reference frame. The lateral inter-frame displacement and the longitudinal inter-frame displacement are used to measure the inter-frame displacement of the pump shell weld seam image in the lateral and longitudinal directions, respectively.

[0074] A stable mapping function is constructed based on the local motion estimate. The stable mapping function aligns the pump shell weld seam original image frames to the reference frame to obtain aligned pump shell weld seam image frames. A steady-state pump shell weld seam image stream is constructed for all aligned pump shell weld seam image frames.

[0075] In embodiment 1, the stable mapping function takes the pump shell weld seam original image frame and the reference frame as input, performs spatial mapping based on the local motion estimate, and generates an aligned pump shell weld seam image frame after spatial mapping for each pump shell weld seam original image frame. The aligned pump shell weld seam image frame refers to a pump shell weld seam image frame that is spatially consistent with the reference frame after spatial mapping by the stable mapping function. All aligned pump shell weld seam image frames obtained by spatial mapping using the stable mapping function are sequentially combined in time sequence order to construct a steady-state pump shell weld seam image stream. The steady-state pump shell weld seam image stream refers to a set of pump shell weld seam image frames that have been spatially mapped and are consistent with the reference frame.

[0076] The improved DINOv2 visual backbone network is inputted with the steady-state pump shell weld image stream to obtain a domain-adaptive DINOv2 visual embedding feature map;

[0077] In the embodiment, the improved DINOv2 visual backbone network comprises:

[0078] The morphological structure-guided geometric region coding is performed on each pump shell weld image frame in the steady-state pump shell weld image stream, the pump shell weld image frame is inputted into an edge response guiding module, and a weld structure guiding map is generated.

[0079] In the embodiment 1, the edge response fusion algorithm is used to extract the weld region edge features of each pump shell weld image frame in the steady-state pump shell weld image stream, the pump shell weld image frame is inputted into a Canny operator to obtain an edge response map, and a segmentation algorithm is used to divide the weld image frame into four types of regions, i.e., a weld main body region, a weld toe edge, a spatter region and a base material background, according to the continuous and high-amplitude edge pixels in the edge response map and in combination with the prior set weld width, curvature and position constraints, a class label is assigned to each pixel position to form a weld structure guiding map corresponding to the pump shell weld image frame, and each pixel in the weld structure guiding map has a clear structure region class label.

[0080] Specifically, the segmentation algorithm performs binarization processing on the edge response map, filters out the weld main boundary of the weld main body and the surrounding structure, and expands from the weld main boundary along the orthogonal direction, marks the weld main body region according to the weld preset width and structure continuity constraint, divides the weld toe edge region according to the edge density and local gray scale gradient in the boundary band between the weld main body and the base material background, marks the spatter region in the isolated fragments outside the weld main body edge, which have an edge response intensity higher than a threshold value and an area smaller than a threshold value, and divides the remaining region into the base material background.

[0081] The pump shell weld image frame is divided into a plurality of patches by patch segmentation, and is mapped into a patch embedding sequence, the weld structure guiding map is divided in the same way as the patch segmentation, and a structure weight is matched for each patch, and all structure weights form a structure weight sequence in accordance with the patch order;

[0082] In embodiment 1, the patch segmentation module divides the pump shell weld image frame into multiple two-dimensional image patches of consistent size, no overlap or overlap according to the preset spatial division rule. Each patch covers a specific spatial area of the original pump shell weld image frame. For each patch, a linear projection operator is used to extract features from the patch to obtain a corresponding patch embedding vector. All patch embedding vectors are arranged in sequence according to the spatial order of the patches in the image frame to form a patch embedding sequence of the pump shell weld image frame. The weld structure guide image is divided into multiple structure region patches in a spatial division manner consistent with the patch segmentation. For each structure region patch, the distribution of pixels of each class in the covered region is counted. According to the proportion of the preset classes such as the weld main body region, the weld toe edge, the spatter edge and the base material background, a unique class label or class weight is assigned to the structure region patch according to the maximum class principle or the weighted average principle. The class weights of all structure region patches are arranged in sequence according to the spatial order to form a structure weight sequence of the pump shell weld image frame. Each structure weight in the structure weight sequence corresponds to each patch in the patch embedding sequence.

[0083] The value rule of the structure weight is: when the patch belongs to the weld toe edge, the weld toe edge region weight is assigned; when the patch belongs to the weld main body, the weld main body region weight is assigned; when the patch belongs to the spatter edge, the spatter edge region weight is assigned; when the patch belongs to the base material background, the base material background region weight is assigned; the weld toe edge region weight is greater than the weld main body region weight, the weld main body region weight is greater than the spatter edge region weight, and the spatter edge region weight is greater than the base material background region weight. The structure weight sequence is used to express the importance ranking of different structure regions in the pump shell weld image frame for weld quality detection.

[0084] ;

[0085] wherein, is the structure weight sequence, is the weld toe edge region weight, is the weld main body region weight, is the spatter edge region weight, is the base material background region weight.

[0086] The structure weight sequence is integrated into the DINOv2 multi-head attention module as a weight modulation term. The patch embedding sequence is combined to perform an attention function on each group of query vectors, key vectors and value vectors to construct a structure-guided attention function.

[0087] The query vector, the key vector and the value vector are generated by patch embedding sequence, the original attention score is obtained by point product calculation between the query vector and the key vector, the weighted attention score is obtained by multiplying the structural weight corresponding to each patch position with the original attention score of the position, the weighted attention scores of all patch positions are normalized to obtain the structure-guided attention function, the structure-guided attention function can strengthen and improve the significant representation ability of the DINOv2 visual backbone network to the high-risk area of the weld structure, and improve the segmentation and discrimination performance of the abnormal area of the pump shell weld.

[0088] ;

[0089] wherein, 、 、 is the query, key and value matrix in self-attention, is that the weight of each patch position in the structure weight sequence is copied to make each patch attention score correspond to the multiplication of the structural weight of the patch position in the same spatial structure of the original attention score, and a modulation weight matrix consistent with the spatial structure of the attention score matrix is formed, represents element-wise weighted multiplication.

[0090] The difference between the pixel gray value of each pump shell weld image frame and the global gray mean value is calculated as an image brightness deviation metric, and the brightness deviation metrics of all pixels are normalized to obtain a disturbance-guided mapping.

[0091] The image brightness deviation metric is used to describe the degree of illumination disturbance of each pixel position, and the disturbance-guided mapping reflects the sensitivity of the current position to the brightness disturbance at each pixel position.

[0092] In each visual encoding layer, the local mean and the local standard deviation of the patch embedding vector are calculated, and the patch embedding vector is standardized and linearly adjusted by using the value of the corresponding patch position of the disturbance-guided mapping.

[0093] The standardization process realizes dynamic adjustment of the normalization factor by subtracting the local mean, dividing by the local standard deviation, multiplying the scaling coefficient of the disturbance-guided mapping, and adding the bias coefficient of the disturbance-guided mapping, so that the patch embedding vector has adaptive adjustment ability under different illumination and reflection environments.

[0094] ;

[0095] wherein, represents the patch embedding vector of the i-th patch in the j-th layer, ,​​ a local mean and a local standard deviation of the patch embedding vectors, a perturbation guided mapping value corresponding to the position, 、 a learnable adjustment coefficient.

[0096] The region patch embedding vectors related to the weld structure are selected in the output of the improved DINOv2 visual backbone network, and a weld embedding feature subgraph is formed. The full-image patch embedding vectors are aligned with the weld embedding feature subgraph in a self-distillation manner. The distance between the non-weld region patch embedding vectors and the mean of the weld embedding feature subgraph is calculated as an embedding realignment loss.

[0097] In embodiment 1, according to the marking of the region in the weld structure guide map, the patch embedding vector set corresponding to the weld structure is selected, and all patch embedding vectors belonging to the region are sequentially combined to generate a weld embedding feature subgraph. All patch embedding vectors in the same frame are taken as a full-image patch embedding vector set. The patch embedding vectors in the non-weld structure region are aligned with the weld embedding feature subgraph. The Euclidean distance between each patch embedding vector in the non-weld structure region and the mean of all patch embedding vectors in the weld embedding feature subgraph is calculated. The Euclidean distance is taken as the embedding realignment loss of the patch embedding vector. The embedding realignment losses of all patch embedding vectors in the non-weld structure region are summed up as the target of self-distillation optimization. By minimizing the loss, the feature distribution of the full-image patch embedding vectors is actively aggregated to the weld structure related region, and the weld sensitive feature expression ability of the improved DINOv2 visual backbone network in the pump shell weld anomaly detection task is improved.

[0098] ;

[0099] wherein, is the weld embedding feature subgraph, and respectively represent the non-weld region and the weld region index set, represents the region average embedding.

[0100] Each pump shell weld image frame is input into the improved DINOv2 visual backbone network, and a domain adaptive DINOv2 visual embedding feature map is output.

[0101] In embodiment 1, the pump shell weld image frame is divided into a plurality of patches to obtain a patch embedding sequence, the weld structure guide map is encoded into a structure weight sequence, and the structure weight sequence is input into a multi-head attention module as a weight modulation item. The structure weight modulation attention mechanism is realized by weighting each attention calculation unit. For each patch embedding vector, a dynamic normalization process is performed according to the perturbation guide mapping of its corresponding position, and the patch embedding vector is standardized and linearly adjusted to realize the perturbation guide dynamic normalization mechanism. In the last TransformerBlock, the patch embedding vector group of the region related to the weld structure is selected to form a weld embedding feature subgraph to realize the local embedding distillation mechanism. After joint optimization processing of the structure weight modulation attention mechanism, the perturbation guide dynamic normalization mechanism and the local embedding distillation mechanism, the domain adaptive DINOv2 visual embedding feature map corresponding to the pump shell weld image frame is obtained.

[0102] The semantic morphology branch is constructed, and the domain adaptive DINOv2 visual embedding feature map is generated into a semantic morphology segmentation map through adaptive pyramid fusion and cross-scale attention operation.

[0103] In the embodiment, the semantic morphology branch includes:

[0104] The domain adaptive DINOv2 visual embedding feature map is input into the semantic morphology branch module, and a feature map set of different scales is obtained for each preset scale through downsampling operation.

[0105] The feature map set of different scales is used to express the embedding features of the pump shell weld image frame in multiple spatial ranges. The downsampling operation keeps the dimensions of the embedding features consistent, and the scale factors of all scales have the same unit, ensuring that the scale factors can be directly compared.

[0106] For the feature map set of different scales, each scale of the feature map is restored to the original resolution through upsampling processing by adaptive feature fusion operation, an adaptive fusion weight is assigned to each scale, and the upsampled feature maps of all scales are multiplied by their corresponding adaptive fusion weights and then element-wise summed to obtain a fused feature map.

[0107] The sum of the adaptive fusion weights of each scale is equal to 1, and the value range of each adaptive fusion weight is 0 to 1. The fused feature map represents a unified semantic embedding expression in a multi-scale context.

[0108] ;

[0109] Wherein, is the adaptive fusion weight corresponding to the scale . An upsampling function is used to restore the scale feature map to the original resolution, and S is a preset scale.

[0110] The fused feature map is subjected to position enhancement and channel enhancement by using a cross-scale attention module to generate an attention-enhanced feature map.

[0111] The cross-scale attention module includes a spatial attention branch and a channel attention branch. The spatial attention branch generates a spatial attention map, and the channel attention branch generates a channel attention map. The fused feature map is multiplied by the spatial attention map pixel by pixel and multiplied by the channel attention map channel by channel to obtain the attention-enhanced feature map. The attention-enhanced feature map simultaneously contains weighted expressions of spatial position information and channel category information.

[0112] The generation process of the spatial attention map is as follows: the feature values of each spatial position in the channel dimension of the fused feature map are averaged to obtain a two-dimensional spatial mapping. The two-dimensional spatial mapping is used to measure the overall response intensity of each spatial position in all channels. The two-dimensional spatial mapping is input into a spatial weight generation network composed of convolution, normalization and nonlinear activation function to output a spatial attention map consistent with the spatial resolution of the feature map. Each element of the spatial attention map is used to describe the importance weight of the corresponding spatial position in the overall feature expression.

[0113] The generation process of the channel attention map is as follows: the global average pooling is performed on all spatial positions of each channel in the spatial dimension of the fused feature map to obtain a one-dimensional channel response vector. The channel response vector is used to measure the response intensity of each channel in the global space. The one-dimensional channel response vector is input into a channel weight generation network composed of a fully connected layer, normalization and a nonlinear activation function to output a channel attention map consistent with the channel number of the feature map. Each element of the channel attention map is used to describe the importance weight of the corresponding channel in the overall feature expression.

[0114] The attention-enhanced feature map is subjected to feature transformation by using a decoder module composed of a plurality of convolution layers, normalization layers and nonlinear activation functions stacked alternately to map into a segmentation prediction map.

[0115] Each volume stack extracts local spatial features from the attention-enhanced feature map, the convolution operation keeps the spatial resolution of the output consistent with the input or adjusts the spatial resolution according to the set rule, each normalization layer normalizes the output features of the convolution layer, and each nonlinear activation function activates the normalized output element by element. The convolution layer, the normalization layer and the activation function are connected in series, so that the attention-enhanced feature map undergoes multi-level feature transformation and gradually fuses high-dimensional semantic information. Finally, a convolution operation adjusts the channel number to the total number of structure categories, and outputs a segmentation prediction map. The segmentation prediction map is consistent with the input pump shell weld image frame in spatial resolution, and equal to the total number of structure categories in the number of channels. The segmentation prediction map represents the unnormalized score of each pixel belonging to each structure category.

[0116] The Softmax operation is performed on the segmentation prediction map at each pixel position along the channel direction to obtain the normalized probability value of each pixel belonging to each category.

[0117] The output of the Softmax operation is used to measure the normalized probability of the pixel belonging to each structure category. The probability of each pixel belonging to a certain structure category is the ratio of the exponential value of the unnormalized score of the pixel in the segmentation prediction map to the sum of the exponential values of the unnormalized scores of all structure categories of the pixel.

[0118] The structure category with the maximum normalized probability value at each pixel position is selected as the predicted class label of the current pixel. The class labels of all pixels are combined to form a semantic morphology segmentation map. The semantic morphology segmentation maps of each frame are combined in time sequence to form a semantic morphology segmentation map sequence.

[0119] In embodiment 1, the normalized probability values of all structure categories corresponding to each pixel position are compared, and the structure category with the maximum probability value is selected as the predicted class label of the pixel. The above operation is repeated for all pixel positions in the pump shell weld image frame. Finally, the predicted class labels of each pixel are arranged in order according to their spatial positions to form a semantic morphology segmentation map of the pump shell weld image frame. The semantic morphology segmentation maps of all pump shell weld image frames are combined in time sequence and arranged in order to form a semantic morphology segmentation map sequence. Each pixel of the semantic morphology segmentation map has a unique structure category label, including the weld body, the weld toe contour, the heat-affected zone and the base material background category, representing the structure layout of the pump shell weld image frame.

[0120] A deep texture contrast branch is constructed to generate structured contrast samples by applying directional filtering and band-pass frequency domain disturbance to the steady-state pump shell weld image stream, and a deep texture contrast embedding map is output by contrast learning.

[0121] In this embodiment, the deep texture contrast branch includes:

[0122] For each pump shell weld image frame in the steady-state pump shell weld image stream, a Gabor kernel of a direction set is used for convolution operation to obtain corresponding direction response maps in all specified directions respectively;

[0123] The direction response map measures the response intensity of the texture feature of the pump shell weld image frame in the current direction, and each direction response map is obtained by convolving the pump shell weld image frame with the Gabor kernel corresponding to the direction, wavelength and scale.

[0124] ;

[0125] wherein, represents the Gabor kernel with the direction , the wavelength and the scale , is a pixel coordinate, is a convolution operator, is the pump shell weld image frame.

[0126] Performing discrete Fourier transform on each pump shell weld image frame obtains a frequency spectrum map corresponding to the pump shell weld image frame, a band-pass mask is constructed according to a preset lower radius and upper radius of the band-pass, and the band-pass mask and the frequency spectrum map are multiplied element by element at a corresponding frequency domain position to retain the medium and high frequency components and suppress low and high frequency noise, and inverse transform operation is performed on the screened frequency spectrum map to obtain a band-pass frequency domain disturbance map;

[0127] The band-pass mask is used to screen the components in the medium and high frequency band in the frequency spectrum, and the band-pass frequency domain disturbance map is used to enhance the sensitivity of the pump shell weld image frame to abnormal textures in the medium and high frequency band, and the medium and high frequency, the low frequency and the high frequency are set according to actual requirements.

[0128] Taking the pump shell weld image frame as an anchor view, selecting its direction response map as a positive sample view, and selecting the result after random phase disturbance of the band-pass frequency domain disturbance map as a difficult negative sample view, a structured contrast sample triplet is formed;

[0129] The positive sample view is obtained by directional filtering, the difficult negative sample view is generated by phase disturbance on the band-pass frequency domain disturbance map, the structured contrast sample triplet is used for the contrast learning stage, and the feature interval between normal and abnormal textures is actively expanded.

[0130] Each view in the structured contrast sample triplet is divided into local patches of the same size, and is respectively input into a texture encoder to extract a feature vector of each patch, so as to obtain an anchor patch feature vector, a positive patch feature vector and a negative patch feature vector;

[0131] In embodiment 1, each view in the structured contrastive sample triple is divided into multiple local patches with consistent size and spatial order according to a preset spatial division rule, each patch covers a specific spatial area of the corresponding view, and for each patch, the feature vector of the patch is extracted by inputting the pixel data of the patch into a same texture encoder, and the anchor patch feature vector, the positive patch feature vector and the negative patch feature vector corresponding to the spatial division order are obtained by repeating the division and encoding process for the anchor view, the positive sample view and the difficult negative sample view.

[0132] The texture encoder is obtained in the following manner: a neural network model with convolution feature extraction capability is initialized as the texture encoder, the texture encoder includes a plurality of convolution layers, normalization layers and nonlinear activation functions which are alternately stacked, and the parameters of the texture encoder are trained end-to-end through a contrastive learning process driven by the structured contrastive sample triple.

[0133] Based on the anchor patch feature vector, the positive patch feature vector and the negative patch feature vector of each triple, a contrastive learning loss is calculated, and the average of the contrastive learning losses calculated in all time frames and all directions is taken as the overall contrastive learning loss.

[0134] The contrastive learning loss is used to measure the similarity between the anchor patch feature vector and the positive patch feature vector and the difference between the anchor patch feature vector and the negative patch feature vector, and the overall contrastive learning loss is used to guide the reverse optimization of the texture encoder to make the anchor patch feature vector and the positive patch feature vector closer and the anchor patch feature vector and the negative patch feature vector farther.

[0135] ;

[0136] wherein, is the anchor patch feature vector, is the positive patch feature vector, is the negative patch feature vector, is a temperature scaling coefficient, and are the inner product similarities of the anchor patch feature vector and the positive and negative patch feature vectors, respectively.

[0137] The total loss is accumulated for all time indices t and direction indices k:

[0138] ;

[0139] wherein, T is the total number of frames of the steady-state pump shell weld image stream, and K is the total number of direction sets.

[0140] After the contrastive learning optimization is completed, the patch feature vectors are reorganized in the original spatial order to obtain a deep texture contrast embedding map.

[0141] In embodiment 1, the feature vectors of all patches are arranged in a two-dimensional grid according to the row and column order of the patches in the original image space, the feature vector of each patch is assigned to the area corresponding to it in the original image space, the seamless splicing and alignment of patch features are completed in the entire image space, and the deep texture contrast embedding map with the same spatial resolution as the pump shell weld image frame is generated by combining all patch feature vectors according to the spatial position. The embedding vector at each spatial position of the deep texture contrast embedding map corresponds to the local texture feature of the corresponding patch in the original pump shell weld image frame, ensuring that the deep texture contrast embedding map is completely corresponding to the pump shell weld image frame at the pixel level in terms of spatial expression.

[0142] After the semantic morphological segmentation map and the deep texture contrast embedding map are input into the post-fusion segmentation module, a semantic-texture fusion mask is generated.

[0143] In this embodiment, the semantic-texture fusion mask is generated, including:

[0144] The semantic morphological segmentation map and the deep texture contrast embedding map are input into the post-fusion segmentation module, and the post-fusion segmentation module applies linear transformation to the semantic feature map and the texture embedding map in the channel dimension respectively to obtain semantic feature representations and texture feature representations with consistent channel numbers.

[0145] The linear transformation is realized by applying a group of one-by-one convolution kernels to the feature vector of each spatial position, and all spatial positions share the same group of convolution kernel weights. The convolution operation does not change the spatial resolution but only adjusts the channel number, and the channel numbers of the semantic feature representations and the texture feature representations are unified to the fusion channel number.

[0146] The semantic feature representations and the texture feature representations are spliced in the channel dimension, and the spliced feature map is input into a group of one-by-one convolution kernels to output a single-channel attention weight map.

[0147] The attention weight map is normalized to between zero and one by the Sigmoid function, and the attention weight map at each spatial position represents the fusion proportion of the semantic feature branch. The one minus value of the spatial position represents the fusion proportion of the texture feature branch.

[0148] For each spatial position, according to the value of the spatial position in the attention weight map, the semantic feature representation and the texture feature representation are weighted and fused to obtain an attention-guided fusion feature map.

[0149] The weighted fusion mode is to multiply the semantic feature representation of the spatial position by the corresponding value of the attention weight map, and to multiply the texture feature representation of the spatial position by a minus corresponding value of the attention weight map, so that the spatial resolution and the number of channels of the attention-guided fusion feature map remain consistent with the original input.

[0150] The morphological consistency constraint module is constructed to perform boundary alignment measurement on the edge response of the semantic morphological segmentation graph of the semantic feature branch and the attention-guided fusion feature map, and obtain a boundary difference loss.

[0151]

[0152] wherein, the Sobel operator is represented by, the attention-guided fusion feature map is, the semantic morphological segmentation graph is.

[0153] The attention-guided fusion feature map is input into the fusion mask generator, and after multi-level feature transformation, a fusion mask is output.

[0154] In embodiment 1, the fusion mask generator is implemented by stacking a layer-by-layer structure, the fusion mask generator applies a group of convolution kernels to the input attention-guided fusion feature map at each spatial position, the window of the convolution kernel slides within the spatial neighborhood to realize spatial aggregation of local features, a normalization layer is connected after each group of convolution operations to normalize the channel of the convolution output, and a nonlinear activation function is connected after the normalization layer to enhance the feature expression ability, the convolution layer, the normalization layer and the nonlinear activation function are sequentially connected and multi-level stacked, the output of the previous stage is used as the input of the next stage to form a multi-level feature transformation path, and in the last convolution layer, the same number of convolution kernels as the number of structure categories is used to map the high-dimensional fusion features of each spatial position of the fusion feature map to the output of the structure category number dimension, each output channel corresponds to a structure category, the output feature map is consistent with the input pump shell weld image frame in spatial resolution, the number of channels is equal to the number of structure categories, and the fusion mask output of each spatial position is the unnormalized score of the position belonging to each structure category.

[0155] The fusion mask is subjected to Softmax normalization along the structure category direction at each spatial position to obtain the normalized probability of each spatial position belonging to each structure category, the structure category with the maximum normalized probability value is selected as the final class label of the spatial position, and the final class labels of all spatial positions form a frame-level fusion prediction map, and all frame-level fusion prediction maps are arranged in sequence to generate a semantic-texture fusion mask.

[0156] ​The pixel-level confidence map is generated according to the semantic-texture fusion mask using Monte Carlo dropout inference, and the semantic-texture fusion mask is subjected to uncertainty weighted optimization combined with the pixel-level confidence map to obtain a weld defect instance map.

[0157] In the embodiment, the semantic modality segmentation map and the depth texture contrast embedding map are input into a post-fusion segmentation module, which includes:

[0158] The semantic-texture fusion mask is input into an inference model containing Monte Carlo dropout inference, and the semantic-texture fusion mask is subjected to multiple independent forward propagations in the test phase, each forward propagation using a different random dropout mask to obtain a specified number of semantic-texture fusion mask samples.

[0159] In the embodiment, the inference model containing Monte Carlo dropout inference uses a deep neural network with Dropout structure as the inference model, and Dropout operation is inserted into each layer. The Dropout operation randomly sets the output of part of the neurons to zero with a preset probability, forming a deep neural network containing Monte Carlo dropout inference. In the inference phase, the Dropout operation is switched from being enabled only during training in the training phase to being continuously enabled in the inference phase. The Monte Carlo dropout inference of the Dropout operation is kept unchanged during forward propagation, ensuring that there is random inactivation of neuron output in each forward propagation. The inference model containing Monte Carlo dropout inference is obtained, so that the output generated by each forward propagation has randomness and uncertainty, realizing the Monte Carlo approximation sampling capability of the model prediction distribution. The same semantic-texture fusion mask is input in each inference process, and the inference model containing Monte Carlo dropout inference can output different fusion mask samples. After multiple sampling, semantic-texture fusion mask samples can be formed.

[0160] For all semantic-texture fusion mask samples, the variance of each spatial position and each structure category is calculated, and the average of the variances is calculated to obtain a pixel-level confidence map.

[0161] Each spatial position of the pixel-level confidence map is used to measure the stability of the semantic-texture fusion mask segmentation result at the current spatial position. The higher the pixel-level confidence map value at the spatial position, the more stable the segmentation result.

[0162] The normalized probability map of each spatial position in the semantic-texture fusion mask sample is point-by-point weighted fused with the pixel-level confidence map to obtain an uncertainty weighted optimization mask.

[0163] The normalized probability map is a result of performing a Softmax operation on each semantic-texture fusion mask sample in the category dimension, which is a probability value of being segmented into the corresponding category at each spatial position and each category, and the sum of all category probability values is 1. The numerical representation of each spatial position and each structure category of the uncertainty weighted optimization mask is a segmentation confidence response map after confidence weighting.

[0164] ;

[0165] wherein, is a spatial position in the semantic-texture fusion mask sample In the first class of normalized probability, is a segmentation confidence response map of the weighted structure category, is a pixel-level confidence map.

[0166] For each spatial position of the uncertainty weighted optimization mask, all structure categories are traversed, and the structure category with the largest numerical value is selected as the final segmentation output category label of the corresponding spatial position. The final segmentation output category labels of all spatial positions are combined into a frame-level weld defect instance map.

[0167] The frame-level weld defect instance map is consistent with the pump shell weld image frame in spatial resolution, and each spatial position has a unique category label. The frame-level weld defect instance map is sequentially arranged in time order to generate a pump shell weld defect instance sequence, and the pump shell weld defect instance sequence describes the defect spatial distribution and evolution process of the pump shell weld area in the entire image stream.

[0168] Based on the weld defect instance map and the pixel-level confidence map, a defect area proportion index and a texture deviation index are calculated, and a quality level result is obtained according to a preset threshold to generate an interpretable texture abnormality heat map.

[0169] In the embodiment, the quality level result obtained according to the preset threshold includes:

[0170] For each frame, all spatial positions are traversed in combination with the weld defect instance map and the pixel-level confidence map, and spatial positions with a category label of defect in the weld defect instance map are filtered out. Further, pixels with a pixel-level confidence map confidence higher than a confidence threshold in the spatial positions are filtered out to determine effective defect regions. According to the ratio of the number of effective defect regions to the total number of all spatial positions of the current frame, a defect area proportion index is calculated.

[0171] The binary mask of the effective defect area is used to identify whether each spatial position belongs to the defect category and is a high-confidence weld defect pixel at the same time, the defect area proportion index represents the distribution proportion of the high-confidence weld defect area of the frame in the overall image space, and the larger the defect area proportion index, the wider the distribution range of the high-confidence weld defect.

[0172] The depth texture contrast embedding vectors corresponding to the spatial positions of all effective defect areas are extracted, the mean of the depth texture contrast embedding vectors is calculated as the texture center vector of the effective defect area, and the texture deviation index is calculated according to the average of the Euclidean distances between the depth texture contrast embedding vectors of all spatial positions in the effective defect area and the texture center vector.

[0173] In embodiment 1, the depth texture contrast embedding vectors corresponding to the spatial positions of all effective defect areas are extracted from the depth texture contrast embedding map, and the depth texture contrast embedding vectors of all effective defect areas ensure that the texture feature analysis only covers the high-confidence real defect area, and the texture deviation index measures the dispersion degree of the texture features of the effective defect area.

[0174] The preset quality level evaluation rules are set, and the current frame is divided into different quality level results according to the defect area proportion index and the texture deviation index.

[0175] The Euclidean distances between the depth texture contrast embedding vectors of all spatial positions and the texture center vector of the defect area of the frame are calculated, and all spatial positions are normalized to generate an interpretable texture anomaly heat map.

[0176] The Euclidean distance is used to describe the degree of deviation of the texture features of each spatial position from the texture features of the defect center, and the interpretable texture anomaly heat map is used to highlight the deviation degree of the texture features of the weld area, which assists in weld quality analysis and defect visualization tracking.

[0177] In this embodiment, the determination rules of the quality level results include:

[0178] The excellent level is that the defect area proportion index is not greater than the first proportion index and the texture deviation index is not greater than the first texture deviation threshold.

[0179] The good level is that the defect area proportion index is greater than the first proportion index and not greater than the second proportion index, or the texture deviation index is greater than the first texture deviation threshold and not greater than the second texture deviation threshold.

[0180] The poor level is that the defect area proportion index is greater than the second proportion index or the texture deviation index is greater than the second texture deviation threshold.

[0181] The pump shell weld image data with a confidence level lower than a preset threshold in the quality grade result triggers a re-inspection process, and supplementary pump shell weld image data is obtained through multi-angle retakes or manual review and the steady-state pump shell weld image stream is updated.

[0182] In the welding production line of a certain batch of pump shells, the implementer configures the present application, and the system collects a pump shell weld image sequence numbered "PS2023121801", a total of 72 frames. During the collection process, due to the fine tuning of the on-site light source and the shot blasting treatment of the pump shell surface, strong reflection, high brightness bands and local shadows appear in some weld areas, and the weld boundary position is complex and variable. The system automatically adjusts the camera exposure parameters, uses the motion compensation algorithm to align the space of all frames, and finally generates a set of steady-state weld image stream.

[0183] After the steady-state image stream input detection process, the DINOv2 backbone network extracts the visual embedding features of each frame. Due to the presence of fine wire drawing lines and blurred weld toe contours in some weld areas of this batch of pump shells, the domain adaptation module automatically normalizes the embedding features based on historical light disturbance, clearly distinguishing the weld body, weld toe, spatter area, and base material background. In example 2, strong reflection bands appear in the lower left weld area of the 14th and 47th original images. After optimization by the structure-guided attention mechanism, the segmentation model accurately marks the weld body boundary without false segmentation.

[0184] The system automatically enters the depth texture contrast encoding link. For the 22nd frame, several weak contrast micro-holes are detected on the surface of the weld body. Traditional segmentation methods often fail to identify such subtle defects. After Gabor direction filtering of the frame image, the signal intensity of the micro-hole area is 0.41-0.56 in the response graph at 90° direction, while the normal area around it only has 0.12-0.17. After bandpass frequency domain disturbance, the system constructs anchor-positive-difficult-negative triplets samples, and in the feature space output by the texture encoder, the average embedding distance between the micro-hole and the normal area is improved to 0.39 (only 0.13 for traditional methods), significantly increasing the separation degree of defects and background by improving the DINOv2 visual backbone network.

[0185] The semantic morphology segmentation map and texture contrast embedding map of all frames are sent to the post-fusion segmentation module. Taking the 57th frame as an example, there is a boundary that is not clear and local reflection between the weld spatter area and the base material. In the fusion stage, the attention gating mechanism automatically assigns the weight of the area with a segmentation confidence level lower than 0.63 to the texture branch, and finally outputs the weld defect instance map. Three continuous micro-cracks (the longest major axis length is 15.7 pixels, and the width is 2.3 pixels) are detected, and the average overlap degree between the system segmentation boundary and the manual annotation boundary is 0.91.

[0186] The system performs Monte Carlo discard inference, and the pixel-level confidence map shows that the mean confidence of the main defect area is 0.81 and the maximum variance is 0.09. The system automatically triggers a re-inspection prompt for the low-confidence area and saves the corresponding abnormal heat map.

[0187] Based on the instance map and the confidence map of each frame, the system calculates that the average defect area ratio of the 72 frames in this batch is 2.3%, and the average texture deviation index is 0.15. The quality evaluation rule determines that the level is "good". Taking the 68th frame as an example, the system detects an early slag on the left side of the main weld with an area ratio of 0.72% and a texture deviation of 0.08, and the quality level is automatically determined as "excellent".

[0188] Comparative analysis shows that in the same batch of 16 special interference images, the traditional U-Net model (trained with 640 labeled data) has 7 false reports of spatter area, 4 misjudgments of the main weld boundary, and 3 missed detections of fine pores. The average IoU of segmentation is 0.82, and the defect detection rate is 81%. The method of the present application only has 2 slight missegmentation, the average IoU is improved to 0.91, and the defect detection rate is as high as 96%.

[0189] In terms of detection cycle, the traditional method has a single-frame processing delay of 72 milliseconds, while the method of the present application has a single-frame processing delay of 63 milliseconds. The system can meet the online detection needs of the production line at a rate of 60 pieces per minute.

[0190] The detection results are automatically archived by the MES system, and the structured evaluation indicators and texture anomaly heat maps are uploaded synchronously. Engineers can quickly lock the quality problem area according to the automatically pushed re-inspection frame ID and highlighted area, realizing closed-loop management and data traceability of online segmentation detection of pump shell welds.

[0191] The above description is only a preferred embodiment of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can make equivalent replacements or changes to the technical solutions and inventive concepts of the present application within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for detecting the quality of a pump shell weld based on image segmentation, characterized in that, The method comprises the following steps: S1, collecting pump shell weld image data, and generating a steady-state pump shell weld image stream under the driving of motion compensation; S2, inputting the steady-state pump shell weld image stream into an improved DINOv2 visual backbone network to obtain a domain-adapted DINOv2 visual embedding feature map; S3, constructing a semantic morphology branch, and generating a semantic morphology segmentation map through adaptive pyramid fusion and cross-scale attention operation on the domain-adapted DINOv2 visual embedding feature map; S4, constructing a deep texture contrast branch, applying directional filtering and band-pass frequency domain disturbance to the steady-state pump shell weld image stream to generate a structured contrast sample, and outputting a deep texture contrast embedding map through contrast learning; S5, inputting the semantic morphology segmentation map and the deep texture contrast embedding map into a post-fusion segmentation module, and generating a semantic-texture fusion mask by means of an attention gate mechanism and a morphology consistency constraint; S6, generating a pixel-level confidence map by using Monte Carlo dropout inference according to the semantic-texture fusion mask, and implementing uncertainty weighted optimization on the semantic-texture fusion mask combined with the pixel-level confidence map to obtain a weld defect instance map; S7, calculating a defect area proportion index and a texture deviation index based on the weld defect instance map and the pixel-level confidence map, and generating an interpretable texture abnormality heat map according to a preset threshold as a quality level result; S8, triggering a re-inspection process for pump shell weld image data with a confidence lower than a preset threshold in the quality level result, and obtaining supplementary pump shell weld image data through multi-view retakes or manual review and updating the steady-state pump shell weld image stream; The S2 comprises the following steps: performing morphology structure guided geometric region coding on each pump shell weld image frame in the steady-state pump shell weld image stream, inputting the pump shell weld image frame into an edge response guided module to generate a weld structure guided map; dividing the pump shell weld image frame into a plurality of patches and mapping the pump shell weld image frame into a patch embedding sequence, dividing the weld structure guided map in the same manner as the patch division, and matching a structure weight for each patch, and all structure weights are combined into a structure weight sequence according to the patch order; integrating the structure weight sequence into the DINOv2 multi-head attention module as a weight modulation term, combining the patch embedding sequence to execute an attention function on each group of query vectors, key vectors and value vectors to construct a structure guided attention function; calculating the difference between the pixel gray value of each pump shell weld image frame and the global gray mean value as an image brightness deviation measure, and performing normalization processing on the brightness deviation measures of all pixels to obtain a disturbance guided mapping; in each visual coding layer, the local mean and local standard deviation of the patch embedding vector are calculated, and the patch embedding vector is standardized and linearly adjusted by using the value of the corresponding patch position of the disturbance guided mapping; In the improved DINOv2 visual backbone network, the region patch embedding vectors related to the weld structure are selected from the output of the last TransformerBlock, and a weld embedding feature subgraph is formed. A self-distillation method is used to align the full-image patch embedding vectors with the weld embedding feature subgraph, and the distance between the patch embedding vectors in the non-weld region and the average of the weld embedding feature subgraph is calculated as the embedding re-alignment loss. Each pump shell weld image frame is input into the improved DINOv2 visual backbone network, and a domain-adaptive DINOv2 visual embedding feature map is output.

2. The method for detecting the welding quality of the pump shell based on image segmentation according to claim 1, characterized in that, The S1 includes the following steps: An industrial vision camera is used to capture images of the pump shell weld area to form a set of pump shell weld original images. Frame-to-frame stability evaluation and registration compensation are performed on the set of pump shell weld original images, and local motion estimation is calculated between each frame of pump shell weld images. Based on the local motion estimation, a stable mapping function is constructed to align the pump shell weld original image frames to the reference frame to obtain aligned pump shell weld image frames. A stable pump shell weld image stream is constructed based on all the aligned pump shell weld image frames.

3. The method for detecting the welding quality of the pump shell based on image segmentation according to claim 1, characterized in that, The S3 includes the following steps: The domain-adaptive DINOv2 visual embedding feature map is input into a semantic morphology branch module, and a feature map set of different scales is obtained for each preset scale through downsampling operation. For the feature map set of different scales, each scale of feature map is restored to the original resolution through upsampling processing by adaptive feature fusion operation, and an adaptive fusion weight is assigned to each scale. The upsampled feature maps of all scales are multiplied by their corresponding adaptive fusion weights and then element-wise summed to obtain a fused feature map. The fused feature map is subjected to position enhancement and channel enhancement using a cross-scale attention module to generate an attention-enhanced feature map. The attention-enhanced feature map is subjected to feature transformation using a decoder module to map it into a segmentation prediction map. A Softmax operation is performed on each pixel position of the segmentation prediction map along the channel direction to obtain normalized probability values of each pixel belonging to each class. The structure class with the maximum normalized probability value at each pixel position is selected as the predicted class label of the current pixel, and the class labels of all pixels are combined to form a semantic morphology segmentation map. The semantic morphology segmentation maps of each frame are combined in chronological order to form a semantic morphology segmentation map sequence.

4. The method for detecting the welding quality of the pump shell based on image segmentation according to claim 3, characterized in that, The S4 includes the following steps: For each pump shell weld image frame in the stable pump shell weld image stream, a Gabor kernel of a direction set is used for convolution operation to obtain corresponding direction response maps in all specified directions. A discrete Fourier transform is performed on each pump shell weld image frame to obtain a frequency spectrum map of the corresponding pump shell weld image frame. A bandpass mask is constructed based on the preset lower and upper radius of the bandpass, and the bandpass mask and the frequency spectrum map are multiplied element by element at the corresponding frequency domain position to retain the medium and high frequency components and suppress low and high frequency noise. An inverse transform operation is performed on the filtered frequency spectrum map to obtain a bandpass frequency domain disturbance map. With the pump shell weld image frame as the anchor view, a direction response graph thereof is selected as a positive sample view, and a result of a band-pass frequency domain perturbation graph after random phase perturbation is selected as a difficult negative sample view, to be combined into a structured contrast sample three-tuple; Each view in the structured contrast sample three-tuple is divided into local patches of the same size, and is respectively input into a texture encoder to extract a feature vector of each patch, so as to obtain an anchor patch feature vector, a positive patch feature vector and a negative patch feature vector; Based on the anchor patch feature vector, the positive patch feature vector and the negative patch feature vector of each three-tuple, a contrast learning loss is calculated, and an average value of the contrast learning losses calculated in all time frames and in all directions is taken as an overall contrast learning loss; After the contrast learning optimization is completed, the patch feature vectors are recombined in the original spatial order to obtain a deep texture contrast embedding graph.

5. The method for detecting the welding quality of the pump shell based on image segmentation according to claim 4, characterized in that, The S5 includes the following steps: The semantic morphology segmentation graph and the deep texture contrast embedding graph are input into a post-fusion segmentation module, the fusion segmentation module applies linear transformation to the semantic feature graph and the texture embedding graph in the channel dimension respectively to obtain a semantic feature representation and a texture feature representation with consistent channel numbers; The semantic feature representation and the texture feature representation are spliced in the channel dimension, and the spliced feature graph is input into a group of one-by-one convolution kernels to output a single-channel attention weight graph; For each spatial position, the semantic feature representation and the texture feature representation are weighted and fused according to the value of the spatial position in the attention weight graph to obtain an attention-guided fusion feature graph; A morphology consistency constraint module is constructed to perform boundary alignment measurement on the edge response of the semantic morphology segmentation graph of the semantic feature branch and the attention-guided fusion feature graph to obtain a boundary difference loss; The attention-guided fusion feature graph is input into a fusion mask generator, and a fusion mask is output after multi-level feature transformation; The fusion mask is subjected to Softmax normalization in the direction of the structure class at each spatial position to obtain the normalized probability of each spatial position belonging to each structure class, the structure class with the maximum normalized probability value is selected as the final class label of the spatial position, and the final class labels of all spatial positions form a frame-level fusion prediction graph, and all frame-level fusion prediction graphs are arranged in sequence to generate a semantic-texture fusion mask.

6. The method for detecting the welding quality of the pump shell based on image segmentation according to claim 5, characterized in that, The S6 includes the following steps: The semantic-texture fusion mask is input into an inference model containing a random dropout mechanism, the semantic-texture fusion mask is subjected to multiple independent forward propagations in the test stage, different random dropout masks are used in each forward propagation, and a specified number of semantic-texture fusion mask samples are obtained; For all semantic-texture fusion mask samples, the variance of each spatial position and each structure class is calculated, and the variances are averaged to obtain a pixel-level confidence map; The normalized probability graph of each spatial position in the semantic-texture fusion mask sample is weighted and fused with the pixel-level confidence map point by point to obtain an uncertainty weighted optimization mask. For each spatial position of the uncertainty-weighted optimization mask, all structure categories are traversed, and the structure category with the largest numerical value is selected as the final segmentation output category label of the corresponding spatial position. The final segmentation output category labels of all spatial positions are combined into a frame-level weld defect instance map.

7. The method for detecting the welding quality of the pump shell based on image segmentation according to claim 6, characterized in that, The S7 comprises the following steps: For each frame, the weld defect instance map and the pixel-level confidence map are combined to traverse all spatial positions, filter out spatial positions with a category label of defect in the weld defect instance map, and further filter out pixels with a pixel-level confidence map confidence value higher than a confidence threshold value to determine effective defect regions. According to the ratio of the number of spatial positions of the effective defect regions to the total number of all spatial positions of the current frame, a defect area ratio index is calculated. The depth-texture contrast embedding vectors corresponding to the spatial positions of all effective defect regions are extracted, the mean of the depth-texture contrast embedding vectors is calculated as the texture center vector of the effective defect region, and according to the average of the Euclidean distances between the depth-texture contrast embedding vectors of all spatial positions in the effective defect region and the texture center vector, a texture deviation index is calculated. A preset quality level evaluation rule is set, and according to the defect area ratio index and the texture deviation index, the current frame is divided into different quality level results.

8. The method for detecting the welding quality of the pump shell based on image segmentation according to claim 7, characterized in that, The determination rule of the quality level result comprises: The excellent level, the defect area ratio index is not greater than a first ratio index and the texture deviation index is not greater than a first texture deviation threshold value; The good level, the defect area ratio index is greater than the first ratio index and not greater than a second ratio index or the texture deviation index is greater than the first texture deviation threshold value and not greater than a second texture deviation threshold value; The poor level, the defect area ratio index is greater than the second ratio index or the texture deviation index is greater than the second texture deviation threshold value.

Images