A method and apparatus for weld seam structured light image segmentation based on a lightweight segmentation model

Abstract

This application discloses a method and apparatus for segmenting weld seam structured light images based on a lightweight segmentation model, relating to the field of image processing technology. The method includes: constructing an encoder based on a first convolutional block, a first multi-frequency domain multilayer perceptron module, and a saliency-guided downsampling layer; constructing a decoder based on a second multi-frequency domain multilayer perceptron module, a second convolutional block, and an edge-aware upsampling layer; constructing an initial image segmentation model based on the encoder, skip connection paths, the decoder, and a one-dimensional convolutional lightweight gating; training and validating the initial image segmentation model using an annotated weld seam structured light image dataset to obtain a target image segmentation model; and performing feature extraction, downsampling, upsampling, and feature fusion on the weld seam structured light image to be segmented using the target segmentation model to obtain a segmentation result image. This application not only improves the accuracy and efficiency of weld seam structured light image segmentation but also reduces the deployment cost of the model through a lightweight design.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for segmenting weld seam structured light images based on a lightweight segmentation model. Background Technology

[0002] The accuracy and real-time performance of weld inspection are crucial in industrial inspection, largely relying on the rapid and accurate segmentation of weld structured light images. Precise and efficient image segmentation technology is indispensable in the weld inspection process of industrial production. Weld structured light image segmentation methods are mainly divided into traditional methods and deep learning methods. Traditional methods include thresholding, edge detection, region growing, and clustering (such as K-Means); deep learning methods include fully convolutional networks (FCN), U-Net, lightweight networks (such as UNeXt and MobileNet-UNet), and Transformer-based models (such as Swin-UNet). Among them, UNeXt, as an image segmentation network based on convolution and multilayer perceptrons (MLP), aims to achieve image segmentation with fewer parameters and lower computational complexity, bringing new ideas to fields such as medical image segmentation.

[0003] Traditional methods, while computationally simple and fast, are sensitive to changes in lighting and prone to segmentation failure under noise interference such as welding spatter. They also lack the ability to handle multi-scale defects in irregular welds, making them unsuitable for precise measurement. Deep learning-based segmentation methods, while possessing strong feature extraction capabilities, are prone to losing spatial information during downsampling, resulting in coarse edge segmentation. Furthermore, they involve a large number of parameters and computational complexity, making them difficult to deploy on embedded devices. They also heavily rely on large-scale labeled data, leading to high labeling costs in industrial scenarios. When UNeXt is applied to weld image segmentation, the downsampling module over-compresses weld details, the MLP feature extraction module lacks the ability to capture the complex spatial structure features of the weld, the upsampling stage handles edge details coarsely, and the fusion of cross-layer features lacks low-level detail support, ultimately affecting segmentation accuracy. Therefore, improving the accuracy and efficiency of weld structured light image segmentation while reducing deployment costs is a pressing issue. Summary of the Invention

[0004] The purpose of this application is to provide a method and apparatus for segmenting weld structured light images based on a lightweight segmentation model, aiming to solve the technical problem of how to improve the accuracy and efficiency of weld structured light image segmentation and reduce deployment costs.

[0005] To achieve the above objectives, this application proposes a weld seam structured light image segmentation method based on a lightweight segmentation model, the method comprising:

[0006] An encoder is constructed based on the first convolutional block, the first multi-frequency domain multilayer perceptron module, and the saliency-guided downsampling layer.

[0007] The decoder is constructed based on the second multi-frequency domain multilayer perceptron module, the second convolutional block, and the edge-sensing upsampling layer.

[0008] An initial multi-frequency edge lightweight image segmentation model is constructed based on the encoder, the jump connection path, the decoder, and the one-dimensional convolutional lightweight gating.

[0009] The initial multi-frequency edge lightweight image segmentation model was trained and validated using the labeled weld seam structured light image dataset to obtain the target multi-frequency edge lightweight image segmentation model.

[0010] The target multi-frequency edge lightweight image segmentation model is used to extract features, downsample, upsample, and fuse features in the structured light image of the weld seam to be segmented, resulting in a segmentation image.

[0011] Furthermore, to achieve the above objectives, this application also proposes a weld seam structured light image segmentation device based on a lightweight segmentation model, the device comprising:

[0012] An encoder construction module is used to construct an encoder based on a first convolutional block, a first multi-frequency domain multilayer perceptron module, and a saliency-guided downsampling layer.

[0013] A decoder building module is used to build a decoder based on the second multi-frequency domain multilayer perceptron module, the second convolutional block, and the edge-aware upsampling layer.

[0014] The model building module is used to build an initial multi-frequency edge lightweight image segmentation model based on the encoder, the skip connection path, the decoder, and the one-dimensional convolutional lightweight gating.

[0015] The model training module is used to train and validate the initial multi-frequency edge lightweight image segmentation model using the labeled weld seam structured light image dataset, so as to obtain the target multi-frequency edge lightweight image segmentation model.

[0016] The image segmentation module is used to perform feature extraction, downsampling, upsampling, and feature fusion on the target multi-frequency edge lightweight image segmentation model to obtain a segmentation result image.

[0017] Furthermore, to achieve the above objectives, this application also proposes a weld seam structured light image segmentation device based on a lightweight segmentation model. The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the weld seam structured light image segmentation method based on the lightweight segmentation model described above.

[0018] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the weld seam structured light image segmentation method based on the lightweight segmentation model described above.

[0019] One or more technical solutions proposed in this application have at least the following technical effects:

[0020] First, the computer system constructs an encoder by sequentially concatenating three sets of convolutional blocks, adding a multi-frequency domain multilayer perceptron module after each set of convolutional blocks, and then adding a saliency-guided downsampling layer after each multi-frequency domain multilayer perceptron module. This structure enables the encoder to effectively extract hierarchical features of the input image and preserve details of key regions during downsampling, improving the efficiency and accuracy of feature extraction. Next, the computer system constructs a decoder by adding convolutional blocks between the two sets of multi-frequency domain multilayer perceptron modules and adding an edge-aware upsampling layer after each module. The edge-aware upsampling layer enhances the recovery of edge details, allowing for better preservation of weld edge information during upsampling, thereby improving the accuracy of the segmentation results. Then, the computer system connects the constructed encoder and decoder through skip connections, adding a one-dimensional convolutional lightweight gating module to each skip connection path to construct an initial multi-frequency edge lightweight image segmentation model. The combination of skip connections and one-dimensional convolutional lightweight gating modules allows the model to better balance feature information at different levels during feature fusion, further improving the accuracy and efficiency of segmentation. Subsequently, the computer system trained and validated the initial lightweight multi-frequency edge image segmentation model using the labeled weld seam structured light image dataset. The model parameters were adjusted using the training set, enabling it to learn how to accurately segment the weld seam region; the model performance was evaluated using the validation set, ensuring good segmentation results even on unseen data. This process allowed the model to adapt to different weld seam image features, improving its generalization ability and robustness. Finally, the computer system applied the trained and validated lightweight multi-frequency edge image segmentation model to the weld seam structured light image to be segmented, performing feature extraction, downsampling, upsampling, and feature fusion to obtain the segmentation result image. This entire process not only improved the accuracy and efficiency of weld seam structured light image segmentation but also reduced the model's deployment cost through its lightweight design, making it more suitable for real-time and efficient weld quality detection in industrial scenarios. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating an embodiment of the weld seam structured light image segmentation method based on a lightweight segmentation model in this application.

[0024] Figure 2 This is a schematic diagram of the multi-frequency MLP module provided in Embodiment 1 of the weld seam structured light image segmentation method based on a lightweight segmentation model in this application.

[0025] Figure 3 This is a schematic diagram of the high-frequency MLP module provided in Embodiment 1 of the weld seam structured light image segmentation method based on a lightweight segmentation model in this application.

[0026] Figure 4 This is a schematic diagram of the intermediate frequency MLP module provided in Embodiment 1 of the weld seam structured light image segmentation method based on a lightweight segmentation model in this application;

[0027] Figure 5 This is a schematic diagram of a low-frequency MLP module provided in Embodiment 1 of the weld seam structured light image segmentation method based on a lightweight segmentation model in this application.

[0028] Figure 6 This is a schematic diagram of the framework of the lightweight image segmentation model for target multi-frequency edges provided in Embodiment 1 of the weld structured light image segmentation method based on the lightweight segmentation model of this application.

[0029] Figure 7 This is a flowchart illustrating Embodiment 2 of the weld seam structured light image segmentation method based on a lightweight segmentation model in this application.

[0030] Figure 8 This is a schematic diagram of the module structure of the weld seam structured light image segmentation device based on a lightweight segmentation model according to an embodiment of this application. The objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0031] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application. To better understand the technical solutions of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the executing entity of the embodiments of this application can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or computer system capable of implementing the above functions. The following description uses a computer system as an example to illustrate this embodiment and the subsequent embodiments.

[0032] Based on this, embodiments of this application provide a method for segmenting weld structured light images based on a lightweight segmentation model, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the weld seam structured light image segmentation method based on a lightweight segmentation model according to this application.

[0033] In this embodiment, the weld seam structured light image segmentation method based on a lightweight segmentation model includes steps S10 to S50:

[0034] Step S10: Construct an encoder based on the first convolutional block, the first multi-frequency domain multilayer perceptron module, and the saliency-guided downsampling layer.

[0035] It's important to note that the first convolutional block is the fundamental building block of the encoder, consisting of a 2D convolutional layer (2DConv), a batch normalization layer (BN layer), and a rectified linear unit (ReLU) linearly connected in series. The 2D convolutional layer is a neural network layer used to extract local features from the input feature map. It generates new feature maps by sliding a convolution kernel (or filter) across the feature map, performing a convolution operation at each location. The batch normalization layer is a layer used to stabilize and accelerate neural network training. It reduces internal covariate shift (i.e., variations in the distribution of the input data) by normalizing the features of each mini-batch of data. The rectified linear unit is a non-linear activation function used to introduce non-linearity, enabling the model to learn more complex feature representations.

[0036] The first multi-frequency MLP module is a key module in the encoder used to enhance feature extraction capabilities. It consists of a Discrete Cosine Transform (DCT) module, a high-frequency MLP, a mid-frequency MLP, and a low-frequency MLP. The DCT module is used to decompose the feature map into different frequency components. Based on the Discrete Cosine Transform, it divides the input feature map into high-frequency, mid-frequency, and low-frequency parts. Please refer to [reference needed]. Figure 2 , Figure 2 This is a schematic diagram of the multi-frequency MLP module provided in Embodiment 1 of the weld seam structured light image segmentation method based on a lightweight segmentation model of this application. The DCT module first receives the input feature map and decomposes it into three feature maps of different frequencies: feature... Figure 1 ,feature Figure 2 and characteristics Figure 3 These correspond to high-frequency, mid-frequency, and low-frequency information, respectively. Subsequently, these feature maps are input into their respective high-frequency MLP, mid-frequency MLP, and low-frequency MLP for processing. The high-frequency MLP focuses on feature extraction. Figure 1 Detailed information in the middle frequency MLP processing characteristics Figure 2 To capture the transition features of the image, while low-frequency MLP analyzes the features. Figure 3 To understand the overall structure of the image. Processed features. Figure 4 ,feature Figure 5 and characteristics Figure 6 These features are then merged to reconstruct a multi-frequency fusion feature map containing rich frequency information. This process enhances the feature representation capability, enabling the model to more accurately understand and segment weld structured light images. Ultimately, this multi-frequency MLP module improves the accuracy and efficiency of image segmentation by integrating feature information from different frequencies.

[0037] High-frequency MLP is a submodule of the multi-frequency domain multilayer perceptron module, specifically designed for processing high-frequency feature maps. These feature maps primarily contain detailed image information, such as edges and textures. High-frequency MLP utilizes an orientation-adaptive rotational convolution module to dynamically adjust the angle of the convolution kernel based on the gradient direction of the input feature map, thereby extracting detailed features from different directions more accurately. Please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram of the high-frequency MLP module provided in Embodiment 1 of the weld seam structured light image segmentation method based on a lightweight segmentation model of this application. The diagram illustrates the internal structure and processing flow of the high-frequency MLP module, where features... Figure 2 As input, it is first decomposed into two branches for processing. The first branch processes the features. Figure 2Flattening the matrix along the width dimension yields a matrix of shape (HC) × W. Then, an MLP is used to process the features along the width dimension, maintaining the (HC) × W shape in the final result. The second branch then... Figure 2 The matrix is ​​flattened along the height dimension to obtain a (WC)×H matrix. The height dimension features are then processed using an MLP, maintaining the (WC)×H shape. Next, the outputs of these two branches are fused through an addition operation, and then passed through a non-linear cross-channel spatial convolutional layer to finally generate the features. Figure 5 This design allows the model to capture features in different directions and enhances the expressive power of features through nonlinear activation functions, thereby extracting high-frequency features such as edges and textures more accurately, which is crucial for improving the accuracy of weld seam structured light image segmentation.

[0038] The Intermediate Frequency (IF) MLP is a submodule of the Multi-Frequency Domain Multilayer Perceptron (MFLP) module, specifically designed for processing intermediate frequency feature maps. These feature maps primarily contain intermediate-scale information of the image, such as transition regions and medium-sized structures. The IF MLP enhances inter-channel information interaction and spatial feature extraction capabilities by using a nonlinear cross-channel spatial convolution module to nonlinearly aggregate multi-channel information before performing convolution operations. Please refer to [link / reference]. Figure 4 , Figure 4 This diagram illustrates the intermediate frequency (IF) MLP module provided in Embodiment 1 of the weld seam structured light image segmentation method based on a lightweight segmentation model according to this application. The diagram shows the processing flow of the IF MLP module, where features... Figure 1 As input, it is split into two parallel processing paths. In the first path, features... Figure 1 First, the features are reshaped into a matrix of shape (HC) × W, then feature processing is performed on the width dimension using an MLP, and the processed result retains the shape (HC) × W. In the second path, the features... Figure 1 The matrix is ​​reshaped into a (WC)×H matrix and then processed in the height dimension using an MLP, preserving the (WC)×H shape. These two processed matrices are then summed to fuse feature information from different dimensions. Next, the fused features are passed through an orientation-adaptive rotation convolutional layer, which further extracts features using non-linear cross-channel spatial convolution techniques, ultimately generating the feature set. Figure 4 This process enhances the expressive power of features by applying multilayer perceptrons in different directions and uses orientation-adaptive rotational convolution to accurately capture mid-frequency features, such as transition regions and medium-sized structures in the image, thus providing richer feature information for weld seam structured light image segmentation.

[0039] Low-frequency MLP is a submodule of the multi-frequency domain multilayer perceptron module, specifically designed for processing low-frequency feature maps. Low-frequency feature maps primarily contain macroscopic information about the image, such as overall structure, brightness distribution, and large shape contours. Low-frequency MLP uses a multi-directional shift convolution module to perform convolution operations on the input feature maps using convolution kernels in various directions, and then sums and fuses the results. Please refer to [link / reference]. Figure 5 , Figure 5 This diagram illustrates the low-frequency MLP module provided in Embodiment 1 of the weld seam structured light image segmentation method based on a lightweight segmentation model according to this application. The diagram shows the processing flow of the low-frequency MLP module, where features... Figure 3 As input, it is first split into two parallel processing paths. In the first path, features... Figure 3 The feature is reshaped into a matrix of shape (HC) × W, and then processed in the width dimension using an MLP, maintaining the (HC) × W shape. In the second path, the feature... Figure 3 The matrix is ​​reshaped into a (WC)×H matrix and then processed in the height dimension using an MLP, preserving the (WC)×H shape. These two processed matrices are then summed to fuse feature information from different dimensions. Next, the fused features are passed through a multi-directional shift convolutional layer, which further extracts features by applying convolutional kernels in different directions, ultimately generating the final feature set. Figure 6 This process enhances the expressive power of features by applying multilayer perceptrons and multidirectional shift convolutions in different directions, and is able to capture low-frequency features, such as the overall contour and large shape structure of the image, thus providing richer feature information for weld seam structured light image segmentation, which helps to improve the accuracy and robustness of segmentation.

[0040] The salience-guided lightweight downsampling layer is a module in the encoder used to perform downsampling operations. It quickly locates key and smooth regions in the feature map through saliency detection and adopts different downsampling strategies based on the characteristics of these regions.

[0041] The encoder is the core part of the image segmentation model. It consists of three sets of convolutional blocks connected in series, two multi-frequency domain multilayer perceptron modules, and a saliency-guided downsampling layer. Its main function is to progressively extract features from the input image and downsample them spatially, while increasing the number of channels in the feature map to create hierarchical features.

[0042] It is understandable that the encoder is obtained by connecting three sets of first convolutional blocks in series; adding a first multi-frequency domain multilayer perceptron module after each set of first convolutional blocks; and adding a saliency-guided downsampling layer after each set of first multi-frequency domain multilayer perceptron modules.

[0043] Step S20: Construct a decoder based on the second multi-frequency domain multilayer perceptron module, the second convolutional block, and the edge-aware upsampling layer.

[0044] It should be noted that the second multi-frequency domain multilayer perceptron module is a key component in the decoder. It is similar to the multi-frequency domain multilayer perceptron module in the encoder, but it is used in the decoding stage.

[0045] The second convolutional block is the basic building block in the decoder, similar to the convolutional block in the encoder, but used in the decoding stage.

[0046] Edge-Aware Lightweight Upsampling is a module in the decoder used to perform upsampling operations. It enhances the restoration of edge details by identifying edge regions in the image and using special interpolation methods in these regions.

[0047] The decoder is a crucial component of an image segmentation model. Its main function is to progressively restore the resolution of the original image from the low-resolution feature maps extracted by the encoder, generating the final segmentation result. The decoder consists of two multi-frequency domain multilayer perceptron modules, three sets of convolutional blocks, and an edge-aware upsampling layer. Through the collaborative work of these components, the decoder can effectively recover the detailed information of the image while maintaining edge sharpness and accuracy.

[0048] It is understandable that two multi-frequency domain multilayer perceptron modules are connected in series; three sets of second convolutional blocks are added after each multi-frequency domain multilayer perceptron module; and an edge-aware upsampling layer is added before each set of convolutional blocks to obtain a decoder.

[0049] Step S30: Construct an initial multi-frequency edge lightweight image segmentation model based on the encoder, the jump connection path, the decoder, and the one-dimensional convolutional lightweight gating.

[0050] It should be noted that a skip connection is a channel connecting the corresponding layers in the encoder and decoder. It is used to directly pass the shallow features extracted by the encoder to the corresponding layer in the decoder. This can effectively preserve and utilize the detailed information extracted in the encoder stage and prevent the loss of important features during downsampling.

[0051] One-dimensional convolution lightweight gating is a lightweight feature adjustment and filtering mechanism added to skip connection paths. It adjusts features through one-dimensional convolution operations and automatically balances the contribution of upsampling results and skip connection features through a gating mechanism.

[0052] The initial multi-frequency edge lightweight image segmentation model is an image segmentation model based on an encoder-decoder architecture, combining multi-frequency feature extraction, edge-aware upsampling, and lightweight gating techniques. This model extracts hierarchical features through the encoder, uses skip connection paths to pass shallow features, recovers image details through the decoder, and optimizes feature fusion through one-dimensional convolutional lightweight gating. This design aims to achieve efficient and accurate image segmentation with fewer parameters and lower computational complexity, making it particularly suitable for applications requiring high precision and real-time performance, such as weld seam structured light image segmentation.

[0053] As an example, the step of constructing an initial multi-frequency edge lightweight image segmentation model based on the encoder, the skip connection path, the decoder, and the one-dimensional convolutional lightweight gating includes: establishing skip connection paths between corresponding layers of the encoder and the decoder; adding a one-dimensional convolutional lightweight gating module to each skip connection path to obtain the initial multi-frequency edge lightweight image segmentation model.

[0054] First, the computer system identifies corresponding layers in the encoder and decoder and creates skip connection paths connecting these layers. This step ensures that the decoder can directly utilize the detailed information extracted from the encoder during upsampling, thereby enhancing the accuracy of the segmentation results. Second, the system inserts a one-dimensional convolutional lightweight gating module into each skip connection path. This module adjusts and filters the features transmitted by the skip connections, aiming to automatically balance the contributions from the decoder's upsampling results and the encoder's skip connection features, avoiding noise fusion and strengthening the model's understanding of image edge structures.

[0055] Step S40: The initial multi-frequency edge lightweight image segmentation model is trained and validated using the labeled weld seam structured light image dataset to obtain the target multi-frequency edge lightweight image segmentation model.

[0056] It should be noted that the weld seam structured light image dataset refers to a set of labeled weld seam structured light images and their corresponding labels. These images are acquired through structured light technology and can clearly display the geometric shape and texture features of the weld seam. The annotation work usually includes precise marking of the weld seam area so that the model can learn how to distinguish the weld seam from the background.

[0057] The Target Multi-Frequency Edge Lightweight Image Segmentation Model (MF - EdgeUNeXt Image Segmentation Model) is a final image segmentation model that has been trained and validated, capable of accurately segmenting weld seam structured light images. This model is built upon an initial multi-frequency edge lightweight image segmentation model and trained using a labeled weld seam structured light image dataset. The model learns how to extract features from the input image and accurately segment the weld seam region. Evaluation on a validation set optimizes the model's performance, ensuring its accuracy and robustness in practical applications. The target model is lightweight, suitable for deployment on embedded devices, while maintaining high segmentation accuracy, meeting the needs of applications such as industrial weld seam inspection. The model's simplified execution process is as follows: First, an input image of size H×W is input into the encoder; then, it is processed through the encoder's convolutional blocks, multi-frequency MLP module, and saliency-guided lightweight downsampling layer to create hierarchical features. The output feature maps of each level of the encoder have sizes of H / 2×W / 2×C1, H / 4×W / 4×C2, H / 8×W / 8×C3, H / 16×W / 16×C4, and H / 32×W / 32×C5, where C1=32, C2=64, C3=128, C4=160, and C5=256; finally, the segmentation result is output through the decoder.

[0058] Please refer to Figure 6 , Figure 6This is a schematic diagram of the target multi-frequency edge lightweight image segmentation model provided in Embodiment 1 of the weld seam structured light image segmentation method based on a lightweight segmentation model of this application. The model mainly consists of an encoder and a decoder. The input image has a size of H×W. It first enters the encoder part, where feature extraction and spatial size reduction are performed step by step through saliency-guided lightweight downsampling (marked as 'a'), while increasing the number of channels of the feature map from C1 to C5, and the feature map size is reduced from H×W to H / 32×W / 32. After each downsampling step, the feature map is processed through a two-dimensional convolutional layer (Conv2D), batch normalization (BN), and rectified linear unit (ReLU) activation layer. Skip connections (marked as 'b') are used to transfer feature information between the encoder and decoder to preserve important details. At the bottom of the encoder, the feature map passes through two multi-frequency domain multilayer perceptron (MF-MLP) modules to further enhance the feature extraction capability. The decoder gradually recovers the spatial size of the feature map through edge-aware lightweight upsampling (labeled 'c'), while reducing the number of channels until it is restored to a size close to the original H×W. In each upsampling step, the feature map is adjusted using a 1×1 convolutional lightweight gating (labeled 'G') and fused with the feature map in the encoder through skip connections. Finally, after processing by 2D convolutional layers, batch normalization, and a modified linear unit activation function (Conv 2D+BN+ReLU), the segmentation result map with a size of H×W is obtained. The entire model is designed to improve the accuracy and efficiency of weld seam structured light image segmentation and reduce deployment costs.

[0059] Understandably, firstly, the computer system divides the labeled weld seam structured light image dataset into training and validation sets according to a predetermined ratio, typically 80% for training and 20% for validation. This ensures the model learns on a sufficient amount of data while also evaluating its performance on independent datasets, avoiding overfitting. Secondly, the computer system uses the training set to train an initial multi-frequency edge lightweight image segmentation model. By inputting images from the training set into the model, the loss function between the model's output segmentation result and the annotations is calculated, such as cross-entropy loss. Then, the backpropagation algorithm is used to adjust the model's parameters based on the value of the loss function. This process is repeated for multiple iterations until the model's performance on the training set no longer improves or reaches a preset number of training epochs. The goal is for the model to learn how to accurately segment the weld seam region from the input image. Finally, the computer system validates the trained model using a validation set. Images from the validation set are input into the model to evaluate the consistency between the model's output segmentation results and the annotations. The model's performance is measured by calculating metrics such as accuracy, recall, and intersection-union ratio. Based on the results of these metrics, the model is finally evaluated and selected. The resulting lightweight image segmentation model for multi-frequency edges can accurately segment weld structured light images while maintaining lightweight design, meeting the accuracy and efficiency requirements of industrial inspection.

[0060] Step S50: The target multi-frequency edge lightweight image segmentation model is used to extract features, downsample, upsample, and fuse features in the structured light image of the weld seam to be segmented, so as to obtain the segmentation result image.

[0061] It should be noted that the structured light image of the weld to be segmented refers to the structured light image of the weld that needs to be segmented. These images are used in industrial inspection to evaluate welding quality, and the weld area needs to be accurately separated from the background using image segmentation technology for further analysis and measurement.

[0062] The segmentation result image refers to the image obtained after processing by a lightweight image segmentation model for target multi-frequency edges. In this image, the weld area and background area are clearly separated and marked. It is typically presented as a binary image, with the weld area marked as the foreground (usually white) and the background area marked as the background (usually black). The segmentation result image is used for subsequent weld quality assessment, such as measuring parameters like weld width, depth, and continuity, and is a key output in industrial weld inspection.

[0063] Understandably, the computer system first inputs the structured light image of the weld seam to be segmented into a lightweight image segmentation model for target multi-frequency edges. The encoder part of the model performs feature extraction and downsampling operations on the input image, gradually reducing the spatial resolution of the feature map while increasing the number of channels to extract layered features of the image. Next, the decoder part upsamples the low-resolution feature map output by the encoder, gradually restoring the spatial resolution of the feature map. Simultaneously, it fuses the shallow features from the encoder with the high-level features from the decoder through skip connection paths, enhancing the detail information of the feature map. Finally, the feature map after feature fusion generates a segmentation result image through the model's output layer. This image clearly separates the weld seam region from the background region, providing accurate segmentation results for subsequent weld seam quality assessment.

[0064] This embodiment provides a method for weld seam structured light image segmentation based on a lightweight segmentation model. First, the computer system constructs an encoder by sequentially connecting three sets of convolutional blocks, adding a multi-frequency domain multilayer perceptron module after each set of convolutional blocks, and then adding a saliency-guided downsampling layer after each multi-frequency domain multilayer perceptron module. This structure enables the encoder to effectively extract hierarchical features of the input image, retaining detailed information of key regions during downsampling while reducing redundant information, thus improving the efficiency and accuracy of feature extraction. Next, the computer system constructs a decoder by adding convolutional blocks between the two sets of multi-frequency domain multilayer perceptron modules and adding an edge-aware upsampling layer after each module. The edge-aware upsampling layer enhances the recovery of edge details, allowing for better preservation of weld seam edge information during upsampling, thereby improving the accuracy of the segmentation results. Then, the computer system connects the constructed encoder and decoder through skip connections, adding a one-dimensional convolutional lightweight gating module to each skip connection path to construct an initial multi-frequency edge lightweight image segmentation model. The combination of skip connection paths and a lightweight one-dimensional convolutional gating module enables the model to better balance feature information at different levels during feature fusion, further improving segmentation accuracy and efficiency. Subsequently, the computer system trains and validates the initial multi-frequency edge lightweight image segmentation model using an annotated weld seam structured light image dataset. The model parameters are adjusted using the training set, allowing the model to learn how to accurately segment weld seam regions; the model performance is evaluated using the validation set, ensuring good segmentation results even on unseen data. This process allows the model to adapt to different weld seam image features, improving its generalization ability and robustness. Finally, the computer system applies the trained and validated target multi-frequency edge lightweight image segmentation model to the weld seam structured light image to be segmented, performing feature extraction, downsampling, upsampling, and feature fusion to obtain the segmentation result image. The entire process not only improves the accuracy and efficiency of weld seam structured light image segmentation but also reduces the model's deployment cost through its lightweight design, making it more suitable for real-time and efficient weld quality detection in industrial scenarios.

[0065] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 7 , Figure 7 This is a flowchart illustrating the second embodiment of the weld seam structured light image segmentation method based on a lightweight segmentation model according to this application. The target multi-frequency edge lightweight image segmentation model includes an encoder, a decoder, a skip connection path, and a one-dimensional convolutional lightweight gating module. Step S50 of the weld seam structured light image segmentation method based on the lightweight segmentation model includes steps S51 to S55:

[0066] Step S51: The encoder performs convolution, feature extraction, and downsampling on the structured light image of the weld seam to be segmented to obtain a layered feature map.

[0067] It should be noted that hierarchical feature maps refer to a series of feature maps with different resolutions and numbers of channels obtained by performing multiple convolution and downsampling operations on the input image during the encoder stage of an image segmentation model. These feature maps gradually decrease in spatial dimension while gradually increasing in the number of channels, thus forming a multi-layered feature representation. Each layer of feature maps contains information about the input image at different scales, from the details of the original image to more abstract high-level features.

[0068] As an example, the encoder includes a first convolutional block, a first multi-frequency domain multilayer perceptron module, and a saliency-guided downsampling layer. The hierarchical feature map includes a primary feature map, a multi-frequency feature map, and a downsampled feature map. The step of obtaining the hierarchical feature map by performing convolution, feature extraction, and downsampling on the structured light image of the weld seam to be segmented through the encoder includes: performing two-dimensional convolution, normalization, and nonlinear activation on the structured light image of the weld seam to be segmented through the first convolutional block to obtain a primary feature map; performing discrete cosine transform decomposition and frequency adaptive feature extraction on the primary feature map through the first multi-frequency domain multilayer perceptron module to obtain a multi-frequency feature map; and performing key region detection and adaptive pooling on the multi-frequency feature map through the saliency-guided downsampling layer to obtain a downsampled feature map.

[0069] The primary feature map refers to the feature map processed by the first convolutional block. It contains basic feature information of the input image, including local texture and edge information. The multi-frequency feature map refers to the feature map processed by the first multi-frequency domain multilayer perceptron module. It contains feature information of different frequency components and can capture high-frequency details (such as edges and textures) and low-frequency structures (such as overall shape and background information) in the image. The downsampled feature map refers to the feature map processed by the saliency-guided downsampling layer. It has a lower spatial resolution than the multi-frequency feature map but retains important feature information of key regions. In weld seam structured light image segmentation, the downsampled feature map effectively reduces redundant information in the feature map through key region detection and adaptive pooling operations, while preserving the key features of the weld seam region.

[0070] First, the computer system performs a two-dimensional convolution operation on the structured light image of the weld seam to be segmented using a first convolutional block, followed by normalization and nonlinear activation to generate a primary feature map. This series of operations aims to extract the basic features of the image and enhance its representational power. Next, the system inputs the primary feature map into a first multi-frequency domain multilayer perceptron module. In this module, discrete cosine transform decomposition is performed on the primary feature map, dividing it into components of different frequencies. Then, frequency-adaptive feature extraction is performed on these components to obtain a multi-frequency feature map. This process can more comprehensively capture the details and structural information in the image. Finally, the computer system processes the multi-frequency feature map through a saliency-guided downsampling layer. It first detects key regions in the image, then performs adaptive pooling based on the characteristics of these key regions, ultimately obtaining a downsampled feature map. This step aims to reduce the spatial resolution of the feature map while retaining important feature information of key regions, providing optimized feature representations for subsequent segmentation tasks, thereby improving the accuracy and efficiency of weld seam structured light image segmentation and reducing deployment costs.

[0071] As an example, the first multi-frequency domain multilayer perceptron module includes a discrete cosine transform module, a high-frequency processing unit, an intermediate-frequency processing unit, and a low-frequency processing unit. The step of performing discrete cosine transform decomposition and frequency adaptive feature extraction on the primary feature map through the first multi-frequency domain multilayer perceptron module to obtain a multi-frequency feature map includes: performing discrete cosine transform decomposition on the primary feature map through the discrete cosine transform module to obtain high-frequency components, intermediate-frequency components, and low-frequency components; performing orientation-adaptive rotational convolution on the high-frequency components through the high-frequency processing unit to obtain orientation-sensitive features; performing nonlinear cross-channel spatial convolution on the intermediate-frequency components through the intermediate-frequency processing unit to obtain channel interaction features; performing multi-directional shift convolution on the low-frequency components through the low-frequency processing unit to obtain multi-scale structural features; and fusing the orientation-sensitive features, the channel interaction features, and the multi-scale structural features along the channel dimension to obtain a multi-frequency fused feature map.

[0072] The Discrete Cosine Transform (DCT) module is a component of the first multi-frequency domain multilayer perceptron module. It decomposes the input primary feature map into components of different frequencies, including high-frequency, mid-frequency, and low-frequency components. Through DCT, this module effectively separates the detailed information, transitional information, and macroscopic structural information in the feature map, providing a foundation for subsequent frequency-adaptive feature extraction. The high-frequency processing unit, also a component of the first multi-frequency domain multilayer perceptron module, is specifically designed to process high-frequency components. Through orientation-adaptive rotational convolution, it dynamically adjusts the angle of the convolution kernel according to the gradient direction of the input feature map, thereby extracting detailed features from different directions more accurately.

[0073] The intermediate frequency (IF) processing unit is a component of the first multi-frequency domain multilayer perceptron module, specifically designed for processing IF components. It enhances inter-channel information interaction and spatial feature extraction capabilities by performing nonlinear cross-channel spatial convolution operations, nonlinearly aggregating multi-channel information before convolution. The low-frequency (LFM) processing unit is also a component of the first multi-frequency domain multilayer perceptron module, specifically designed for processing LFM components. It performs multi-directional shift convolution operations, using convolution kernels in various directions to convolve the input feature map and then summing and fusing the results.

[0074] High-frequency components refer to the high-frequency information contained in the feature map after Discrete Cosine Transform (DCT) decomposition. They mainly reflect detailed features in the image, such as edges, textures, and rapidly changing regions. Mid-frequency components refer to the mid-frequency information contained in the feature map after DCT decomposition. They mainly reflect transitional features in the image, such as medium-scale structures and gradual changes in regions. Low-frequency components refer to the low-frequency information contained in the feature map after DCT decomposition. They mainly reflect the macroscopic structure and overall shape of the image. In weld seam structured light image segmentation, low-frequency components help capture the overall contour and background information of the weld seam, providing global contextual information for the model.

[0075] Orientation-sensitive features refer to features obtained after processing by the high-frequency processing unit. These features can capture detailed information from different directions in the image. Channel interaction features refer to features obtained after processing by the mid-frequency processing unit. These features are generated through nonlinear cross-channel spatial convolution operations, which can enhance the information interaction between channels and the spatial feature extraction capability. This helps the model better understand the complex spatial relationships of features, thereby more accurately capturing the transition information and medium-scale structure of the weld area. Multi-scale structural features refer to features obtained after processing by the low-frequency processing unit. These features are generated through multi-directional shift convolution operations, which can enrich the directional diversity of feature extraction.

[0076] First, the computer system performs Discrete Cosine Transform (DCT) decomposition on the primary feature map using a DCT module, decomposing the feature map into three components: high-frequency, mid-frequency, and low-frequency. This process is based on the mathematical formula of DCT and is achieved by calculating the discrete cosine coefficients of each pixel, thus obtaining three components containing information of different frequencies for subsequent targeted processing of features at different frequencies. Second, the system performs orientation-adaptive rotational convolution on the high-frequency components, obtaining the gradient through the convolution operation.

[0077]

[0078]

[0079] in, Indicates the coordinates in the input feature map Pixel value at; This represents the horizontal convolution kernel of the Sobel operator; This represents the convolution kernel of the Sobel operator in the vertical direction; This represents the convolution operation; Represents the input feature map In coordinates The point is connected to a Sobel convolution kernel in the horizontal direction. The horizontal gradient obtained from convolution; Represents the input feature map In coordinates The point is filled with a Sobel convolution kernel in the vertical direction. The vertical gradient obtained from convolution. The gradient direction is calculated from the input feature map. :

[0080]

[0081] in, Represents the input feature map In coordinates The gradient direction at that point.

[0082] Statistically analyze the gradient direction distribution and select the rotation angle. Divide the gradient direction into several intervals, count the number of pixels in each interval, and select the center angle corresponding to the interval with the larger number of pixels as the rotation angle.

[0083] gradient direction The range of values ​​is divided into N intervals. , , , .

[0084] Let the first The pixel values ​​within each interval are For each pixel on the feature map According to its gradient direction Determine its interval:

[0085]

[0086] in, It is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise.

[0087] Select the interval with the most pixels, let's say this interval is the [number of pixels]. The interval has a central angle of . This angle is the rotation angle.

[0088] Rotation convolution is performed based on the selected angle. After obtaining the selected rotation angle, the convolution kernel is rotated by the corresponding angle according to the rotation convolution process, and the convolution operation is performed on the input feature map.

[0089] Let the convolution kernel be ,in and These are the coordinates of the elements within the convolution kernel. Rotating the convolution kernel around its center... Angle, the rotated convolution kernel is It can be calculated using the rotation matrix:

[0090]

[0091] Then input feature map Perform convolution operations on the top layer to obtain the output feature map. :

[0092]

[0093] Then, the system performs nonlinear cross-channel spatial convolution on the intermediate frequency components. The specific operation steps are as follows:

[0094] (1) Perform a nonlinear aggregation operation on the input feature map to aggregate the information of multiple channels into a single channel feature.

[0095] Input feature map , shape ,in It refers to the batch size. It is the number of channels. and These are the height and width of the feature map, respectively.

[0096] ① Squaring the eigenvalues ​​of each channel: for each element ,calculate Among them, lowercase letters It is typically used as an index or iteration variable to represent a specific position in a corresponding dimension. For example, in middle:

[0097] Represents the first in the batch Each sample has values ​​ranging from 1 to... ; Representing the Each channel has a value range from 1 to... ; Represents height The first Pixel position; The first representing the width Pixel position;

[0098] ② Summation along the channel dimension (first dimension): ;

[0099] ③ Calculate the average of the sum of squares: ;

[0100] ④ The square root of the average value is used to obtain the single-channel feature map. : .

[0101] (2) For the single channel obtained in (1) Perform spatial convolution. Let the convolution kernel be... , shape ,in It is the size of the convolution kernel. This is the number of output channels.

[0102]

[0103] (3) Output results. After the spatial convolution operation on S2, the final output feature map is obtained. , Shape , and It is determined by parameters such as stride and padding of the convolution.

[0104] Next, the system performs multi-directional shift convolution on the low-frequency components. The specific operation steps are as follows:

[0105] (1) Define different directions Convolution kernels can be categorized by direction: left, right, top, bottom, top right, and bottom right.

[0106]

[0107]

[0108]

[0109] (2) Input feature map Perform convolution operations in different directions:

[0110]

[0111]

[0112]

[0113] (3) Add and fuse, the output feature map is :

[0114]

[0115] Finally, the system fuses the obtained direction-sensitive features, channel interaction features, and multi-scale structural features along the channel dimension to obtain a multi-frequency fusion feature map. This fusion process can combine the advantages of different frequency components, providing a more comprehensive and accurate feature representation for subsequent segmentation tasks, thereby improving the accuracy and efficiency of weld seam structured light image segmentation and reducing deployment costs.

[0116] As an example, the step of performing key region detection and adaptive pooling on the multi-frequency feature map through the saliency-guided downsampling layer to obtain a downsampled feature map includes: compressing the channels of the multi-frequency feature map through the saliency-guided downsampling layer to obtain a compressed feature map; summing the absolute values ​​of the channels of the compressed feature map to obtain a single-channel saliency map; dividing the saliency map into key regions and smooth regions according to a preset saliency threshold; performing average pooling on the key regions to obtain key region features; performing max pooling on the smooth regions to obtain smooth region features; concatenating the key region features with the smooth region features to obtain a pooled feature map; multiplying the multi-frequency feature map element-wise with the single-channel saliency map to obtain a residual feature map; and adding the pooled feature map to the residual feature map after one-dimensional convolution to obtain a downsampled feature map.

[0117] Compressed feature maps refer to feature maps that have undergone saliency-guided downsampling layer channel compression processing.

[0118] A single-channel saliency map is a single-channel image obtained by summing the absolute values ​​of each channel in a compressed feature map. It reflects the saliency of each location in the feature map; a larger value indicates that the location is more important. In weld seam structured light image segmentation, single-channel saliency maps are used to quickly locate key regions with rich textures and large gradient changes, such as the edges of weld seam stripes.

[0119] The preset saliency threshold is a threshold used to divide a single-channel saliency map into critical regions and smooth regions. It is preset based on the distribution of the saliency map.

[0120] Key regions refer to areas in a single-channel saliency map where the saliency value is higher than a preset saliency threshold. These regions typically contain important features in the image, such as the edges and textures of welds.

[0121] Smooth regions refer to areas in a single-channel saliency map where the saliency value is lower than or equal to a preset saliency threshold. They typically contain less detailed information, such as smooth portions of the background or weld seams.

[0122] Key region features refer to the features obtained after performing average pooling on key regions. Average pooling retains important features by calculating the mean information of key regions, avoiding the loss of multi-peak details.

[0123] Smooth region features refer to the features obtained by max pooling a smooth region. Max pooling achieves efficient dimensionality reduction by selecting the maximum value of the smooth region while preserving background information.

[0124] Pooling feature maps are feature maps obtained by concatenating features from key regions and smooth regions. They combine feature information from both key and smooth regions, providing a more comprehensive feature representation for subsequent feature fusion and upsampling operations.

[0125] The residual feature map is a feature map obtained by multiplying the multi-frequency feature map with the single-channel saliency map element by element. It retains important information from the original feature map through saliency weighting.

[0126] First, the computer system performs channel compression on the multi-frequency feature map using a saliency-guided downsampling layer. Specifically, this involves convolving the multi-frequency feature map with a 1×1 convolution kernel to reduce the number of channels, thus obtaining a compressed feature map. This process aims to reduce computational complexity while preserving important feature information. Next, the system sums the absolute values ​​of each channel in the compressed feature map to obtain a single-channel saliency map. This map quickly locates key and smooth regions in the image based on the magnitude of the saliency values, as shown in the following formula:

[0127]

[0128] in, This is a generated single-channel saliency map, in which a larger value indicates that the corresponding region is more important; It is the input feature map; For input feature maps Perform a 1×1 convolution operation; the purpose of this operation is to compress the number of channels. This refers to taking the absolute value of the result of a 1×1 convolution; This indicates summing over all channels; It is the number of channels in the feature map.

[0129] Then, based on a preset saliency threshold, the system divides the saliency map into critical regions and smooth regions. Specifically, regions in the saliency map above the threshold are marked as critical regions, and regions below or equal to the threshold are marked as smooth regions. Critical regions contain important texture and edge information, while smooth regions contain less detailed information. Average pooling is performed on the critical regions, specifically calculating the average value of each local region within the critical region to obtain the critical region features, thus preserving the mean information of the critical regions and avoiding the loss of multi-peak details. Max pooling is performed on the smooth regions, specifically selecting the maximum value of each local region within the smooth region to obtain the smooth region features, achieving efficient dimensionality reduction while preserving background information. Afterward, the system concatenates the key region features and smooth region features, specifically by concatenating the two feature maps along the channel dimension to obtain a pooled feature map. This feature map combines the feature information of both the critical and smooth regions, providing a more comprehensive feature representation for subsequent operations. Simultaneously, the system element-wise multiplies the multi-frequency feature map with the single-channel saliency map to obtain the residual feature map. Specifically, each element of the multi-frequency feature map is multiplied with the corresponding element of the single-channel saliency map, retaining important information from the original feature map through saliency weighting. Finally, the system adds the pooled feature map to the residual feature map after one-dimensional convolution. Specifically, the residual feature map is first subjected to a one-dimensional convolution operation to adjust its channel count and feature representation, and then added element-wise with the pooled feature map to obtain the downsampled feature map, as shown in the following formula:

[0130]

[0131] in, This refers to the feature map after downsampling; This refers to the significance plot The generated binary mask, in the key area, =1; In the smooth region, =0; This indicates element-wise multiplication; This refers to performing an average pooling operation on the input feature map; This refers to the input feature map Perform max pooling operation; when When =1 (critical region), (1- )=0 indicates that max pooling is not used; when When =0 (smooth region), (1- `)=1` indicates that max pooling is used. This process enhances the key region information in the feature map through feature map matrix operations rather than simple numerical calculations, improving the accuracy and efficiency of segmentation and reducing deployment costs.

[0132] As an example, the first convolutional block includes a two-dimensional convolutional layer, a batch normalization layer, and a modified linear unit activation layer. The step of performing two-dimensional convolution, normalization, and nonlinear activation on the structured light image of the weld seam to be segmented through the first convolutional block to obtain a primary feature map includes: extracting spatial features from the structured light image of the weld seam to be segmented through the two-dimensional convolutional layer to obtain an initial convolutional feature map; performing channel normalization on the initial convolutional feature map through the batch normalization layer to obtain a normalized feature map; and performing nonlinear transformation on the normalized feature map through the modified linear unit activation layer to obtain a primary feature map.

[0133] The initial convolutional feature map refers to the original output of the two-dimensional convolutional layer after extracting spatial features from the input weld structured light image. This feature map directly carries the low-level visual features such as local texture and edges extracted by the convolutional kernel. However, since it has not been standardized, its numerical distribution may be unstable due to uneven lighting, welding spatter and other industrial noise.

[0134] Normalized feature maps refer to the results of channel standardization of the initial convolutional feature maps by the batch normalization layer. By adjusting the mean and variance of feature values ​​according to the channel dimension, common brightness fluctuation interference in industrial scenarios (such as strong arc reflection) is eliminated, making the feature distribution more stable.

[0135] First, the computer system uses a two-dimensional convolutional layer to extract spatial features from the structured light image of the weld seam to be segmented. Specifically, a predefined convolutional kernel is slid across each position in the image, and a weighted sum is performed on the pixel at each position and its neighboring pixels to obtain an initial convolutional feature map. This process captures local texture and edge information in the image. Second, the system performs channel normalization on the initial convolutional feature map using a batch normalization layer. Specifically, the mean and standard deviation of each feature channel are calculated, and then the feature values ​​of each channel are normalized to ensure the normalized feature map has zero mean and unit variance. This process helps stabilize the training process, reduces internal covariate bias, and accelerates model convergence. Finally, the system performs a nonlinear transformation on the normalized feature map using a modified linear unit activation layer. Specifically, each element in the normalized feature map is processed using the ReLU function, retaining positive values ​​and setting negative values ​​to zero, resulting in a primary feature map. This process introduces nonlinear factors, enabling the model to learn more complex feature representations and enhancing its expressive and generalization abilities.

[0136] Step S52: Input the hierarchical feature map into the one-dimensional convolutional lightweight gating module according to the jump connection path to obtain the jump connection feature map.

[0137] It should be noted that the skip connection feature map refers to the output of the encoder's hierarchical feature map passed through the skip connection and then adjusted by one-dimensional convolutional lightweight gating. In essence, it is the dynamic fusion result of the encoder's shallow features (such as edges and textures) and the decoder's high-level semantic features.

[0138] As an example, the step of inputting the hierarchical feature map into the one-dimensional convolutional lightweight gating module according to the skip connection path to obtain the skip connection feature map includes: concatenating the hierarchical feature map and the upsampled feature map of the corresponding layer of the decoder along the channel dimension according to the skip connection path to obtain a first concatenated feature map; convolving the first concatenated feature map through the one-dimensional convolutional lightweight gating module to obtain a hierarchical-upsampled compressed feature map; performing a nonlinear transformation on the hierarchical-upsampled compressed feature map through the Sigmoid activation function to generate a first gating weight map; multiplying the first gating weight map element-wise with the upsampled feature map to obtain a first weighted upsampled feature map; performing a numerical inversion operation on the first gating weight map to obtain a first inverse gating weight map; multiplying the first inverse gating weight map element-wise with the hierarchical feature map to obtain a weighted hierarchical feature map; and adding the first weighted upsampled feature map with the weighted hierarchical feature map to obtain the skip connection feature map.

[0139] The first spliced ​​feature map refers to the encoder's hierarchical feature map ( ) and decoder upsampled feature map ( The channel cascading results of the ) provide a fusion basis for gating screening by preserving complementary information of original spatial details (such as weld edges) and high-level semantics (such as weld area contours) by splicing along the channel dimension.

[0140] Layered-upsampled compressed feature maps refer to the intermediate representation of the first concatenated feature map after being compressed through a 1×1 convolution with lightweight gating. Essentially, it aggregates features across channels using convolutional kernels, reduces redundant information, and extracts common patterns of cross-layer feature interactions.

[0141] First gating weight map ( The weight matrix (range [0,1]) is generated by sigmoid activation from the hierarchical upsampled compressed feature map. Each pixel weight represents the importance of the decoder's upsampled features at that location. High-weight regions (such as weld seam stripes) enhance detail delivery, while low-weight regions (such as smooth backgrounds) suppress noise. The formula is as follows:

[0142]

[0143] in, This is the upsampling result. It is a jump within the same level. This represents a one-dimensional convolution operation. This represents the Sigmoid activation function, which normalizes the output values ​​to the range (0,1) and uses them as the weights for gating.

[0144]

[0145] in, This indicates element-wise multiplication. It is the final output that combines upsampling results and skip connection features after gating.

[0146] The first weighted upsampled feature map refers to the element-wise product of the first gated weight map and the decoder upsampled feature map (formula: Through spatial adaptive weighting, more high-frequency details are retained in key areas such as weld edges, providing a foundation for refined reconstruction for subsequent fusion.

[0147] The first inverse gating weight map refers to the result of inverting the values ​​of the first gating weight map (formula: Its weight distribution complements the original gating, with low-weight regions (such as the background) corresponding to high inverse weights, ensuring that the integrity of the encoder's hierarchical features is not excessively suppressed.

[0148] The weighted hierarchical feature map refers to the element-wise product of the first inverse gating weight map and the encoder hierarchical feature map (formula: The key is to retain the original features that have been weakened in the gating weight graph (such as small weld structures that are misclassified as noise) to prevent the loss of effective information.

[0149] First, the computer system stacks the hierarchical feature maps from the encoder and the upsampled feature maps from the corresponding layers of the decoder along the channel dimension, based on the skip connection paths, to obtain the first concatenated feature map. This process combines detailed information from the encoder and upsampled information from the decoder, providing rich contextual information for subsequent feature fusion. Second, the system performs convolution operations on the first concatenated feature map using a one-dimensional convolutional lightweight gating module. Specifically, it uses a one-dimensional convolution kernel to convolve the concatenated feature map, adjusting the number of channels and compressing it to obtain a hierarchical-upsampled compressed feature map. This operation reduces computational complexity while preserving key feature information. Then, the system maps each element in the hierarchical-upsampled compressed feature map to the (0,1) interval using the Sigmoid activation function, generating the first gating weight map. This weight map controls the importance of each feature in the upsampled feature map, achieving weighted feature fusion. Next, the system multiplies each element in the first gating weight map with the corresponding element in the upsampled feature map to obtain the first weighted upsampled feature map. This weighting operation preserves important features and suppresses unimportant features. Simultaneously, the system inverts each element in the first gating weight map (i.e., subtracts the element's value from 1) to obtain the first inverse gating weight map. This inverse operation generates a weight map used to control the importance of each feature in the hierarchical feature map. Then, the system multiplies each element in the first inverse gating weight map with the corresponding element in the hierarchical feature map to obtain a weighted hierarchical feature map. This weighting operation further optimizes the feature fusion process. Finally, the system adds the first weighted upsampled feature map with the corresponding elements in the weighted hierarchical feature map to obtain a jump-connected feature map. This process, through feature map matrix operations rather than simple numerical calculations, enhances the key region information in the feature map, improves segmentation accuracy and efficiency, and reduces deployment costs.

[0150] Step S53: The downsampled feature map in the hierarchical feature map is subjected to directional weighted interpolation through the edge-aware upsampling layer of the decoder to obtain the upsampled feature map.

[0151] It should be noted that the upsampled feature map refers to the high-resolution feature representation generated by performing directional weighted interpolation on the low-resolution feature map through a lightweight upsampled layer with edge awareness. Its core value lies in dynamically restoring the key edge structures (such as stripe contours and weld wave boundaries) in the weld structured light image, solving the edge blurring problem caused by conventional bilinear interpolation, and providing geometric fidelity assurance for high-precision segmentation.

[0152] As an example, the step of performing directional weighted interpolation on the downsampled feature map in the hierarchical feature map through the edge-aware upsampling layer of the decoder to obtain the upsampled feature map includes: calculating the edge response intensity map of the downsampled feature map using the Sobel operator through the edge-aware upsampling layer of the decoder, wherein the downsampled feature map includes edge regions and smooth regions, the edge response intensity corresponding to the edge regions is greater than a preset intensity threshold, and the edge response intensity corresponding to the smooth regions is less than or equal to the preset intensity threshold; generating dynamic interpolation weights based on the edge response intensity map; performing directional weighted interpolation on the edge regions and bilinear interpolation on the smooth regions based on the dynamic interpolation weights to obtain an interpolation result map; and performing resolution upsampling on the interpolation result map to obtain the upsampled feature map.

[0153] The Sobel operator is a discrete differential operator used for quickly calculating the gradient intensity of an image. It locates regions of abrupt gray-level changes (such as weld edges) in a feature map through convolution operations, providing directional guidance for direction-weighted interpolation. Its horizontal kernel ( ) and vertical core ( The following are respectively:

[0154]

[0155] Edge response intensity map ( The feature map () refers to the single-channel response map generated after convolution with the Sobel operator, where pixel values ​​characterize the edge intensity at each location in the feature map. Edge regions refer to the set of pixels in the edge response map whose intensity is greater than a preset intensity threshold, corresponding to key structures in the weld seam structured light image (such as weld wave contours and fish-scale edges). Smooth regions refer to the set of pixels in the edge response map whose intensity is less than or equal to the preset intensity threshold, corresponding to the base material background and uniformly illuminated areas. The preset intensity threshold is the critical parameter for segmenting edges and smooth regions, set through experimental calibration.

[0156] Dynamic interpolation weights refer to a spatially adaptive weight matrix generated based on edge response intensity. In edge regions, the weights of pixels adjacent to the gradient direction are increased (e.g., horizontal edges emphasize horizontal interpolation), while smooth regions maintain conventional bilinear interpolation weights. The interpolated result map is an intermediate output obtained by initially upsampling a low-resolution feature map using dynamic interpolation weights. Its characteristics include: enhanced directionality in edge regions, preserving weld seam sharpness; and maintaining interpolation efficiency in smooth regions, avoiding redundant computation. The upsampled feature map is a high-dimensional feature representation of the interpolated result map after resolution upscaling (by a factor of 2), providing geometrically aligned high-resolution input for skip connection fusion.

[0157] First, the computer system uses the edge-aware upsampling layer of the decoder to perform convolution operations on the downsampled feature maps in the horizontal and vertical directions using the Sobel operator, obtaining edge intensity maps in both directions. Then, the edge intensity maps in these two directions are added element-wise to obtain a comprehensive edge response intensity map. The formula is as follows:

[0158]

[0159] in, , It is a horizontal / vertical Sobel kernel. It is the input downsampled feature map (low-resolution feature map).

[0160] This image can distinguish between edge regions and smooth regions in the feature map. This process provides a basis for subsequent interpolation operations, ensuring that different strategies are used for different regions during interpolation, thereby better preserving edge information and reducing interpolation errors. Secondly, the system generates dynamic interpolation weights based on the edge response intensity map. Specifically, for edge regions, the weights are dynamically adjusted according to the edge direction and intensity, making the weights larger in the edge direction to enhance edge information preservation; for smooth regions, smaller weights are used to reduce detail loss during interpolation. The formula is as follows:

[0161] Weight = Regular bilinear interpolation weight + × Directional mask

[0162] in, This refers to the edge enhancement intensity adjustment factor, with a value range of [0.1, 0.5]. It refers to the location The edge response strength value at that location.

[0163] This dynamic weight generation method can flexibly adjust the interpolation strategy according to the characteristics of different regions of the feature map, improving the accuracy and efficiency of interpolation. Then, based on the dynamic interpolation weights, the system interpolates along the edge direction of the edge region, obtaining a new pixel value by calculating a weighted average of pixel values ​​along the edge direction to maintain edge clarity and continuity. Simultaneously, bilinear interpolation is performed on smooth regions. Specifically, the target pixel value is calculated based on the values ​​of the surrounding four pixels using a bilinear interpolation formula, improving interpolation efficiency and maintaining the smoothness of smooth regions. The final interpolated image is obtained. This method, combining direction-weighted interpolation and bilinear interpolation in smooth regions, improves the overall quality and efficiency of interpolation while preserving edge details. Finally, the system performs a resolution upscaling operation on the interpolated image, specifically doubling its original size. This operation yields an upsampled feature map, which is spatially closer to the original image, providing a clearer and more accurate feature representation for subsequent feature fusion and segmentation tasks, thereby improving segmentation accuracy and efficiency.

[0164] Step S54: The upsampled feature map and the jump-connect feature map are fused using the one-dimensional convolutional lightweight gating module to obtain a fused feature map.

[0165] It should be noted that the fused feature map refers to the output result of dynamically fusing upsampled feature maps and jump-connected feature maps through a one-dimensional convolutional lightweight gating module. Its essence is the optimized integration of the encoder's original details (such as weld texture) and the decoder's reconstructed semantics (such as weld region contour). Spatial adaptive feature enhancement is achieved through gating weights, providing high-fidelity input for the final segmentation.

[0166] As an example, the step of fusing the upsampled feature map and the jump-connect feature map through the one-dimensional convolutional lightweight gating module to obtain a fused feature map includes: concatenating the upsampled feature map and the jump-connect feature map along the channel dimension to obtain a second concatenated feature map; performing channel compression on the second concatenated feature map through a one-dimensional convolutional layer in the one-dimensional convolutional lightweight gating module to obtain an upsampled-jump-connect compressed feature map; performing a nonlinear transformation on the upsampled-jump-connect compressed feature map using a sigmoid activation function to obtain a second gating weight map; multiplying the second gating weight map element-wise with the upsampled feature map to obtain a second weighted upsampled feature map; performing a numerical inversion operation on the second gating weight map to obtain a second inverse gating weight map; multiplying the second inverse gating weight map element-wise with the jump-connect feature map to obtain a weighted jump-connect feature map; and adding the second weighted upsampled feature map and the weighted jump-connect feature map to obtain a fused feature map.

[0167] The second concatenated feature map refers to the cascaded result of the upsampled feature map and the skip-connected feature map along the channel dimension. Through spatial alignment, it preserves the high-level semantics reconstructed by the decoder (such as weld seam contours) and the original details transmitted by the encoder (such as fish scale texture), forming the basis for dual-path feature fusion. The upsampled-skip-connected compressed feature map refers to the low-dimensional representation of the second concatenated feature map after channel compression via 1×1 convolution. Its core function is to extract common patterns through cross-channel aggregation, eliminate redundant information (such as background noise), and provide a simplified input for dynamic weight generation. The second gated weight map (… ) refers to the [0,1] interval spatial weight matrix generated by sigmoid activation of the upsampled-jump-connected compressed feature map, whose pixel values ​​dynamically represent the contribution ratio of the upsampled features in the fusion: high weight region ( →1) Strengthen key structures such as weld edges, and low-weight areas ( →0) Suppresses interference such as welding spatter. The second weighted upsampled feature map refers to the element-wise product of the second gated weight map and the upsampled feature map, which enhances the sharpness and directional fidelity of the decoder's reconstructed features in high-frequency regions such as weld stripes and weld wave boundaries. The second inverse gated weight map refers to the result of inverting the values ​​of the second gated weight map (formula: 1− ), forming a complementary distribution with the original gating: low Region → High inverse weights, protecting weak texture features from being discarded; High Regions are assigned low inverse weights to avoid excessive reinforcement of original details. The weighted jump-connect feature map is the element-wise product of the second inverse gating weight map and the jump-connect feature map, which focuses on preserving the encoder's original information (such as tiny cracks masked by noise) in gating weakened regions, ensuring zero loss of effective features.

[0168] First, the computer system stacks the upsampled feature map and the jump-connected feature map along the channel dimension to obtain a second concatenated feature map. This process combines the detailed information in the upsampled feature map with the contextual information in the jump-connected feature map, providing a richer feature representation for subsequent feature fusion and helping to improve the accuracy and robustness of segmentation. Second, the system performs a convolution operation on the second concatenated feature map through a one-dimensional convolutional layer in the one-dimensional convolutional lightweight gating module, reducing the number of channels in the feature map to obtain an upsampled-jump-connected compressed feature map. This operation reduces computational complexity while retaining key feature information, and provides a more compact feature representation for subsequent gating operations. Then, the system maps each element in the upsampled-jump-connected compressed feature map to the (0,1) interval using the Sigmoid activation function, generating a second gating weight map. This weight map is used to control the importance of each feature in the upsampled feature map, realizing weighted feature fusion. This non-linear transformation can automatically adjust the feature weights, avoid noisy fusion, and enhance the model's ability to capture key features. Next, the system multiplies each element in the second gating weight map with the corresponding element in the upsampled feature map to obtain the second weighted upsampled feature map. This weighting operation preserves important features and suppresses unimportant features, helping to improve segmentation accuracy. Simultaneously, the system inverts each element in the second gating weight map (i.e., subtracts the element's value from 1) to obtain the second inverse gating weight map. This inverse operation generates a weight map used to control the importance of each feature in the jump-connected feature map. In this way, the model can better balance feature information from different sources. Then, the system multiplies each element in the second inverse gating weight map with the corresponding element in the jump-connected feature map to obtain the weighted jump-connected feature map. This weighting operation further optimizes the feature fusion process, enabling the model to more effectively utilize the contextual information in the jump-connected feature map. Finally, the system adds the corresponding elements of the second weighted upsampled feature map and the weighted jump-connected feature map to obtain the fused feature map. This process, through feature map matrix operations rather than simple numerical calculations, enhances the key region information in the feature map, improving segmentation accuracy and efficiency, and enabling the model to more efficiently segment weld seam structured light images in practical applications.

[0169] Step S55: The fused feature map is reconstructed using the second multi-frequency domain multilayer perceptron module and the second convolutional block of the decoder to obtain a segmentation result map.

[0170] As an example, the second multi-frequency domain multilayer perceptron module includes a second discrete cosine transform module, a second high-frequency processing unit, a second intermediate-frequency processing unit, and a second low-frequency processing unit. The step of reconstructing the spatial information of the fused feature map through the second multi-frequency domain multilayer perceptron module and the second convolutional block of the decoder to obtain a segmentation result map includes: decomposing the fused feature map into a high-frequency component map, an intermediate-frequency component map, and a low-frequency component map through the second discrete cosine transform module; performing orientation-adaptive rotation convolution on the high-frequency component map through the second high-frequency processing unit to obtain an orientation-sensitive feature map; performing nonlinear cross-channel spatial convolution on the intermediate-frequency component map through the second intermediate-frequency processing unit to obtain a channel interaction feature map; performing multi-directional shift convolution on the low-frequency component map through the second low-frequency processing unit to obtain a multi-scale structural feature map; fusing the orientation-sensitive feature map, the channel interaction feature map, and the multi-scale structural feature map through channels to obtain a multi-frequency reconstructed feature map; and performing two-dimensional convolution, batch normalization, and ReLU activation on the multi-frequency reconstructed feature map through the second convolutional block of the decoder to obtain a segmentation result map.

[0171] The second discrete cosine transform module is a signal processor that splits the fused feature map according to the frequency energy distribution. It decomposes the input feature map into three orthogonal components of high, medium and low frequencies through discrete cosine transform, providing a frequency basis for subsequent differential processing.

[0172] The high-frequency / medium-frequency / low-frequency component maps refer to the three-channel frequency characteristic representations output by the second discrete cosine transform module: the high-frequency component map carries details such as weld edges and fish scale patterns (gradient change > 20% pixel intensity); the medium-frequency component map contains the weld wave transition zone and weak texture structure (gradient change 5%-20%); and the low-frequency component map represents the base material background and overall outline (gradient change < 5%).

[0173] The second high-frequency processing unit refers to the orientation enhancement module of the high-frequency component image, which dynamically adjusts the convolution kernel angle through orientation-adaptive rotational convolution. The orientation-sensitive feature map is the result of processing the high-frequency component image through orientation-adaptive rotational convolution, with its pixel gradient orientation distribution concentration increased by >40% (compared to the original input), significantly enhancing the orientation consistency of weld stripes (such as the tangent direction of straight / curved welds). The second mid-frequency processing unit refers to the channel interaction enhancement module of the mid-frequency component image. Its core operations include: ① Nonlinear aggregation, square root of the sum of squares across channels, formula:

[0174]

[0175] ② Spatial convolution, output channel interaction feature map.

[0176] The channel interaction feature map refers to the optimization result of the intermediate frequency component map through nonlinear cross-channel spatial convolution, which improves the correlation between channels by 30%+ and solves the problem of blocky fracture in the periodic texture of the weld wave.

[0177] The second low-frequency processing unit refers to the multi-directional structure extraction module of the low-frequency component map, which uses a six-directional fixed convolution kernel:

[0178]

[0179] Multi-scale structural feature maps refer to the output of low-frequency component maps fused through multi-directional shifting and convolution, covering the response in the 0°-135° direction, thus improving the contour integrity of irregular welds.

[0180] Multi-frequency reconstruction feature map refers to the feature map obtained by fusing orientation-sensitive feature map, channel interaction feature map and multi-scale structural feature map along the channel dimension.

[0181] First, the computer system, through the second discrete cosine transform module, applies the discrete cosine transform formula to decompose the fused feature map into high-frequency component maps, mid-frequency component maps, and low-frequency component maps. This process can separate the detailed, transitional, and macroscopic structural information in the feature map, providing a foundation for subsequent frequency-adaptive feature extraction and helping to improve the model's ability to process features of different frequencies. Next, the system performs orientation-adaptive rotational convolution on the high-frequency component map through the second high-frequency processing unit. Specifically, it calculates the gradient direction of the input feature map, dynamically adjusts the angle of the convolution kernel according to the gradient direction, and then performs convolution on the high-frequency component map to obtain an orientation-sensitive feature map. This processing method can enhance the model's ability to capture high-frequency details and improve the segmentation accuracy, especially when processing directional weld structures. Then, the system performs nonlinear cross-channel spatial convolution on the intermediate frequency component map through the second intermediate frequency processing unit. Specifically, it first squares the feature values ​​of each channel of the intermediate frequency component map, then sums them along the channel dimension, calculates the average of the sums of squares, and takes the square root to obtain a single-channel feature map. Finally, it performs spatial convolution on this single-channel feature map to obtain a channel interaction feature map. This nonlinear operation enhances the information interaction between channels, better captures the spatial relationships of features, and helps improve the model's ability to process intermediate frequency features. Next, the system performs multi-directional shift convolution on the low-frequency component map through the second low-frequency processing unit. Specifically, it defines convolution kernels in different directions, performs convolution operations on the low-frequency component map separately, and then adds and fuses the convolution results of all directions to obtain a multi-scale structural feature map. This multi-directional convolution operation enriches the directional diversity of feature extraction, helps capture multi-scale structural information in the image, and improves the model's ability to process low-frequency features. Then, the system fuses the orientation-sensitive feature map, channel interaction feature map, and multi-scale structural feature map along the channel dimension. Specifically, these three feature maps are stacked along the channel dimension to obtain a multi-frequency reconstructed feature map. This fusion process can combine the advantages of different frequency components, providing a more comprehensive and accurate feature representation for subsequent segmentation tasks, which helps improve the accuracy and robustness of segmentation. Finally, the system processes the multi-frequency reconstructed feature map through the second convolutional block of the decoder. Specifically, it first extracts spatial features through a two-dimensional convolutional layer, then performs channel normalization through a batch normalization layer, and finally performs nonlinear transformation through a ReLU activation layer to obtain the segmentation result map. This process can further optimize the quality of the feature map, improve the accuracy and efficiency of segmentation, and reduce deployment costs, enabling the model to perform more efficient segmentation of weld seam structured light images in practical applications.

[0182] This embodiment first performs convolution, feature extraction, and downsampling on the structured light image of the weld seam to be segmented using an encoder to obtain a hierarchical feature map. This process preserves detailed information of key areas during downsampling while reducing redundant information, improving the efficiency and accuracy of feature extraction. Next, the system inputs the hierarchical feature map into a one-dimensional convolutional lightweight gating module based on skip connection paths to obtain a skip connection feature map. The combination of skip connection paths and the one-dimensional convolutional lightweight gating module allows the model to better balance feature information at different levels during feature fusion, further improving segmentation accuracy and efficiency. Then, the system uses an edge-aware upsampling layer in the decoder to perform directional weighted interpolation on the downsampled feature map in the hierarchical feature map, obtaining an upsampled feature map. The edge-aware upsampling layer enhances the recovery of edge details, allowing for better preservation of weld seam edge information during upsampling, thereby improving the accuracy of the segmentation results. Finally, the system uses a one-dimensional convolutional lightweight gating module to fuse the upsampled feature map and the skip connection feature map to obtain a fused feature map. This fusion method integrates feature information from different sources, further optimizing the feature fusion process and improving segmentation accuracy and robustness. Finally, the system reconstructs spatial information from the fused feature map using the second multi-frequency domain multilayer perceptron module and the second convolutional block of the decoder, obtaining the segmentation result map. The second multi-frequency domain multilayer perceptron module can process features of different frequencies, further enhancing the feature representation capability, while the second convolutional block further optimizes the quality of the feature map through two-dimensional convolution, batch normalization, and ReLU activation, ultimately obtaining an accurate segmentation result map. The entire process not only improves the accuracy and efficiency of weld seam structured light image segmentation but also reduces the deployment cost of the model through a lightweight design, making it more suitable for real-time and efficient detection of weld seam quality in industrial scenarios.

[0183] This application also provides a weld seam structured light image segmentation device based on a lightweight segmentation model. Please refer to [link / reference]. Figure 8 The weld seam structured light image segmentation device based on a lightweight segmentation model includes:

[0184] Encoder construction module 10 is used to construct an encoder based on a first convolutional block, a first multi-frequency domain multilayer perceptron module, and a saliency-guided downsampling layer.

[0185] Decoder building module 20 is used to build a decoder based on the second multi-frequency domain multilayer perceptron module, the second convolutional block and the edge-aware upsampling layer;

[0186] Model building module 30 is used to build an initial multi-frequency edge lightweight image segmentation model based on the encoder, the jump connection path, the decoder, and the one-dimensional convolutional lightweight gating.

[0187] The model training module 40 is used to train and validate the initial multi-frequency edge lightweight image segmentation model using the labeled weld seam structured light image dataset, so as to obtain the target multi-frequency edge lightweight image segmentation model.

[0188] The image segmentation module 50 is used to perform feature extraction, downsampling, upsampling, and feature fusion on the target multi-frequency edge lightweight image segmentation model to obtain a segmentation result image.

[0189] This application provides a weld seam structured light image segmentation device based on a lightweight segmentation model. The weld seam structured light image segmentation device based on the lightweight segmentation model includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the weld seam structured light image segmentation method based on the lightweight segmentation model in the above embodiment 1.

[0190] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the weld seam structured light image segmentation method based on a lightweight segmentation model in the above embodiments.

[0191] The weld seam structured light image segmentation device, equipment, and storage medium based on a lightweight segmentation model provided in this application, employing the weld seam structured light image segmentation method based on a lightweight segmentation model as described in the above embodiments, can solve the technical problem of how to improve the accuracy and efficiency of weld seam structured light image segmentation and reduce deployment costs. Compared with the prior art, the beneficial effects of the weld seam structured light image segmentation device, equipment, and storage medium based on a lightweight segmentation model provided in this application are the same as the beneficial effects of the weld seam structured light image segmentation method based on a lightweight segmentation model provided in the above embodiments, and other technical features in the weld seam structured light image segmentation device, equipment, and storage medium based on a lightweight segmentation model are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0192] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for segmenting weld seam structured light images based on a lightweight segmentation model, characterized in that, The method includes: An encoder is constructed based on the first convolutional block, the first multi-frequency domain multilayer perceptron module, and the saliency-guided downsampling layer. The decoder is constructed based on the second multi-frequency domain multilayer perceptron module, the second convolutional block, and the edge-sensing upsampling layer. An initial multi-frequency edge lightweight image segmentation model is constructed based on the encoder, the jump connection path, the decoder, and the one-dimensional convolutional lightweight gating. The initial multi-frequency edge lightweight image segmentation model was trained and validated using the labeled weld seam structured light image dataset to obtain the target multi-frequency edge lightweight image segmentation model. The target multi-frequency edge lightweight image segmentation model is used to perform feature extraction, downsampling, upsampling, and feature fusion on the structured light image of the weld seam to be segmented, and the segmentation result image is obtained. The target multi-frequency edge lightweight image segmentation model includes an encoder, a decoder, a skip connection path, and a one-dimensional convolutional lightweight gating module. The steps of performing feature extraction, downsampling, upsampling, and feature fusion on the structured light image of the weld seam to be segmented using the target multi-frequency edge lightweight image segmentation model to obtain the segmentation result image include: The encoder performs convolution, feature extraction, and downsampling on the structured light image of the weld seam to be segmented to obtain a layered feature map. The hierarchical feature map is input into the one-dimensional convolutional lightweight gating module according to the jump connection path to obtain the jump connection feature map; The upsampled feature map is obtained by performing directional weighted interpolation on the downsampled feature map in the hierarchical feature map through the edge-aware upsampling layer of the decoder. The upsampled feature map and the jump-connect feature map are fused using the one-dimensional convolutional lightweight gating module to obtain a fused feature map; The fused feature map is reconstructed using the second multi-frequency domain multilayer perceptron module and the second convolutional block of the decoder to obtain a segmentation result map. The encoder includes a first convolutional block, a first multi-frequency domain multilayer perceptron module, and a saliency-guided downsampling layer. The hierarchical feature map includes a primary feature map, a multi-frequency feature map, and a downsampled feature map. The step of performing convolution, feature extraction, and downsampling on the structured light image of the weld seam to be segmented using the encoder to obtain a layered feature map includes: The first convolutional block is used to perform two-dimensional convolution, normalization, and nonlinear activation on the structured light image of the weld seam to be segmented to obtain a primary feature map. The first multi-frequency domain multilayer perceptron module performs discrete cosine transform decomposition and frequency adaptive feature extraction on the primary feature map to obtain a multi-frequency feature map. The saliency-guided downsampling layer performs key region detection and adaptive pooling on the multi-frequency feature map to obtain the downsampled feature map; The first multi-frequency domain multilayer perceptron module includes a discrete cosine transform module, a high-frequency processing unit, a mid-frequency processing unit, and a low-frequency processing unit; The step of performing discrete cosine transform decomposition and frequency adaptive feature extraction on the primary feature map through the first multi-frequency domain multilayer perceptron module to obtain a multi-frequency feature map includes: The primary feature map is decomposed by the discrete cosine transform module to obtain high-frequency components, mid-frequency components and low-frequency components. The high-frequency components are subjected to orientation-adaptive rotational convolution by the high-frequency processing unit to obtain orientation-sensitive features. The intermediate frequency components are nonlinearly convolved across channels by the intermediate frequency processing unit to obtain channel interaction features. The low-frequency components are subjected to multi-directional shift convolution by the low-frequency processing unit to obtain multi-scale structural features. The direction-sensitive features, the channel interaction features, and the multi-scale structural features are fused along the channel dimension to obtain a multi-frequency fusion feature map.

2. The method according to claim 1, characterized in that, The step of performing key region detection and adaptive pooling on the multi-frequency feature map through the saliency-guided downsampling layer to obtain the downsampled feature map includes: The multi-frequency feature map is compressed by the saliency-guided downsampling layer to obtain a compressed feature map. The channel absolute values ​​of the compressed feature map are summed to obtain a single-channel saliency map; The saliency map is divided into critical regions and smooth regions based on a preset saliency threshold. Average pooling is performed on the key regions to obtain key region features; Max pooling is performed on the smoothed region to obtain the smoothed region features; The key region features are concatenated with the smooth region features to obtain a pooled feature map. The multi-frequency feature map is multiplied element-wise with the single-channel saliency map to obtain the residual feature map; The pooled feature map is added to the residual feature map after one-dimensional convolution to obtain the downsampled feature map.

3. The method according to claim 1, characterized in that, The first convolutional block includes a two-dimensional convolutional layer, a batch normalization layer, and a modified linear unit activation layer; The step of performing two-dimensional convolution, normalization, and nonlinear activation on the structured light image of the weld seam to be segmented using the first convolutional block to obtain a primary feature map includes: The spatial features of the structured light image of the weld seam to be segmented are extracted by the two-dimensional convolutional layer to obtain an initial convolutional feature map. The initial convolutional feature map is normalized by the batch normalization layer to obtain a normalized feature map. The normalized feature map is nonlinearly transformed by the modified linear unit activation layer to obtain a primary feature map.

4. The method according to claim 1, characterized in that, The step of performing directional weighted interpolation on the downsampled feature map in the hierarchical feature map through the edge-aware upsampling layer of the decoder to obtain the upsampled feature map includes: The edge response intensity map of the downsampled feature map is calculated by the Sobel operator through the edge-aware upsampling layer of the decoder. The downsampled feature map includes edge regions and smooth regions. The edge response intensity of the edge regions is greater than a preset intensity threshold, and the edge response intensity of the smooth regions is less than or equal to the preset intensity threshold. Dynamic interpolation weights are generated based on the edge response intensity map; Based on the dynamic interpolation weights, directional weighted interpolation is performed on the edge region, and bilinear interpolation is performed on the smooth region to obtain the interpolation result map; The resolution of the interpolated image is increased to obtain an upsampled feature map.

5. The method according to claim 1, characterized in that, The step of inputting the hierarchical feature map into the one-dimensional convolutional lightweight gating module according to the skip connection path to obtain the skip connection feature map includes: According to the jump connection path, the hierarchical feature map is concatenated with the upsampled feature map of the corresponding layer of the decoder along the channel dimension to obtain the first concatenated feature map; The first stitched feature map is convolved by the one-dimensional convolution lightweight gating module to obtain a layered-upsampled compressed feature map. The hierarchical-upsampled compressed feature map is nonlinearly transformed by the Sigmoid activation function to generate a first gated weight map. The first gated weight map is multiplied element-wise with the upsampled feature map to obtain the first weighted upsampled feature map; Perform a numerical inverse operation on the first gated weight graph to obtain a first inverse gated weight graph; The first inverse gated weight map is multiplied element-wise with the hierarchical feature map to obtain a weighted hierarchical feature map; The first weighted upsampled feature map is added to the weighted hierarchical feature map to obtain the jump-connect feature map.

6. The method according to claim 1, characterized in that, The step of fusing the upsampled feature map and the jump-connect feature map through the one-dimensional convolutional lightweight gating module to obtain the fused feature map includes: The upsampled feature map and the jump-connected feature map are concatenated along the channel dimension to obtain a second concatenated feature map; The second stitched feature map is channel compressed by the one-dimensional convolutional layer in the one-dimensional convolutional lightweight gating module to obtain an upsampled-skipped compressed feature map. The upsampled-jumple compressed feature map is nonlinearly transformed using the Sigmoid activation function to obtain the second gated weight map; The second gated weight map is multiplied element-wise with the upsampled feature map to obtain the second weighted upsampled feature map; Perform a numerical inverse operation on the second gated weight graph to obtain a second inverse gated weight graph; The second reverse gating weight map is multiplied element by element with the jump connection feature map to obtain a weighted jump connection feature map; The second weighted upsampled feature map is added to the weighted jump-connect feature map to obtain the fused feature map.

7. A weld seam structured light image segmentation device based on a lightweight segmentation model, characterized in that, The device includes: An encoder construction module is used to construct an encoder based on a first convolutional block, a first multi-frequency domain multilayer perceptron module, and a saliency-guided downsampling layer. A decoder building module is used to build a decoder based on the second multi-frequency domain multilayer perceptron module, the second convolutional block, and the edge-aware upsampling layer. The model building module is used to build an initial multi-frequency edge lightweight image segmentation model based on the encoder, the jump connection path, the decoder, and the one-dimensional convolutional lightweight gating. The model training module is used to train and validate the initial multi-frequency edge lightweight image segmentation model using the labeled weld seam structured light image dataset, so as to obtain the target multi-frequency edge lightweight image segmentation model. The image segmentation module is used to perform feature extraction, downsampling, upsampling, and feature fusion on the target multi-frequency edge lightweight image segmentation model to obtain a segmentation result image. The target multi-frequency edge lightweight image segmentation model includes an encoder, a decoder, a skip connection path, and a one-dimensional convolution lightweight gating module. The step of performing feature extraction, downsampling, upsampling, and feature fusion on the target multi-frequency edge lightweight image segmentation model to obtain a segmentation result image includes: performing convolution, feature extraction, and downsampling on the target multi-frequency edge lightweight image segmentation model to obtain a layered feature map; and inputting the layered feature map into the one-dimensional convolution module according to the skip connection path. The encoder uses a lightweight gating module to obtain a jumper feature map; the edge-aware upsampling layer of the decoder performs direction-weighted interpolation on the downsampled feature map in the hierarchical feature map to obtain an upsampled feature map; the one-dimensional convolutional lightweight gating module fuses the upsampled feature map and the jumper feature map to obtain a fused feature map; the second multi-frequency domain multilayer perceptron module and the second convolutional block of the decoder reconstruct spatial information from the fused feature map to obtain a segmentation result map; the encoder includes a first convolutional block, a first multi-frequency domain multilayer perceptron module, and a saliency-guided downsampling layer, and the hierarchical feature map includes a primary feature map, a multi-frequency feature map, and a downsampled feature map; the encoder uses the structured light of the weld seam to be segmented. The steps of image convolution, feature extraction, and downsampling to obtain a layered feature map include: performing two-dimensional convolution, normalization, and nonlinear activation on the structured light image of the weld seam to be segmented using the first convolution block to obtain a primary feature map; performing discrete cosine transform decomposition and frequency adaptive feature extraction on the primary feature map using the first multi-frequency domain multilayer perceptron module to obtain a multi-frequency feature map; and performing key region detection and adaptive pooling on the multi-frequency feature map using the saliency-guided downsampling layer to obtain a downsampled feature map. The first multi-frequency domain multilayer perceptron module includes a discrete cosine transform module, a high-frequency processing unit, a mid-frequency processing unit, and a low-frequency processing unit. The steps of performing discrete cosine transform decomposition and frequency adaptive feature extraction on the feature map to obtain a multi-frequency feature map include: performing discrete cosine transform decomposition on the primary feature map through the discrete cosine transform module to obtain high-frequency components, mid-frequency components, and low-frequency components; performing orientation-adaptive rotational convolution on the high-frequency components through the high-frequency processing unit to obtain orientation-sensitive features; performing nonlinear cross-channel spatial convolution on the mid-frequency components through the mid-frequency processing unit to obtain channel interaction features; performing multi-directional shift convolution on the low-frequency components through the low-frequency processing unit to obtain multi-scale structural features; and fusing the orientation-sensitive features, the channel interaction features, and the multi-scale structural features along the channel dimension to obtain a multi-frequency fused feature map.

Images