A laser stripe semantic segmentation method under multiple reflective light interference

Abstract

The application discloses a laser stripe semantic segmentation method under multiple reflective light interference, and relates to the technical field of line structured light active vision measurement. A plurality of laser stripe pictures containing various reflective light interferences are acquired, and salt and pepper noise is added to the laser stripe pictures by using a data enhancement method; the two laser stripe pictures containing various reflective light interferences obtained in step 1 are normalized to a set size, and the two laser stripe pictures containing various reflective light interferences are manually labeled to generate a plurality of label pictures, and then a reflective light interference dataset is obtained; the reflective light interference dataset is divided according to a set proportion; the training set, the verification set and the test set are preprocessed; a local attention mechanism Local-Attention is designed; a global attention mechanism Global-Attention is designed; a guide network GuidNet model is built; the training set is used to train the GuidNet model for a set number of rounds to obtain a trained GuidNet model; and the test set is input into the GuidNet model to obtain a laser stripe segmentation result.

Description

Technical Field

[0001] This invention relates to the field of line structured light active vision measurement technology, and in particular to a method for semantic segmentation of laser stripes under multiple reflective interferences. Background Technology

[0002] Structured light active vision measurement technology is a 3D measurement method based on the principle of laser triangulation. First, a laser projector projects structured laser lines onto the object's surface. Image processing algorithms then extract the pixel coordinates of the laser stripe centerline, and the object's 3D point cloud data is reconstructed based on a sensor model. This technology boasts significant advantages such as high measurement accuracy, high speed, and non-contact operation, and has been widely applied in fields such as online inspection, vision guidance, and 3D digitization.

[0003] However, line structured light active vision measurement technology also has some limitations, such as sensitivity to ambient light and the susceptibility of light stripe imaging quality to the reflective properties of the object being measured. Especially when multiple specular reflections occur, the interference from these reflections is extremely similar to the imaging brightness and topological shape of the real laser stripes, significantly increasing the difficulty of extracting the real laser stripes. Specifically, the presence of reflection interference leads to the following problems, making the identification and extraction of real laser stripes exceptionally difficult:

[0004] 1) The reflective interference and the laser stripe are similar in brightness, which means that the laser stripe is not the only bright area;

[0005] 2) The geometric shape of the reflected light interference is similar to that of the laser stripe, making the laser stripe not the only stripe area in the image; the reflected light interference may be very close to the laser stripe, or even overlap, causing some information of the laser stripe to be obscured or confused and unable to be clearly displayed.

[0006] Traditional image processing-based laser stripe extraction methods, such as the geometric center method, gray-level centroid method, morphological method, and thresholding method, extract the laser stripe region by analyzing the physical characteristics of the laser stripes. Current applications show that these methods suffer from poor flexibility and robustness, and are easily affected by noise generated during welding in complex industrial environments, such as reflections, arc light, and smoke. In recent years, many researchers have designed different detection models and proposed various laser stripe extraction methods based on the unique linear shape and geometric characteristics of laser stripes. These methods often rely on complex image processing operators, such as image filtering, ROI extraction, and line extraction, and the initialization key factors required for these image processing operators depend on expert experience. Therefore, although these methods offer some improvement over traditional methods, they still cannot accurately identify laser stripes under reflective interference. Summary of the Invention

[0007] The problem this invention aims to solve is to address the shortcomings of the existing technology by proposing a semantic segmentation method for laser stripes under multiple reflection interferences. By constructing an end-to-end laser stripe segmentation network, deeper semantic features are extracted from laser stripe images affected by reflection noise, thereby effectively solving the problem of laser stripe extraction under multiple reflection interferences and improving the accuracy and robustness of laser stripe extraction.

[0008] The technical solution of this invention is as follows:

[0009] Step 1: Obtain several laser stripe images containing various types of reflective interference. Use data augmentation to add salt-and-pepper noise to the laser stripe images to obtain the same number of laser stripe images with salt-and-pepper noise containing various types of reflective interference as the original images containing various types of reflective interference. The various types of reflective interference include specular reflection, scattered reflection, multiple reflections, and refraction.

[0010] Step 2: Normalize the two laser stripe images containing multiple reflective interferences obtained in Step 1 to a set size, and manually label the two laser stripe images containing multiple reflective interferences to generate several label images, thereby obtaining a reflective interference dataset; the reflective interference dataset consists of two laser stripe images containing multiple reflective interferences and label images.

[0011] Step 3: Divide the reflective interference dataset according to a set ratio to obtain the training set, validation set, and test set;

[0012] Step 4: Preprocess the training set, validation set, and test set to obtain preprocessed training set, validation set, and test set; the preprocessing method is to reduce the image size to a set size and randomly apply one or more data augmentation methods for data augmentation; the data augmentation methods include flipping, rotating, and translating;

[0013] Step 5: Design a local attention mechanism, Local-Attention;

[0014] The Local Attention mechanism transforms the input feature map into a query tensor, a key tensor, and a value tensor using a linear mapping method. It then performs average pooling on the query tensor and the key tensor according to a defined region to obtain the matrix Q. avg and K avg Then calculate Q avg and (K) avg ) TThe product of these terms yields the region-to-region relation matrix Rel. Applying the topk operator to Rel finds the k most relevant regions for each region, resulting in the most relevant region index matrix Idx. The gather method is then used to collect the key and value tensors using the index matrix Idx, thus obtaining the collected key tensor K. g and the collection value tensor V g Finally, using the query tensor Query and the collection key tensor K... g and the collection value tensor V g Attention is calculated to obtain the local attention feature map Attn. local ;

[0015] The relation matrix Rel is:

[0016] Rel = Q avg (K avg ) T (1)

[0017] Where Rel is the relation matrix; Q avg To query the matrix obtained by average pooling of the tensor; K avg The matrix obtained by average pooling the key tensor;

[0018] The most relevant region index matrix Idx is:

[0019] Idx = topk(Rel) (2)

[0020] Where Idx is the index matrix of the most relevant regions; topk represents the topk operator;

[0021] The collection key tensor and collection value tensor are:

[0022] K g =gather(K,Idx),V g =gather(V,Idx) (3)

[0023] Among them, K g To collect key tensors; V g For collecting value tensors; K is the key tensor; V is the value tensor; gather represents the gather method of torch;

[0024] The method for calculating attention is as follows:

[0025]

[0026] Among them, Attn local For local attention feature maps; softmax represents the softmax function; d headQ represents the number of channels in the input feature map; Q is the query tensor.

[0027] Step 6: Design a global attention mechanism, Global-Attention;

[0028] The Global-Attention mechanism transforms the input feature map into a query tensor Q', a key tensor K', and a value tensor V' using a linear mapping method. Then, it performs softmax normalization on the rows and columns of the query tensor Q' and the key tensor K' respectively to obtain a row-normalized query tensor Q'. softmax Normalized bond tensor K softmax Finally, attention is calculated using the row-normalized query tensor, column-normalized key tensor, and value tensor to obtain the global attention feature map Attn. global ;

[0029] The row-normalized query tensor and column-normalized key tensor are:

[0030]

[0031] Among them, Q softmax Let Q' be the row-normalized query tensor; Q' be the query tensor; K' be the key tensor; K softmax For column-normalized bond tensors; softmax row This indicates that the softmax function is applied to the rows; softmax col This indicates that the softmax function is applied to the column; d head This indicates the number of channels in the input feature map;

[0032] The method for calculating attention is as follows:

[0033] Attn global =Q softmax ((K softmax ) T V') (6)

[0034] Among them, Attn global For global attention feature maps; Q softmax K represents the row-normalized query tensor; softmax V' represents the column normalized bond tensor; V' represents the value tensor.

[0035] Step 7: Build the GuidNet model based on local and global attention mechanisms;

[0036] The GuidNet model includes a convolution stem, an encoder, and a decoder.

[0037] The initial convolution stem is used to extract features from the input laser stripe image containing various reflective interferences, obtain image tag tokens, and send them to the encoder.

[0038] The initial convolution stem consists of three sequentially configured convolution groups, each consisting of two sequentially configured convolution layers.

[0039] The encoder is used to extract deep semantic and spatial features from image tokens, and uses the semantic features to guide and enhance the feature representation of spatial features, while reducing the influence of reflection interference, to obtain several output feature maps at different levels and send them to the decoder.

[0040] The encoder has a dual-branch structure, including a semantic branch and a spatial branch.

[0041] The semantic branch is used to simultaneously obtain semantic features at different levels from image tag tokens, obtain several semantic feature maps at different levels, and send them to the spatial branch.

[0042] The Spatial Branch is used to obtain spatial features at different levels from image tokens, resulting in several spatial feature maps at different levels; it receives semantic feature maps at different levels sent by the Semantic Branch and corrects the spatial feature maps based on the semantic feature maps at different levels, resulting in several output feature maps at different levels, which are then sent to the decoder; the spatial features are the shape features and edge features of the laser stripes.

[0043] The semantic branch consists of four globally fused local attention mechanism layers (GAL-TransformerLayer) set sequentially.

[0044] The first global fusion local attention mechanism layer, GAL-Transformer Layer 1, is used to perform multi-head self-attention computation on image-labeled tokens to obtain semantic features. Figure 1 and the obtained semantic features Figure 1 Send to the second global fusion local attention mechanism layer GAL-Transformer Layer 2 and the first guidance module Guidance Block 1;

[0045] The second global fusion local attention mechanism layer, GAL-Transformer Layer 2, is used for semantic features. Figure 1Further multi-head self-attention computation is performed to obtain semantic features. Figure 2 and the obtained semantic features Figure 2 It is sent to the third global fusion local attention mechanism layer GAL-Transformer Layer 3 and the second guidance module Guidance Block 2;

[0046] The third global fusion local attention mechanism layer, GAL-Transformer Layer 3, is used for semantic features. Figure 2 Further multi-head self-attention computation is performed to obtain semantic features. Figure 3 and the obtained semantic features Figure 3 It is sent to the fourth global fusion local attention mechanism layer GAL-Transformer Layer 4 and the third guidance module Guidance Block 3;

[0047] The fourth global fusion local attention mechanism layer, GAL-Transformer Layer 4, is used for semantic features. Figure 3 Further multi-head self-attention computation is performed to obtain semantic features. Figure 4 and the obtained semantic features Figure 4 Send to the fourth bootstrap module, Guidance Block 4;

[0048] Each of the Global Fusion Local Attention Mechanism Layers (GAL-Transformer Layer) consists of two serially configured Global Fusion Local Attention Blocks (GAL-Transformer Blocks).

[0049] The GAL-Transformer Block consists of a first normalized Layer Norm layer, a Local-Attention module, a first fusion layer, a second normalized Layer Norm layer, a first Mix-FFN layer, a second fusion layer, a Global-Attention module, a third fusion layer, a third normalized Layer Norm layer, a second Mix-FFN layer, and a fourth fusion layer.

[0050] The first normalization layer (Layer Norm layer) is used to perform layer normalization on the initial input feature map to obtain the normalized features. Figure 1 And send it to the Local-Attention module, the local attention mechanism;

[0051] The Local Attention module is used to process the normalized features sent to the first normalized Layer Norm layer through a local attention mechanism. Figure 1 Attention calculation is performed to obtain features. Figure 2 And send it to the first fusion layer;

[0052] The first fusion layer is used to process the feature map of the initial input and the features output by the Local-Attention module. Figure 2 Perform feature fusion to obtain the fused features. Figure 3 And send it to the second normalized Layer Norm layer and the second fusion layer;

[0053] The second normalized Layer Norm layer is used to process the fused features output by the first fusion layer. Figure 3 Perform layer normalization to obtain normalized features. Figure 4 And send it to the first hybrid feedforward network Mix-FFN layer;

[0054] The first Mix-FFN layer of the hybrid feedforward network is used to normalize the features output by the second normalized Layer Norm layer. Figure 4 Further nonlinear transformations and feature extraction are performed to obtain the features. Figure 5 And send it to the second fusion layer;

[0055] The second fusion layer is used to process the features output by the first Mix-FFN layer of the hybrid feedforward network. Figure 5 Features output by the first fusion layer Figure 3 Perform feature fusion to obtain the fused features. Figure 6 And send it to the Global-Attention module and the third fusion layer;

[0056] The Global-Attention module is used to process the features output by the second fusion layer through a global attention mechanism. Figure 6 Attention calculation is performed to obtain features. Figure 7 And send it to the third fusion layer;

[0057] The third fusion layer is used to process the features output by the Global-Attention module. Figure 7 Features output by the second fusion layer Figure 6 Feature fusion is performed to obtain the fused feature map 8, which is then sent to the third normalized Layer Norm layer and the fourth fusion layer.

[0058] The third normalization layer is used to normalize the feature map 8 output by the third fusion layer to obtain the normalized feature map 9 and send it to the second hybrid feedforward network Mix-FFN layer.

[0059] The second Mix-FFN layer of the hybrid feedforward network is used to perform further nonlinear transformation and feature extraction on the feature map 9 output by the third normalized Layer Norm layer to obtain feature map 10 and send it to the fourth fusion layer.

[0060] The fourth fusion layer is used to fuse the feature map 10 output by the second Mix-FFN layer with the feature map 8 output by the third fusion layer to obtain the fused feature map 11 and send it to the next GAL-Transformer Block.

[0061] The feature map output by the fourth fusion layer of the second GAL-Transformer Block in each global fusion local attention mechanism layer (GAL-Transformer Layer) is the semantic feature map output by that global fusion local attention mechanism layer.

[0062] The spatial branch consists of four serially configured guide extraction modules;

[0063] Each of the guided extraction modules consists of a serially connected Double Conv Block and a Guidance Block.

[0064] The first double convolutional module, Double Conv Block 1, is used to extract features from the image tokens sent by the initial convolutional layer, the convolution stem, to obtain spatial features. Figure 1 And send it to the first bootstrap module, Guidance Block 1;

[0065] The first guidance module, Guidance Block 1, is used to receive spatial features sent by the first double convolutional module, Double Conv Block 1. Figure 1 The semantic features sent by the first global fusion local attention mechanism layer, GAL-Transformer Layer 1 Figure 1 Utilizing semantic features provided by semantic branches Figure 1 To guide the improvement of the spatial characteristics of Double Conv Block1 Figure 1 This leads to the output features. Figure 1 And send it to the second double convolutional module, Double Conv Block 2, and the decoder;

[0066] The second double convolutional module, Double Conv Block 2, is used to process the output features of the first guidance module, Guidance Block 1. Figure 1 Feature extraction is performed to obtain spatial features. Figure 2 And send it to the second boot module GuidanceBlock2;

[0067] The second guidance module, Guidance Block 2, is used to receive spatial features sent by the second double convolutional module, Double Conv Block 2. Figure 2 Semantic features sent by the second global fusion local attention mechanism layer GAL-Transformer Block2 Figure 2 Utilizing stylistic features provided by semantic branches Figure 2 To guide and improve the spatial characteristics of the Double Conv Block2 output Figure 2 This leads to the output features. Figure 2 And send it to the third double convolutional module, Double Conv Block 3, and the decoder;

[0068] The third double convolutional module, Double Conv Block 3, is used to process the output features of the second guidance module, Guidance Block 2. Figure 2 Feature extraction is performed to obtain spatial features. Figure 3 And send it to the third bootstrap module GuidanceBlock3;

[0069] The third guidance module, Guidance Block 3, is used to receive spatial features sent by the third double convolutional module, Double Conv Block 3. Figure 3 Semantic features sent by the third global fusion local attention mechanism layer GAL-Transformer Block3 Figure 3 Utilizing stylistic features provided by semantic branches Figure 3 To guide and improve the spatial characteristics of Double Conv Block3 output Figure 3 This leads to the output features. Figure 3 And send it to the fourth double convolutional module, Double Conv Block 4, and the decoder;

[0070] The fourth double convolutional module, Double Conv Block 4, is used to obtain the output features from the third guidance module, Guidance Block 3. Figure 3 Feature extraction is performed to obtain spatial features. Figure 4 And send it to the fourth boot module, GuidanceBlock4;

[0071] The fourth guidance block (Guidance Block 4) is used to receive spatial features sent by the fourth double convolutional block (Double Conv Block 4). Figure 4 Semantic features sent by the fourth global fusion local attention mechanism layer, GAL-Transformer Block4. Figure 4 Utilizing stylistic features provided by semantic branches Figure 4 To guide and improve the spatial characteristics of Double Conv Block4 output Figure 4 This leads to the output features. Figure 4 And send it to the decoder;

[0072] The guidance module includes a first convolutional layer, a second convolutional layer, a similarity calculation module, a third convolutional layer, a similarity mapping module, a first feature selection module, a second feature selection module, and a similarity fusion module;

[0073] The first convolutional layer is used to further extract features from the spatial feature map, obtain the spatial feature extraction map, and send it to the similarity calculation module;

[0074] The second convolutional layer is used to further extract features from the semantic feature map, obtain the semantic feature extraction map, and send it to the similarity calculation module;

[0075] The similarity calculation module calculates the similarity of corresponding pixels between the spatial feature extraction map and the semantic feature extraction map, obtains a similarity metric map, and sends it to the third convolutional layer;

[0076] The method for calculating similarity is as follows:

[0077] map sim =sigmoid(spt·sem) (7)

[0078] Among them, map sim For similarity measurement graphs; sigmoid represents the sigmoid function; spt represents the spatial feature extraction graph; sem represents the semantic feature extraction graph.

[0079] The third convolutional layer is used to further extract features from the similarity metric map to obtain a similarity metric feature extraction map and send it to the similarity mapping module;

[0080] The similarity mapping module is used to apply the sigmoid function to the similarity metric feature extraction map to obtain a similarity mapping map and send it to the first feature selection module and the second feature selection module;

[0081] The first feature selection module is used to multiply the similarity map by the spatial feature map to obtain a spatial feature selection map and send it to the similarity fusion module;

[0082] The second feature selection module is used to multiply the similarity map by the semantic feature map to obtain a semantic feature selection map and send it to the similarity fusion module;

[0083] The similarity fusion module is used to fuse the spatial feature selection map and the semantic feature selection map to obtain an output feature map;

[0084] The calculation method for feature selection and similarity fusion is as follows:

[0085] out fusion =(1-map) sim )·spt featuremap +map sim ·sem featuremap (8)

[0086] Among them, out fusion To output feature maps; map sim Represents a similarity metric graph; spt featuremap Represents spatial feature maps; sem featuremap Represents semantic feature maps;

[0087] The Double ConvBlock module consists of two serially configured convolutional layers.

[0088] The decoder is used to calculate the importance of several output feature maps at different levels and aggregate the output feature maps according to their importance to obtain the final dense semantic segmentation result.

[0089] The decoder includes a first importance calculation module, a second importance calculation module, a third importance calculation module, a fourth importance calculation module, a first importance feature extraction module, a second importance feature extraction module, a third importance feature extraction module, a first addition and fusion module, a fourth importance feature extraction module, and a second addition and fusion module;

[0090] The first importance calculation module is used to calculate the output features. Figure 1 The importance level is calculated to obtain the importance level. Figure 1 And send it to the first important feature extraction module;

[0091] The second importance calculation module is used to calculate the output features. Figure 2 The importance level is calculated to obtain the importance level. Figure 2 And send it to the second important feature extraction module;

[0092] The third importance calculation module is used to calculate the output features. Figure 3 The importance level is calculated to obtain the importance level. Figure 3 And send it to the third important feature extraction module;

[0093] The fourth importance calculation module is used to calculate the output features. Figure 4 The importance level is calculated to obtain the importance level. Figure 4 And send it to the fourth important feature extraction module;

[0094] Each of the importance calculation modules includes a convolutional layer and a sigmoid function in sequence.

[0095] The method for calculating the importance level is as follows:

[0096]

[0097] Where, x i At represents the i-th output feature map; i imp is the i-th output feature map after passing through the convolutional layer. i This represents the i-th importance graph; conv denotes a convolutional layer; sigmoid denotes the sigmoid function;

[0098] The first important feature extraction module is used to process the output features. Figure 1 And importance Figure 1 Perform dot product operation to obtain important feature extraction. Figure 1 And send it to the first addition and fusion module;

[0099] The second important feature extraction module is used to process the output features. Figure 2 And importance Figure 2 Perform dot product operation to obtain important feature extraction. Figure 2 And send it to the first addition and fusion module;

[0100] The third important feature extraction module is used to process the output features. Figure 3 And importance Figure 3 Perform dot product operation to obtain important feature extraction. Figure 3 And send it to the first addition and fusion module;

[0101] The first addition and fusion module is used to determine the degree of importance. Figure 1 Importance Figure 2 And importance Figure 3 Perform an addition operation to obtain the sum. Figure 1 And send it to the fourth important feature extraction module;

[0102] The fourth important feature extraction module is used for addition and fusion. Figure 1 And importance Figure 4 Perform dot product operation to obtain important feature extraction. Figure 4 And send it to the second addition and fusion module;

[0103] The second addition and fusion module is used for extracting important features. Figure 4 Output features Figure 4 Perform an addition operation to obtain the sum. Figure 2 That is, the final output segmentation result;

[0104] The segmentation result is expressed as follows:

[0105]

[0106] Among them, out decoder This is the segmentation result.

[0107] Step 8: Train the GuidNet model for a set number of rounds using the training set to obtain the trained GuidNet model;

[0108] Step 9: Input the test set into the GuidNet model to obtain the laser stripe segmentation results.

[0109] Step 9.1: Input the two laser stripe images and label images containing various reflective interferences from the test set into the initial convolution stem to obtain image tag tokens;

[0110] Step 9.2: Input the image tag tokens into the encoder, and use spatial branching and semantic branching to perform step-by-step feature extraction to obtain several output feature maps at different levels;

[0111] Step 9.3: Input the output feature maps of several different levels into the decoder to obtain the laser stripe segmentation results.

[0112] Compared with the prior art, the technical solution adopted in this invention has the following technical effects:

[0113] This invention provides a semantic segmentation method for laser stripes under multiple reflective interferences. It employs an encoder-decoder framework, featuring a novel dual-branch structure in the encoder: a semantic branch and a spatial branch. The semantic branch uses a proposed GAL-Transformer to perform correlation calculations on all spatial locations of the input image, effectively extracting semantic information from the real laser stripe image under multiple reflective interferences. The spatial branch captures detailed information from the input image using convolutional layers. Furthermore, a Guidance Block is used to calculate pixel-level similarity between the feature maps of the semantic and spatial branches, enhancing the shape and edge details in the spatial branch. This allows for more comprehensive capture of laser stripe features while mitigating the impact of reflective interference. In the decoder, attention weights are calculated on the output feature map of the Guidance Block to determine the relative importance of different feature layers, selectively fusing features from each layer. This significantly improves the accuracy and robustness of laser stripe extraction. Attached Figure Description

[0114] Figure 1 This is a flowchart of a laser stripe semantic segmentation method under multiple reflective interference in an embodiment of the present invention;

[0115] Figure 2 This is a schematic diagram of the Local-Attention and Global-Attention structures in an embodiment of the present invention;

[0116] Figure 3 This is a structural diagram of the GuidNet model in an embodiment of the present invention;

[0117] Figure 4 This is a schematic diagram of the GAL-Transformer Block in an embodiment of the present invention;

[0118] Figure 5 This is a structural diagram of the Guidance Block module in an embodiment of the present invention;

[0119] Figure 6 This is a diagram of the decoder structure in an embodiment of the present invention;

[0120] Figure 7 This is a comparison of laser stripe image segmentation results under different conditions with the other two methods in this embodiment of the invention;

[0121] Wherein, (a) is the input image; (b) is the result of the implementation of the present invention; (c) is the result of the Transception model; and (d) is the result of the MISSFormer model. Detailed Implementation

[0122] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0123] This embodiment provides a method for semantic segmentation of laser stripes under multiple reflective interferences, such as... Figure 1 As shown, the method includes the following steps:

[0124] Step 1: Obtain several laser stripe images containing various types of reflective interference. Use data augmentation to add salt-and-pepper noise to the laser stripe images to obtain the same number of laser stripe images with salt-and-pepper noise containing various types of reflective interference as the original images containing various types of reflective interference. The various types of reflective interference include specular reflection, scattered reflection, multiple reflections, and refraction.

[0125] This embodiment utilizes a line-structured light robot vision measurement system platform. This system primarily comprises a degree-of-freedom Motoman industrial robot, a line-structured light sensor, a laser projector, an industrial camera, and an image processing and control computer. The system projects line-structured light onto the surface of the object being measured using a laser projector. The camera captures images of the laser line structure, and image processing and computation techniques are then used to achieve three-dimensional measurement of the object's surface. This method offers advantages such as non-contact operation, high precision, high efficiency, and automation. The line-structured light vision measurement system acquired 500 images of laser stripes containing various reflective interferences, and through data augmentation, obtained 500 more images of laser stripes with salt-and-pepper noise and various reflective interferences, totaling 1000 images.

[0126] Step 2: Normalize the two laser stripe images containing multiple reflective interferences obtained in Step 1 to a set size, and manually label the two laser stripe images containing multiple reflective interferences to generate several label images, thereby obtaining a reflective interference dataset; the reflective interference dataset consists of two laser stripe images containing multiple reflective interferences and label images.

[0127] In the dataset creation phase, this implementation method normalizes the image size to 1280×600 pixels. Then, the images are manually labeled to generate label images. In the experiment, the laser stripe area is labeled as one category, represented by red, while other reflective interference elements and the background are classified as another category, represented by black.

[0128] Step 3: Divide the reflective interference dataset according to a set ratio to obtain the training set, validation set, and test set;

[0129] This implementation divides the dataset into three parts in an 8:1:1 ratio: a training set, a validation set, and a test set. This division ensures that there is sufficient data for training, parameter tuning, and final performance evaluation during model development and evaluation.

[0130] Step 4: Preprocess the training set, validation set, and test set to obtain preprocessed training set, validation set, and test set; the preprocessing method is to reduce the image size to a set size and randomly apply one or more data augmentation methods for data augmentation; the data augmentation methods include flipping, rotating, and translating;

[0131] To make the network as lightweight as possible, the input image is preprocessed to 448×448 pixels in this embodiment, and one to three data augmentations, including flipping, rotation, and translation, are randomly applied to the input image.

[0132] Step 5: Design a local attention mechanism, Local-Attention;

[0133] Since attention calculations involve relational computations across all spatial locations, they exhibit high computational complexity and significant memory consumption. However, when the laser stripes occupy a very small percentage of pixels in an image, it is unnecessary to perform these calculations across all locations in space. To reduce computational complexity and memory consumption while still effectively capturing the important features of the laser stripes in the image, we introduce a novel method that calculates attention only for each region and its k most relevant regions.

[0134] like Figure 2 As shown, the Local Attention mechanism transforms the input feature map into a query tensor, a key tensor, and a value tensor through a linear mapping method. It then performs average pooling on the query tensor and the key tensor according to a defined region to obtain matrix Q. avg and K avg Then calculate Q avg and (K) avg ) T The product of these terms yields the region-to-region relation matrix Rel. Applying the topk operator to Rel finds the k most relevant regions for each region, resulting in the most relevant region index matrix Idx. Since these most relevant regions are scattered throughout the feature map, the gather method is used to collect the key and value tensors using the index matrix Idx, thus obtaining the collected key tensor K. g and the collection value tensor V g Finally, using the query tensor Query and the collection key tensor K... g and the collection value tensor V g Attention is calculated to obtain the local attention feature map Attn. local .

[0135] The relation matrix Rel is:

[0136] Rel = Q avg (K avg ) T (1)

[0137] Where Rel is the relation matrix; Q avg To query the matrix obtained by average pooling of the tensor; K avg The matrix obtained by average pooling the key tensor;

[0138] The most relevant region index matrix Idx is:

[0139] Idx = topk(Rel) (2)

[0140] Where Idx is the index matrix of the most relevant regions; topk represents the topk operator;

[0141] The collection key tensor and collection value tensor are:

[0142] K g =gather(K,Idx),V g =gather(V,Idx) (3)

[0143] Among them, K g To collect key tensors; V g For collecting value tensors; K is the key tensor; V is the value tensor; gather represents the gather method of torch;

[0144] The method for calculating attention is as follows:

[0145]

[0146] Among them, Attn local For local attention feature maps; softmax represents the softmax function; d head This represents the number of channels in the input feature map; Q is the query tensor.

[0147] Step 6: Design a global attention mechanism, Global-Attention;

[0148] Local-Attention effectively captures local information such as reflective features and real laser stripe morphology. However, while it effectively captures high-frequency details, it lacks awareness of low-frequency global information. To overcome this limitation, we add Global-Attention. Global-Attention improves the dot product calculation of attention by adjusting its calculation order. This method splits the matrix dot product calculation, and we correspondingly split the softmax calculation as well. Specifically, we apply the softmax function to the rows and columns of Q and K respectively. In dot product attention, an attention map is generated for each position by calculating the similarity between each position and other positions. In Global-Attention, instead of generating a separate attention map for each position, a global attention map is obtained through the calculation of K and V. In this way, the method of this invention not only obtains a global description of the input features but also effectively reduces the complexity of attention calculation.

[0149] like Figure 2 As shown, the Global Attention mechanism transforms the input feature map into a query tensor Q', a key tensor K', and a value tensor V' using a linear mapping method. Then, it performs softmax normalization on the rows and columns of the query tensor Q' and the key tensor K' respectively to obtain a row-normalized query tensor Q'. softmax Normalized bond tensor K softmax Finally, attention is calculated using the row-normalized query tensor, column-normalized key tensor, and value tensor to obtain the global attention feature map Attn. global .

[0150] The row-normalized query tensor and column-normalized key tensor are:

[0151]

[0152] Among them, Q softmax Let Q' be the row-normalized query tensor; Q' be the query tensor; K' be the key tensor; K softmax For column-normalized bond tensors; softmax row This indicates that the softmax function is applied to the rows; softmax col This indicates that the softmax function is applied to the column; d head This indicates the number of channels in the input feature map;

[0153] The method for calculating attention is as follows:

[0154] Attn global =Q softmax ((Ksoftmax ) T V') (6)

[0155] Among them, Attn global For global attention feature maps; Q softmax K represents the row-normalized query tensor; softmax V' represents the column normalized key tensor; V' represents the value tensor.

[0156] Step 7: Build the GuidNet model based on local and global attention mechanisms;

[0157] like Figure 3 As shown, the GuidNet model in this embodiment includes a convolution initialization layer, an encoder, and a decoder;

[0158] The initial convolution stem is used to extract features from the input laser stripe image containing various reflective interferences, obtain image tag tokens, and send them to the encoder.

[0159] The encoder is used to extract deep semantic and spatial features from image tokens, and uses the semantic features to guide and enhance the feature representation of spatial features, while reducing the influence of reflection interference, to obtain several output feature maps at different levels and send them to the decoder.

[0160] The decoder is used to calculate the importance of several output feature maps at different levels and aggregate several output feature maps according to their importance, thereby enhancing the representation ability of the decoder output features and obtaining the final dense semantic segmentation result.

[0161] Furthermore, the initial convolution stem consists of three sequentially arranged convolution groups, and each convolution group consists of two sequentially arranged convolution layers; in this embodiment, the strides of the two convolution layers in each convolution group are 2 and 1, respectively.

[0162] Furthermore, the encoder employs a dual-branch structure, comprising a semantic branch and a spatial branch. The semantic branch utilizes a global fusion local attention mechanism layer (GAL-Transformer Layer) to extract semantic features, while the spatial branch is responsible for capturing spatial features from image tokens. Although the spatial branch can capture spatial features, it lacks semantic representation capabilities and is prone to misclassifying reflective noise as genuine laser stripes. To address this issue, we designed a guidance block, which integrates semantic features into spatial features to guide the spatial branch in more reliably acquiring features of genuine laser stripes. The guidance block acts as a bridge between semantic and spatial features, thereby improving the accuracy and stability of laser stripe extraction. The spatial features include the shape and edge features of the laser stripes.

[0163] The semantic branch is used to simultaneously obtain semantic features at different levels from image tag tokens, obtain several semantic feature maps at different levels, and send them to the spatial branch.

[0164] The Spatial Branch is used to obtain spatial features at different levels from image tokens, resulting in several spatial feature maps at different levels; it receives semantic feature maps at different levels sent by the Semantic Branch and corrects the spatial feature maps based on the semantic feature maps at different levels, resulting in several output feature maps at different levels, which are then sent to the decoder; the spatial features are the shape features and edge features of the laser stripes.

[0165] The semantic branch consists of four globally fused local attention mechanism layers (GAL-TransformerLayer) set sequentially.

[0166] The first global fusion local attention mechanism layer, GAL-Transformer Layer 1, is used to perform multi-head self-attention computation on image-labeled tokens to obtain semantic features at 1 / 8 resolution. Figure 1 and the obtained semantic features Figure 1 The image is sent to the second global fusion local attention mechanism layer GAL-Transformer Layer 2 and the first guidance module Guidance Block 1; the resolution is the initial image size, i.e., 448×448 pixels.

[0167] The second global fusion local attention mechanism layer, GAL-Transformer Layer 2, is used for semantic features. Figure 1Further multi-head self-attention computation was performed to obtain semantic features at 1 / 16 resolution. Figure 2 and the obtained semantic features Figure 2 It is sent to the third global fusion local attention mechanism layer GAL-Transformer Layer 3 and the second guidance module Guidance Block 2;

[0168] The third global fusion local attention mechanism layer, GAL-Transformer Layer 3, is used for semantic features. Figure 2 Further multi-head self-attention computation was performed to obtain semantic features at 1 / 32 resolution. Figure 3 and the obtained semantic features Figure 3 It is sent to the fourth global fusion local attention mechanism layer GAL-Transformer Layer 4 and the third guidance module Guidance Block 3;

[0169] The fourth global fusion local attention mechanism layer, GAL-Transformer Layer 4, is used for semantic features. Figure 3 Further multi-head self-attention computation was performed to obtain semantic features at 1 / 64 resolution. Figure 4 and the obtained semantic features Figure 4 Send to the fourth bootstrap module, Guidance Block 4;

[0170] The global fusion local attention mechanism layer (GAL-Transformer Layer) consists of two serially configured global fusion local attention blocks (GAL-Transformer Blocks). Each GAL-Transformer Block allows semantic branches to simultaneously acquire local and global semantic features within the same module. This combination enables the model to capture deeper semantic features of the image more comprehensively, thereby improving its expressive power and generalization performance. Figure 4 As shown, the GAL-Transformer Block consists of a first normalized Layer Norm layer, a Local-Attention module, a first fusion layer, a second normalized Layer Norm layer, a first Mix-FFN layer, a second fusion layer, a Global-Attention module, a third fusion layer, a third normalized Layer Norm layer, a second Mix-FFN layer, and a fourth fusion layer.

[0171] The first normalization layer (Layer Norm layer) is used to perform layer normalization on the initial input feature map to obtain the normalized features. Figure 1 And send it to the Local-Attention module, the local attention mechanism;

[0172] The Local Attention module is used to process the normalized features sent to the first normalized Layer Norm layer through a local attention mechanism. Figure 1 Attention calculation is performed to obtain features. Figure 2 And send it to the first fusion layer;

[0173] The first fusion layer is used to process the feature map of the initial input and the features output by the Local-Attention module. Figure 2 Perform feature fusion to obtain the fused features. Figure 3 It is then sent to the second normalized Layer Norm layer and the second fusion layer.

[0174] The second normalized Layer Norm layer is used to process the fused features output by the first fusion layer. Figure 3 Perform layer normalization to obtain normalized features. Figure 4 And send it to the first hybrid feedforward network Mix-FFN layer;

[0175] The first Mix-FFN layer of the hybrid feedforward network is used to normalize the features output by the second normalized Layer Norm layer. Figure 4 Further nonlinear transformations and feature extraction are performed to obtain the features. Figure 5 And send it to the second fusion layer;

[0176] The second fusion layer is used to process the features output by the first Mix-FFN layer of the hybrid feedforward network. Figure 5 Features output by the first fusion layer Figure 3 Perform feature fusion to obtain the fused features. Figure 6 And send it to the Global-Attention module and the third fusion layer;

[0177] The Global-Attention module is used to process the features output by the second fusion layer through a global attention mechanism. Figure 6 Attention calculation is performed to obtain features. Figure 7 And send it to the third fusion layer;

[0178] The third fusion layer is used to process the features output by the Global-Attention module. Figure 7 Features output by the second fusion layer Figure 6Feature fusion is performed to obtain the fused feature map 8, which is then sent to the third normalized Layer Norm layer and the fourth fusion layer.

[0179] The third normalization layer is used to normalize the feature map 8 output by the third fusion layer to obtain the normalized feature map 9 and send it to the second hybrid feedforward network Mix-FFN layer.

[0180] The second Mix-FFN layer of the hybrid feedforward network is used to perform further nonlinear transformation and feature extraction on the feature map 9 output by the third normalized Layer Norm layer to obtain feature map 10 and send it to the fourth fusion layer.

[0181] The fourth fusion layer is used to fuse the feature map 10 output by the second Mix-FFN layer with the feature map 8 output by the third fusion layer to obtain the fused feature map 11 and send it to the next GAL-Transformer Block.

[0182] The feature map output by the fourth fusion layer of the second GAL-Transformer Block in each global fusion local attention mechanism layer is the semantic feature map output by that global fusion local attention mechanism layer.

[0183] Furthermore, the spatial branch consists of four sequentially configured guide extraction modules;

[0184] Each of the guided extraction modules consists of a serially connected Double Conv Block and a Guidance Block.

[0185] The first double convolutional module, Double Conv Block 1, is used to extract features from the image tokens sent by the initial convolutional layer, the convolution stem, to obtain spatial features. Figure 1 And send it to the first bootstrap module, Guidance Block 1;

[0186] The first guidance module, Guidance Block 1, is used to receive spatial features sent by the first double convolutional module, Double Conv Block 1. Figure 1 The semantic features sent by the first global fusion local attention mechanism layer, GAL-Transformer Layer 1 Figure 1 Utilizing semantic features provided by semantic branches Figure 1 To guide the improvement of the spatial characteristics of Double Conv Block1 Figure 1 This leads to the output features. Figure 1 And send it to the second double convolutional module, Double Conv Block 2, and the decoder;

[0187] The second double convolutional module, Double Conv Block 2, is used to process the output features of the first guidance module, Guidance Block 1. Figure 1 Feature extraction is performed to obtain spatial features. Figure 2 And send it to the second boot module GuidanceBlock2;

[0188] The second guidance module, Guidance Block 2, is used to receive spatial features sent by the second double convolutional module, Double Conv Block 2. Figure 2 Semantic features sent by the second global fusion local attention mechanism layer GAL-Transformer Block2 Figure 2 Utilizing stylistic features provided by semantic branches Figure 2 To guide and improve the spatial characteristics of the Double Conv Block2 output Figure 2 This leads to the output features. Figure 2 And send it to the third double convolutional module, Double Conv Block 3, and the decoder;

[0189] The third double convolutional module, Double Conv Block 3, is used to process the output features of the second guidance module, Guidance Block 2. Figure 2 Feature extraction is performed to obtain spatial features. Figure 3 And send it to the third bootstrap module GuidanceBlock3;

[0190] The third guidance module, Guidance Block 3, is used to receive spatial features sent by the third double convolutional module, Double Conv Block 3. Figure 3 Semantic features sent by the third global fusion local attention mechanism layer GAL-Transformer Block3 Figure 3 Utilizing stylistic features provided by semantic branches Figure 3 To guide and improve the spatial characteristics of Double Conv Block3 output Figure 3 This leads to the output features. Figure 3 And send it to the fourth double convolutional module, Double Conv Block 4, and the decoder;

[0191] The fourth double convolutional module, Double Conv Block 4, is used to obtain the output features from the third guidance module, Guidance Block 3. Figure 3 Feature extraction is performed to obtain spatial features. Figure 4 And send it to the fourth boot module, GuidanceBlock4;

[0192] The fourth guidance block (Guidance Block 4) is used to receive spatial features sent by the fourth double convolutional block (Double Conv Block 4). Figure 4 Semantic features sent by the fourth global fusion local attention mechanism layer, GAL-Transformer Block4. Figure 4 Utilizing stylistic features provided by semantic branches Figure 4 To guide and improve the spatial characteristics of Double Conv Block4 output Figure 4 This leads to the output features. Figure 4 And send it to the decoder;

[0193] Each Double Conv Block consists of two serially configured convolutional layers.

[0194] Accurately extracting laser stripe features and distinguishing them from reflective features under multiple reflective interferences is a key challenge. Therefore, a Guidance Block is introduced. This Guidance Block, through feature fusion, can more comprehensively capture laser stripe features and reduce reflective noise interference, thereby significantly improving the accuracy and robustness of laser stripe extraction. Figure 5 As shown, the guidance module includes a first convolutional layer, a second convolutional layer, a similarity calculation module, a third convolutional layer, a similarity mapping module, a first feature selection module, a second feature selection module, and a similarity fusion module;

[0195] The first convolutional layer is used to further extract features from the spatial feature map, obtain the spatial feature extraction map, and send it to the similarity calculation module;

[0196] The second convolutional layer is used to further extract features from the semantic feature map, obtain the semantic feature extraction map, and send it to the similarity calculation module;

[0197] The similarity calculation module calculates the similarity of corresponding pixels between the spatial feature extraction map and the semantic feature extraction map, obtains a similarity metric map, and sends it to the third convolutional layer;

[0198] The method for calculating similarity is as follows:

[0199] map sim=sigmoid(spt·sem) (7)

[0200] Among them, map sim For similarity measurement graphs; sigmoid represents the sigmoid function; spt represents the spatial feature extraction graph; sem represents the semantic feature extraction graph.

[0201] The third convolutional layer is used to further extract features from the similarity metric map to obtain a similarity metric feature extraction map and send it to the similarity mapping module;

[0202] The similarity mapping module is used to apply the sigmoid function to the similarity metric feature extraction map to obtain a similarity mapping map and send it to the first feature selection module and the second feature selection module;

[0203] The first feature selection module is used to multiply the similarity map by the spatial feature map to obtain a spatial feature selection map and send it to the similarity fusion module;

[0204] The second feature selection module is used to multiply the similarity map by the semantic feature map to obtain a semantic feature selection map and send it to the similarity fusion module;

[0205] The similarity fusion module is used to fuse the spatial feature selection map and the semantic feature selection map in order to guide the spatial branch to extract more accurate laser stripe features and obtain the output feature map.

[0206] The calculation method for feature selection and similarity fusion is as follows:

[0207] out fusion =(1-map) sim )·spt featuremap +map sim ·sem featuremap (8)

[0208] Among them, out fusion To output feature maps; map sim Represents a similarity metric graph; spt featuremap Represents spatial feature maps; sem featuremap Represents semantic feature maps;

[0209] Multi-feature fusion is of great significance in semantic segmentation tasks. It can significantly improve feature representation capabilities, enhance the model's generalization ability, improve the accuracy and consistency of segmentation results, and enhance the model's robustness to noise and variations. Therefore, the decoder adopts a multi-feature fusion approach. The decoder is as follows... Figure 6The module includes a first importance calculation module, a second importance calculation module, a third importance calculation module, a fourth importance calculation module, a first important feature extraction module, a second important feature extraction module, a third important feature extraction module, a first addition and fusion module, a fourth important feature extraction module, and a second addition and fusion module.

[0210] The first importance calculation module is used to calculate the output features. Figure 1 The importance level is calculated to obtain the importance level. Figure 1 And send it to the first important feature extraction module;

[0211] The second importance calculation module is used to calculate the output features. Figure 2 The importance level is calculated to obtain the importance level. Figure 2 And send it to the second important feature extraction module;

[0212] The third importance calculation module is used to calculate the output features. Figure 3 The importance level is calculated to obtain the importance level. Figure 3 And send it to the third important feature extraction module;

[0213] The fourth importance calculation module is used to calculate the output features. Figure 4 The importance level is calculated to obtain the importance level. Figure 4 And send it to the fourth important feature extraction module;

[0214] Each of the importance calculation modules includes a convolutional layer and a sigmoid function in sequence.

[0215] The method for calculating the importance level is as follows:

[0216]

[0217] Where, x i At represents the i-th output feature map; i imp is the i-th output feature map after passing through the convolutional layer. i This represents the i-th importance graph; conv denotes a convolutional layer; sigmoid denotes the sigmoid function;

[0218] The first important feature extraction module is used to process the output features. Figure 1 And importance Figure 1 Perform dot product operation to obtain important feature extraction. Figure 1 And send it to the first addition and fusion module;

[0219] The second important feature extraction module is used to process the output features. Figure 2 And importance Figure 2 Perform dot product operation to obtain important feature extraction. Figure 2 And send it to the first addition and fusion module;

[0220] The third important feature extraction module is used to process the output features. Figure 3 And importance Figure 3 Perform dot product operation to obtain important feature extraction. Figure 3 And send it to the first addition and fusion module;

[0221] The first addition and fusion module is used to determine the degree of importance. Figure 1 Importance Figure 2 And importance Figure 3 Perform an addition operation to obtain the sum. Figure 1 And send it to the fourth important feature extraction module;

[0222] The fourth important feature extraction module is used for addition and fusion. Figure 1 And importance Figure 4 Perform dot product operation to obtain important feature extraction. Figure 4 And send it to the second addition and fusion module;

[0223] The second addition and fusion module is used for extracting important features. Figure 4 Output features Figure 4 Perform an addition operation to obtain the sum. Figure 2 That is, the final output segmentation result;

[0224] The segmentation result is expressed as follows:

[0225]

[0226] Among them, out decoder The result of the segmentation;

[0227] Step 8: Train the GuidNet model for a set number of rounds using the training set to obtain the trained GuidNet model;

[0228] In this implementation, the network was trained on an NVIDIA GeForce RTX 3070 for 300 epochs without using any pre-trained weights. The gradient descent optimizer used was an SGD optimizer with an initial learning rate of 0.05, momentum of 0.9, and weight decay of 0.0001. The learning rate scheduler used was a cosine annealing (Cosine AnnealingLR) scheduler, which adjusts the learning rate using a cosine function, allowing for a smooth reduction of the learning rate during training and helping the model converge to a better state. The loss function was calculated jointly using CELoss and DiceLoss, as shown in the formula:

[0229] Loss = 0.4 * Loss ce+0.6*Loss Dice (11)

[0230] Loss ce This is the cross-entropy loss function, which measures the difference between the target pixel and the predicted pixel category. Its goal is to minimize the difference between the predicted result and the true label, thus making the model's prediction closer to reality. Calculation formula:

[0231]

[0232] Among them, Loss ce Cross-entropy loss; N represents the number of samples; C represents the number of classes; y c The weight coefficient for category c is represented by n; the pixel number is represented by x. n c This indicates that the nth pixel is predicted to be of class c; x n j This indicates that the nth pixel is predicted to be of class j;

[0233] Loss Dice This is a loss function used for image segmentation tasks. Its purpose is to evaluate the model's performance by measuring the overlap between the predicted light stripe image and the ground truth label. Its goal is to maximize the overlap between the predicted result and the ground truth label, thereby enabling the model to more accurately capture the boundaries and shape of the target. Calculation formula:

[0234]

[0235] Among them, Loss Dice The image height is h; the image width is w; and the image height is q. n p is the predicted value of the nth pixel. n is the true value of the nth pixel; e is a smoothing coefficient to prevent division by zero, which is 1.0 × 10 in this embodiment of the invention. -5 ;

[0236] Step 9: Input the test set into the GuidNet model to obtain the laser stripe segmentation results;

[0237] Step 9.1: Input the two laser stripe images and label images containing various reflective interferences from the test set into the initial convolution stem to obtain image tag tokens;

[0238] Step 9.2: Input the image tag tokens into the encoder, and use spatial branching and semantic branching to perform step-by-step feature extraction to obtain several output feature maps at different levels;

[0239] Step 9.3: Input the output feature maps of several different levels into the decoder to obtain the laser stripe segmentation results;

[0240] To verify the performance of the semantic segmentation network designed in this invention in extracting laser stripes under multiple reflective interferences, we conducted comparative experiments using a self-made dataset. All experiments were performed on an NVIDIA GeForce RTX 3070 graphics card, and the experimental results are shown in Table 1. The results demonstrate that our designed semantic segmentation algorithm achieves superior performance in extracting reflective laser stripes. Furthermore, our algorithm has only 13.6M parameters and a computational cost of 14.8G, significantly lower than other complex networks such as Unet and Transception. Notably, our algorithm achieves a laser stripe segmentation speed of 28.1 FPS, meeting the requirements for real-time laser stripe recognition.

[0241] To further validate our model's segmentation ability on real laser stripe images, we selected four representative images and demonstrated the algorithm's ability to recognize laser stripes under different conditions, such as... Figure 7 As shown in the figure, our algorithm can robustly identify laser stripes affected by reflective interference, and compared with MISSFormer and Transception methods, the segmented edges of our algorithm are closer to the real image. Notably, even with poor segmentation results, the method of this invention still achieves better edge prediction. This further demonstrates the effectiveness and superiority of the invention.

[0242] In summary, based on this experimental evidence, we can conclude that our designed semantic segmentation algorithm is superior in extracting laser stripes under multiple reflective interferences, has lower model complexity and computational cost, and exhibits robust segmentation capabilities in real-world scenarios.

[0243] Table 1. Comparison results with advanced methods (Figure)

[0244]

Claims

1. A method for semantic segmentation of laser stripes under multiple reflective interferences, characterized in that, Includes the following steps: Step 1: Obtain several laser stripe images containing various types of reflective interference. Use data augmentation to add salt-and-pepper noise to the laser stripe images to obtain the same number of laser stripe images with salt-and-pepper noise containing various types of reflective interference as the original images containing various types of reflective interference. The various types of reflective interference include specular reflection, scattered reflection, multiple reflections, and refraction. Step 2: Normalize the two laser stripe images containing multiple reflective interferences obtained in Step 1 to a set size, and manually label the two laser stripe images containing multiple reflective interferences to generate several label images, thereby obtaining a reflective interference dataset; the reflective interference dataset consists of two laser stripe images containing multiple reflective interferences and label images. Step 3: Divide the reflective interference dataset according to a set ratio to obtain the training set, validation set, and test set; Step 4: Preprocess the training set, validation set, and test set to obtain preprocessed training set, validation set, and test set; the preprocessing method is to reduce the image size to a set size and randomly apply one or more data augmentation methods for data augmentation; the data augmentation methods include flipping, rotating, and translating; Step 5: Design a local attention mechanism, Local-Attention; Step 6: Design a global attention mechanism, Global-Attention; Step 7: Build the GuidNet model based on local and global attention mechanisms; In step 7, the GuidNet model includes a convolution stem, an encoder, and a decoder. The initial convolution stem is used to extract features from the input laser stripe image containing various reflective interferences, obtain image tag tokens, and send them to the encoder. The encoder is used to extract deep semantic and spatial features from image tokens, and uses the semantic features to guide and enhance the feature representation of spatial features, while reducing the influence of reflection interference, to obtain several output feature maps at different levels and send them to the decoder. The decoder is used to calculate the importance of several output feature maps at different levels and aggregate several output feature maps according to their importance to obtain the final dense semantic segmentation result. Step 8: Train the GuidNet model for a set number of rounds using the training set to obtain the trained GuidNet model; Step 9: Input the test set into the GuidNet model to obtain the laser stripe segmentation results.

2. The method for semantic segmentation of laser stripes under multiple reflective interference as described in claim 1, characterized in that, Step 9 includes the following specific steps: Step 9.1: Input the two laser stripe images and label images containing various reflective interferences from the test set into the initial convolution stem to obtain image tag tokens; Step 9.2: Input the image tag tokens into the encoder, and use spatial branching and semantic branching to perform step-by-step feature extraction to obtain several output feature maps at different levels; Step 9.3: Input the output feature maps of several different levels into the decoder to obtain the laser stripe segmentation results.

3. The method for semantic segmentation of laser stripes under multiple reflective interference as described in claim 1, characterized in that, The Local Attention mechanism described in step 5 transforms the input feature map into a query tensor, a key tensor, and a value tensor using a linear mapping method. It then performs average pooling on the query tensor and the key tensor according to a defined region to obtain the matrix Q. avg and K avg Then calculate Q avg and (K) avg ) T The product of these terms yields the region-to-region relation matrix Rel. Applying the topk operator to Rel finds the k most relevant regions for each region, resulting in the most relevant region index matrix Idx. The gather method is then used to collect the key and value tensors using the index matrix Idx, thus obtaining the collected key tensor K. g and the collection value tensor V g Finally, using the query tensor Query and the collection key tensor K... g and the collection value tensor V g Attention is calculated to obtain the local attention feature map Attn. local ; The relation matrix Rel is: (1) Where Rel is the relation matrix; To query the matrix obtained by average pooling of the tensor; The matrix obtained by average pooling the key tensor; The most relevant region index matrix Idx is: (2) Where Idx is the index matrix of the most relevant regions; topk represents the topk operator; The collection key tensor and collection value tensor are: (3) Among them, K g To collect key tensors; V g For collecting value tensors; K is the key tensor; V is the value tensor; gather represents the gather method of torch; The method for calculating attention is as follows: (4) in, This is a local attention feature map; express function; Q represents the number of channels in the input feature map; Q is the query tensor.

4. The method for semantic segmentation of laser stripes under multiple reflective interference as described in claim 1, characterized in that, The Global Attention mechanism described in step 6 transforms the input feature map into a query tensor Q', a key tensor K', and a value tensor V' using a linear mapping method. Then, it performs softmax normalization on the rows and columns of the query tensor Q' and the key tensor K' respectively to obtain the row-normalized query tensor Q'. softmax Normalized bond tensor K softmax Finally, attention is calculated using the row-normalized query tensor, column-normalized key tensor, and value tensor to obtain the global attention feature map Attn. global ; The row-normalized query tensor and column-normalized key tensor are: (5) Among them, Q softmax Let Q' be the row-normalized query tensor; Q' be the query tensor; K' be the key tensor; K softmax For column-normalized bond tensors; softmax row This indicates that the softmax function is applied to the rows; softmax col This indicates that the softmax function is applied to the column; d head This indicates the number of channels in the input feature map; The method for calculating attention is as follows: (6) Among them, Attn global For global attention feature maps; Q softmax K represents the row-normalized query tensor; softmax V' represents the column normalized bond tensor; V' represents the value tensor.

5. A method for semantic segmentation of laser stripes under multiple reflective interference as described in claim 1, characterized in that, The initial convolution stem consists of three sequentially configured convolution groups, each consisting of two sequentially configured convolution layers.

6. The laser stripe semantic segmentation method for multiple reflective interference as described in claim 1, characterized in that, The encoder has a dual-branch structure, including a semantic branch and a spatial branch. The semantic branch is used to simultaneously obtain semantic features at different levels from image tag tokens, obtain several semantic feature maps at different levels, and send them to the spatial branch. The Spatial Branch is used to obtain spatial features at different levels from image tokens, resulting in several spatial feature maps at different levels; it receives semantic feature maps at different levels sent by the Semantic Branch and corrects the spatial feature maps based on the semantic feature maps at different levels, resulting in several output feature maps at different levels, which are then sent to the decoder; the spatial features are the shape features and edge features of the laser stripes.

7. A method for semantic segmentation of laser stripes under multiple reflective interference as described in claim 6, characterized in that, The semantic branch consists of four globally fused local attention mechanism layers (GAL-Transformer Layer) set sequentially. The first global fusion local attention mechanism layer GAL-Transformer Layer 1 is used to perform multi-head self-attention calculation on image tokens to obtain semantic feature map 1 and send the obtained semantic feature map 1 to the second global fusion local attention mechanism layer GAL-Transformer Layer 2 and the first guidance module Guidance Block 1. The second global fusion local attention mechanism layer GAL-Transformer Layer 2 is used to further perform multi-head self-attention calculation on the semantic feature map 1 to obtain the semantic feature map 2, and then send the obtained semantic feature map 2 to the third global fusion local attention mechanism layer GAL-Transformer Layer 3 and the second guidance module Guidance Block 2. The third global fusion local attention mechanism layer GAL-Transformer Layer 3 is used to further perform multi-head self-attention calculation on the semantic feature map 2 to obtain the semantic feature map 3, and then send the obtained semantic feature map 3 to the fourth global fusion local attention mechanism layer GAL-Transformer Layer 4 and the third guidance module Guidance Block 3. The fourth global fusion local attention mechanism layer, GAL-Transformer Layer 4, is used to further perform multi-head self-attention calculation on the semantic feature map 3 to obtain the semantic feature map 4, and then sends the obtained semantic feature map 4 to the fourth guidance module, Guidance Block 4. Each of the Global Fusion Local Attention Mechanism Layers (GAL-Transformer Layer) consists of two serially configured Global Fusion Local Attention Blocks (GAL-Transformer Blocks). The GAL-Transformer Block consists of a first normalized Layer Norm layer, a Local-Attention module, a first fusion layer, a second normalized Layer Norm layer, a first Mix-FFN layer, a second fusion layer, a Global-Attention module, a third fusion layer, a third normalized Layer Norm layer, a second Mix-FFN layer, and a fourth fusion layer. The first normalization layer is used to normalize the initial input feature map to obtain the normalized feature map 1 and send it to the local attention mechanism module. The Local-Attention module is used to perform attention calculation on the normalized feature map 1 sent by the first normalized Layer Norm layer through the local attention mechanism, to obtain feature map 2 and send it to the first fusion layer. The first fusion layer is used to fuse the feature map of the initial input and the feature map 2 output by the Local-Attention module to obtain the fused feature map 3 and send it to the second normalized LayerNorm layer and the second fusion layer. The second normalized layer is used to normalize the fused feature map 3 output by the first fusion layer to obtain the normalized feature map 4 and send it to the first hybrid feedforward network Mix-FFN layer. The first Mix-FFN layer of the hybrid feedforward network is used to perform further nonlinear transformation and feature extraction on the normalized feature map 4 output by the second normalized Layer Norm layer to obtain feature map 5 and send it to the second fusion layer. The second fusion layer is used to fuse the feature map 5 output by the Mix-FFN layer of the first hybrid feedforward network with the feature map 3 output by the first fusion layer to obtain the fused feature map 6 and send it to the Global-Attention module and the third fusion layer. The Global-Attention module is used to perform attention calculation on the feature map 6 output by the second fusion layer through the global attention mechanism, obtain the feature map 7, and send it to the third fusion layer. The third fusion layer is used to fuse the feature map 7 output by the Global-Attention module with the feature map 6 output by the second fusion layer to obtain the fused feature map 8 and send it to the third normalized Layer Norm layer and the fourth fusion layer. The third normalization layer is used to normalize the feature map 8 output by the third fusion layer to obtain the normalized feature map 9 and send it to the second hybrid feedforward network Mix-FFN layer. The second Mix-FFN layer of the hybrid feedforward network is used to perform further nonlinear transformation and feature extraction on the feature map 9 output by the third normalized Layer Norm layer to obtain feature map 10 and send it to the fourth fusion layer. The fourth fusion layer is used to fuse the feature map 10 output by the second Mix-FFN layer with the feature map 8 output by the third fusion layer to obtain the fused feature map 11 and send it to the next GAL-Transformer Block. The feature map output by the fourth fusion layer of the second GAL-Transformer Block in each global fusion local attention mechanism layer (GAL-Transformer Layer) is the semantic feature map output by that global fusion local attention mechanism layer.

8. A method for semantic segmentation of laser stripes under multiple reflective interference as described in claim 6, characterized in that, The spatial branch consists of four serially configured guide extraction modules; Each of the guided extraction modules consists of a serially connected Double Conv Block and a Guidance Block. The first double convolutional module, Double Conv Block 1, is used to extract features from the image tokens sent by the initial convolutional layer, convolution stem, to obtain spatial feature map 1 and send it to the first guidance module, GuidanceBlock 1. The first guidance module, Guidance Block 1, receives the spatial feature map 1 sent by the first double convolutional module, Double Conv Block 1, and the semantic feature map 1 sent by the first global fusion local attention mechanism layer, GAL-Transformer Layer 1. It uses the semantic feature map 1 provided by the semantic branch to guide and improve the spatial feature map 1 of Double Conv Block 1, thereby obtaining the output feature map 1 and sending it to the second double convolutional module, Double Conv Block 2, and the decoder. The second double convolutional module, Double Conv Block 2, is used to extract features from the output feature map 1 of the first guidance module, Guidance Block 1, to obtain the spatial feature map 2, and then send it to the second guidance module, Guidance Block 2. The second guidance module, Guidance Block 2, receives the spatial feature map 2 sent by the second double convolutional module, Double Conv Block 2, and the semantic feature map 2 sent by the second global fusion local attention mechanism layer, GAL-Transformer Block 2. It uses the stylistic feature map 2 provided by the semantic branch to guide and improve the spatial feature map 2 output by Double Conv Block 2, thereby obtaining the output feature map 2 and sending it to the third double convolutional module, Double Conv Block 3, and the decoder. The third double convolutional module, Double Conv Block 3, is used to extract features from the output feature map 2 of the second guidance module, Guidance Block 2, to obtain the spatial feature map 3, and then send it to the third guidance module, Guidance Block 3. The third guidance module, Guidance Block 3, receives the spatial feature map 3 sent by the third double convolutional module, Double Conv Block 3, and the semantic feature map 3 sent by the third global fusion local attention mechanism layer, GAL-Transformer Block 3. It uses the stylistic feature map 3 provided by the semantic branch to guide and improve the spatial feature map 3 output by Double Conv Block 3, thereby obtaining the output feature map 3 and sending it to the fourth double convolutional module, Double Conv Block 4, and the decoder. The fourth double convolutional module, Double Conv Block 4, is used to extract features from the output feature map 3 of the third guidance module, Guidance Block 3, to obtain the spatial feature map 4, and then send it to the fourth guidance module, Guidance Block 4. The fourth guidance block 4 receives the spatial feature map 4 sent by the fourth double convolutional block 4 and the semantic feature map 4 sent by the fourth global fusion local attention mechanism layer GAL-Transformer Block 4. It uses the stylistic feature map 4 provided by the semantic branch to guide and improve the spatial feature map 4 output by the double convolutional block 4, thereby obtaining the output feature map 4 and sending it to the decoder. The guidance module includes a first convolutional layer, a second convolutional layer, a similarity calculation module, a third convolutional layer, a similarity mapping module, a first feature selection module, a second feature selection module, and a similarity fusion module; The first convolutional layer is used to further extract features from the spatial feature map, obtain the spatial feature extraction map, and send it to the similarity calculation module; The second convolutional layer is used to further extract features from the semantic feature map, obtain the semantic feature extraction map, and send it to the similarity calculation module; The similarity calculation module calculates the similarity of corresponding pixels between the spatial feature extraction map and the semantic feature extraction map, obtains a similarity metric map, and sends it to the third convolutional layer; The method for calculating similarity is as follows: (7) Among them, map sim For similarity measurement graphs; sigmoid represents the sigmoid function; spt represents the spatial feature extraction graph; sem represents the semantic feature extraction graph. The third convolutional layer is used to further extract features from the similarity metric map to obtain a similarity metric feature extraction map and send it to the similarity mapping module; The similarity mapping module is used to apply the sigmoid function to the similarity metric feature extraction map to obtain a similarity mapping map and send it to the first feature selection module and the second feature selection module; The first feature selection module is used to multiply the similarity map by the spatial feature map to obtain a spatial feature selection map and send it to the similarity fusion module; The second feature selection module is used to multiply the similarity map by the semantic feature map to obtain a semantic feature selection map and send it to the similarity fusion module; The similarity fusion module is used to fuse the spatial feature selection map and the semantic feature selection map to obtain an output feature map; The calculation method for feature selection and similarity fusion is as follows: (8) Among them, out fusion To output feature maps; map sim Represents a similarity metric graph; spt featuremap Represents spatial feature maps; sem featuremap Represents semantic feature maps; Each Double Conv Block consists of two serially configured convolutional layers.

9. A method for semantic segmentation of laser stripes under multiple reflective interference as described in claim 1, characterized in that, The decoder includes a first importance calculation module, a second importance calculation module, a third importance calculation module, a fourth importance calculation module, a first importance feature extraction module, a second importance feature extraction module, a third importance feature extraction module, a first addition and fusion module, a fourth importance feature extraction module, and a second addition and fusion module; The first importance calculation module is used to calculate the importance of the output feature map 1, obtain the importance map 1, and send it to the first important feature extraction module. The second importance calculation module is used to calculate the importance of the output feature map 2, obtain the importance map 2, and send it to the second importance feature extraction module. The third importance calculation module is used to calculate the importance of the output feature map 3, obtain the importance map 3, and send it to the third importance feature extraction module. The fourth importance calculation module is used to calculate the importance of the output feature map 4, obtain the importance map 4, and send it to the fourth importance feature extraction module; Each of the importance calculation modules includes a convolutional layer and a sigmoid function in sequence. The method for calculating the importance level is as follows: (9) in, This represents the i-th output feature map; This is the i-th output feature map after passing through the convolutional layer; This represents the i-th importance graph; conv denotes a convolutional layer; sigmoid denotes the sigmoid function; The first important feature extraction module is used to perform a dot product operation on the output feature map 1 and the importance map 1 to obtain the important feature extraction map 1 and send it to the first addition and fusion module. The second important feature extraction module is used to perform a dot product operation on the output feature map 2 and the importance map 2 to obtain the important feature extraction map 2 and send it to the first addition and fusion module. The third important feature extraction module is used to perform a dot product operation on the output feature map 3 and the importance map 3 to obtain the important feature extraction map 3 and send it to the first addition and fusion module. The first addition and fusion module is used to add importance map 1, importance map 2 and importance map 3 to obtain addition and fusion map 1 and send it to the fourth importance feature extraction module; The fourth important feature extraction module is used to perform a dot product operation on the summation and fusion image 1 and the importance image 4 to obtain the important feature extraction image 4 and send it to the second summation and fusion module; The second addition and fusion module is used to add the important feature extraction map 4 and the output feature map 4 to obtain the addition and fusion map 2, which is the final output segmentation result. The segmentation result is expressed as follows: (10) Among them, out decoder This is the segmentation result.

Images