A Transformer-based Deeplabv3+ underwater laser stripe center extraction method

Abstract

The present invention discloses a method for extracting the center of underwater laser stripes based on Deeplabv3+ of Transformer, which relates to the technical field of underwater machine vision semantic segmentation, including obtaining underwater laser stripe images and preprocessing them; constructing an underwater laser stripe data set based on the preprocessed underwater laser stripe images; constructing a Transformer-based Deeplabv3+ semantic segmentation model according to the underwater laser stripe data set; and extracting using the Deeplabv3+ semantic segmentation model to obtain the center of underwater laser stripes. The present invention effectively improves the accuracy and robustness of underwater laser stripe center extraction by introducing an improved Deeplabv3+ model based on Transformer, especially in the case of processing low contrast and complex backgrounds, can significantly reduce the phenomenon of missed extraction, improve the connectivity and consistency of segmentation results, and thus significantly improve the accuracy and reliability of underwater three-dimensional reconstruction.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater machine vision semantic segmentation, and in particular to a Transformer-based Deeplabv3+ underwater laser stripe center extraction method. Background Art

[0002] In recent years, deep-sea exploration has become increasingly important, and underwater 3D reconstruction technology has become key. Currently, acoustic detection technology is mainly used. Although it has a wide coverage range and strong penetration, it has low resolution and is easily affected by underwater noise, making it difficult to obtain fine information. In contrast, laser technology has high brightness, good collimation and strong anti-interference ability, which enables underwater structured light detection technology to achieve high spatial resolution and intuitive images, and is suitable for close-range high-precision reconstruction.

[0003] The accuracy of underwater structured light 3D reconstruction systems depends on the accuracy of extracting the centerline of laser stripes. However, existing semantic segmentation-based methods face challenges in processing stripes because the features of stripes are limited and difficult to capture accurately. In addition, the propagation of lasers underwater is affected by refraction and scattering effects, which reduces the brightness and contrast of stripes, making it difficult for existing models to effectively segment the laser stripe area from the background. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a Transformer-based Deeplabv3+ underwater laser stripe center extraction method to solve the problem that the laser stripe extraction accuracy is limited by the segmentation difficulties caused by few features, low brightness and low contrast.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a method for extracting the center of an underwater laser stripe based on Deeplabv3+ of Transformer, which includes obtaining an underwater laser stripe image and performing preprocessing;

[0008] Based on the preprocessed underwater laser streak images, an underwater laser streak dataset is constructed;

[0009] According to the underwater laser stripe dataset, an improved Deeplabv3+ semantic segmentation model based on Transformer is constructed;

[0010] The center of the underwater laser stripe is extracted using the improved Deeplabv3+ semantic segmentation model.

[0011] As a preferred solution of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method of the present invention, wherein: the underwater laser stripe image is obtained and preprocessed, including the following steps:

[0012] Acquire underwater laser stripe images through an underwater multi-line laser scanner binocular camera;

[0013] For the noise at the edge of the scan and the irrelevant object areas in the background, image cropping is used to remove the uninteresting parts of the image, and bilateral filtering is used to remove underwater noise.

[0014] As a preferred solution of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method of the present invention, wherein: based on the preprocessed underwater laser stripe image, an underwater laser stripe dataset is constructed, comprising the following steps:

[0015] Use Labelme to annotate the preprocessed underwater laser streak images and construct an underwater laser streak dataset;

[0016] To solve the problem of insufficient sample size of underwater laser stripe images, data enhancement is performed on the underwater laser stripe dataset.

[0017] Convert the labeled json file into an eight-bit color png file. Each pixel value of the label is the type of the corresponding pixel point, expressed as:

[0018] ;

[0019] Among them, D represents the pixel value of the label image converted into an eight-bit color image, b represents the pixel value of the background, l represents the pixel value of the underwater laser stripe, f represents the index of the pixel point, B represents the background, and L represents the underwater laser stripe.

[0020] As a preferred solution of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method described in the present invention, the underwater laser stripe dataset is divided into a training set, a test set and a validation set according to a set ratio, and the training set is used to train the backbone network, deep feature extraction module and loss function of the Deeplabv3+ semantic segmentation model to obtain the trained Deeplabv3+ semantic segmentation model.

[0021] As a preferred solution of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method of the present invention, the backbone network of the Deeplabv3+ semantic segmentation model is constructed, and the original Deeplabv3+ DCNN backbone extraction network Xception is replaced with MobileNet-V2 using an inverted residual structure, including the following steps:

[0022] First, the input feature matrix height × width × channel is increased in dimension through convolution to increase the size of the channel, and then convolution is performed through the DW convolution kernel. Finally, the convolution kernel is used to reduce the dimension, expanding the receptive field of shallow features without increasing parameters, and obtaining the improved backbone network.

[0023] As a preferred solution of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method of the present invention, wherein: a deep feature extraction module is introduced into the Deeplabv3+ semantic segmentation model, including an improved ASPP module and a Transformer encoding structure, and the following steps are included:

[0024] The hole convolution layer is improved to combine parallel and series connection. The output of the first layer is the input of the second layer, and the output of the second layer is the input of the third layer. In this way, the third layer will have the feature information of the first and second layers. The feature information of the center of the laser stripe is superimposed layer by layer, and finally the features of the center of the laser stripe at multiple scales are gathered.

[0025] The improved ASPP module introduces the channel space attention mechanism CBAM when extracting features in the CNN network and inputting the previous layer of hole convolution. The weight of the feature matrix is ​​recalculated by calculating the weights of each channel of the input image.

[0026] The feature map output by the improved ASPP module is encoded with the multi-head self-attention of the Transformer part, and the compression ratio is calculated according to the input compression ratio formula. The expression is:

[0027] ;

[0028] Among them, C o represents the compression ratio, d represents the input dimension, and d f represents the dimension of the forward propagation network;

[0029] Convolution is used to compress the resolution, split the two-dimensional image block into small image blocks and convert them into a one-dimensional image sequence for storage, so that the Transformer model can process the feature map output by the improved ASPP module in parallel by setting the number of heads;

[0030] The input feature matrix is ​​used to calculate the attention of the query matrix, key matrix and value matrix through the linear transformation matrix. The multi-head attention is calculated using the attention of the query matrix, key matrix and value matrix, and the input feature matrix is ​​weighted to obtain the weighted matrix of the input features.

[0031] After the initial module processing is completed, it is used as the input of the next stage Transformer branch and the feature matrix weights are strengthened.

[0032] As a preferred solution of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method of the present invention, wherein: defining the loss function of the Deeplabv3+ semantic segmentation model includes the following steps:

[0033] The cross entropy loss function of Deeplabv3+ is improved to the loss function of the Deeplabv3+ semantic segmentation model based on the combination of weighted cross entropy and Dice loss function. The expression is:

[0034] ;

[0035] Where, L DWCE represents the improved loss function of the Deeplabv3+ semantic segmentation model, W i Represents the weights of different categories, N represents the total number of samples, n represents the number of pixels, y i represents the true label of the i-th sample, It represents the probability of the i-th sample predicted by the model, and i represents the sample index.

[0036] As a preferred solution of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method of the present invention, wherein: the trained Deeplabv3+ semantic segmentation model is optimized using a validation set and a test set, and finally an improved Deeplabv3+ semantic segmentation model is obtained, comprising the following steps:

[0037] The weighted cross entropy combined with Dice Loss is used as the loss function, and the stochastic gradient descent optimizer is used for training. The images in the validation set are input into the model and the accuracy of laser stripe center segmentation extraction is calculated. The model parameters are continuously optimized through an iterative process, and the network is trained until the model converges to obtain the optimal parameters, thus obtaining a trained Deeplabv3+ semantic segmentation model.

[0038] The test set is normalized and input into the trained Deeplabv3+ semantic segmentation model for effect verification, and finally an improved Deeplabv3+ semantic segmentation model is obtained;

[0039] The improved Deeplabv3+ semantic segmentation model is used to extract the newly acquired underwater laser stripe image and obtain the underwater laser stripe center.

[0040] In a second aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method as described in the first aspect of the present invention is implemented.

[0041] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method as described in the first aspect of the present invention is implemented.

[0042] The beneficial effects of the present invention are as follows: the present invention analyzes the characteristics of underwater laser stripe images, strengthens the global characteristics of laser stripes by improving the ASPP module, uses the attention mechanism to enhance the key information in the input data, constructs a Transformer-based deep feature extraction module to extract laser stripe characteristics, and uses improved loss to optimize model parameters, which significantly improves the accuracy of underwater laser stripe extraction and effectively copes with challenges in complex underwater environments. By accurately extracting the centerline of underwater laser stripes, the present invention helps to improve the accuracy of underwater three-dimensional reconstruction and provides important support for scientific research and applications in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0044] Figure 1 This is a flow chart of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method in Example 1.

[0045] Figure 2 Schematic diagram of the structure of the Transformer-based Deeplabv3+ underwater laser stripe center network model in Example 1.

[0046] Figure 3 Schematic diagram of the improved ASPP module and Transformer deep feature extraction module proposed in Example 1.

[0047] Figure 4 This is a comparison chart of the experimental segmentation extraction results of the network model structure in Example 1 and the existing semantic segmentation model. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] Example 1, reference Figure 1 , Figure 2 , Figure 3 and Figure 4 , which is the first embodiment of the present invention, and provides a method for extracting the center of underwater laser stripes based on Deeplabv3+ of Transformer, comprising the following steps:

[0052] S1. Obtain underwater laser stripe images and perform preprocessing.

[0053] Acquire underwater laser stripe images through an underwater multi-line laser scanner binocular camera;

[0054] For the noise at the edge of the scan and the irrelevant object areas in the background, image cropping is used to remove the uninteresting parts of the image, and bilateral filtering is used to remove underwater noise.

[0055] It should be noted that the underwater laser stripe image is acquired by the underwater multi-line laser scanner binocular camera, and the underwater noise is removed by bilateral filtering, which improves the image quality and facilitates subsequent processing.

[0056] S2. Based on the preprocessed underwater laser streak images, an underwater laser streak dataset is constructed.

[0057] Use Labelme (image annotation tool) to annotate the preprocessed underwater laser streak images and construct an underwater laser streak dataset;

[0058] To solve the problem of insufficient sample size of underwater laser stripe images, data enhancement is performed on the underwater laser stripe dataset.

[0059] Convert the annotated json (JavaScript Object Notation) file into an eight-bit color png (Portable Network Graphics) format file. Each pixel value of the label is the type of the corresponding pixel point, expressed as:

[0060] ;

[0061] Wherein, D represents the pixel value of the label image converted into an eight-bit color image, b represents the pixel value of the background, l represents the pixel value of the underwater laser stripe, f represents the index of the pixel point, B represents the background, and L represents the underwater laser stripe. The present invention sets the pixel value of the background to 0 and the label value to 0, and the pixel value of the underwater laser stripe to 155 and the label value to 1.

[0062] It should be noted that Labelme is used to annotate the preprocessed underwater laser stripe images, which solves the problem of insufficient sample size and increases sample diversity through data enhancement, thereby improving the generalization ability of the model.

[0063] S3. Based on the underwater laser stripe dataset, an improved Deeplabv3+ semantic segmentation model based on Transformer is constructed.

[0064] S3.1. Divide the underwater laser stripe dataset into training set, test set and validation set according to the set ratio. Use the training set to train the backbone network, deep feature extraction module and loss function of the Deeplabv3+ semantic segmentation model to obtain the trained Deeplabv3+ semantic segmentation model.

[0065] S3.1.1. Build the backbone network of the Deeplabv3+ semantic segmentation model, and replace the original Deeplabv3+ DCNN (deep convolutional neural network) backbone extraction network Xception (Xception network) with MobileNet-V2 using an inverted residual structure, including the following steps:

[0066] First, the input feature matrix height × width × channel is increased by 1x1 convolution to increase the channel size, and convolution is performed by 3×3 DW convolution kernel. Finally, the shallow features are obtained by 1x1 convolution kernel, which expands the receptive field of the shallow features without increasing the parameters. After replacing the backbone network, the number of shallow feature channels and deep feature channels of the Xception input network are reduced, from 256 to 24, and the number of deep feature channels is reduced from 2048 to 320, which reduces the number of parameters of the backbone extraction network. Finally, the improved backbone network is obtained.

[0067] It should be noted that the DCNN backbone extraction network Xception of the original Deeplabv3+ is replaced by MobileNet-V2 with an inverted residual structure, which reduces the number of parameters, reduces the computational burden, and improves the model efficiency.

[0068] S3.1.2. Introducing the deep feature extraction module into the Deeplabv3+ semantic segmentation model includes the improved ASPP (atrous spatial pyramid pooling) module and the Transformer encoding structure, including the following steps:

[0069] The pixel width of underwater laser stripes in the image is only 10-15 pixels, and the filling rate of the traditional ASPP module is greater than the width information of the laser stripes. The ASPP module of the present invention optimizes the hole convolution rate to 1, 3, and 5;

[0070] The hole convolution layer is improved to combine parallel and series connection. The output of the first layer is the input of the second layer, and the output of the second layer is the input of the third layer. In this way, the third layer will have the feature information of the first and second layers. The feature information of the center of the laser stripe is superimposed layer by layer, and finally the center features of the laser stripe at multiple scales are gathered. The calculation process is:

[0071] The receptive field size of a single dilated convolution is expressed as:

[0072] ;

[0073] In the formula, F i represents the receptive field obtained after convolution, k represents the convolution kernel size, r represents the dilation rate, when the convolution kernel is 3, the receptive field F1 is 3, r3=3, the receptive field F3 is 7, r3=5, and the receptive field F5 is 11;

[0074] The receptive field size after the superposition of two dilated convolutions is expressed as:

[0075] ;

[0076] In the formula, F represents the receptive field obtained by convolution after superposition, F1 and F2 represent the receptive fields of two different dilated convolutions, m represents the number of dilated convolution layers, and the receptive field sizes of the superposed dilated convolutions are 3, 10, and 18;

[0077] The improved ASPP module introduces the channel space attention mechanism CBAM (channel space attention module) when extracting features in the CNN network and inputting the previous layer of hole convolution. The weight of the feature matrix is ​​recalculated by calculating the weights of each channel of the input image 49×46×304.

[0078] The five-way 49×46×256 feature map output by the improved ASPP module is encoded with the multi-head self-attention of the Transformer. The compression ratio is calculated according to the input compression ratio formula. The expression is:

[0079] ;

[0080] Among them, C o represents the compression ratio, d represents the input dimension, and d f Represents the dimension of the forward propagation network.

[0081] In summary, the improved ASPP module in the present invention divides the output of MobileNet-V2 into five parts for parallel processing, including 1-way 1×1 convolution, 3×3 convolution with a void rate of 1, features after fusion of CBAM (channel space attention) and 3×3 convolution with a void rate of 1, as input of 3×3 convolution with a void rate of 3, features after fusion of CBAM (channel space attention) and 3×3 convolution with a void rate of 3, as input of 3×3 convolution with a void rate of 5, and a global average pooling to obtain the overall global information of the image, the output of the improved ASPP module containing the center features of the laser stripes at multiple scales is adjusted and fused through 1×1 convolution as the input of the Transformer encoding, the convolution is used for resolution compression, the position encoding is used to divide the two-dimensional image block into small image blocks and converted into a one-dimensional image sequence for storage, so that the Transformer model processes the feature map output by the improved ASPP module in parallel by setting the number of heads (heads in the multi-head attention mechanism);

[0082] The input feature matrix is ​​used to calculate the attention of the query matrix, key matrix and value matrix through the linear transformation matrix. The attention of the query matrix, key matrix and value matrix is ​​used to calculate the multi-head attention, and the input feature matrix is ​​weighted to obtain the weighted matrix of the input feature, which is expressed as:

[0083] ;

[0084]

[0085]

[0086]

[0087] Where, d h represents the dimension of K, Q represents the query matrix, K represents the key matrix, V represents the value matrix, A E represents the attention of the E-th group, E represents the E-th head in the multi-head attention, E=(1, 2, ..., e), e=10, H represents the input feature matrix, W h represents the multi-head attention weight, Represents the output weight matrix, M A represents multi-head attention, SoftMax represents the activation function, and concat represents the connection operation;

[0088] After being processed by the multi-head self-attention mechanism, it is added to the original input through a residual connection and a layer normalization operation is performed to accelerate the training process. Next, the output is sent to a fully connected feedforward neural network for processing. After a second residual connection and layer normalization, the weighted matrix of the input features is finally obtained. After 1×1 convolution and 4 times upsampling, it is fused with the shallow features of the encoding part with edge information after 1×1 convolution to restore the image size. For the restored image, 3×3 convolution is used to enhance feature extraction. After a 4-fold upsampling, it is restored to the original image size, and the pixels containing laser stripe information are assigned values ​​to obtain the final predicted image.

[0089] The number of layers in the Transformer encoding part is set to 2. After the initial module is processed, it is used as the input of the Transformer branch in the next stage, so that the network can capture the complex dependencies and contextual information in the input sequence layer by layer.

[0090] It should be noted that the feature extraction capability is enhanced by combining parallel and serial connections, the introduction of the channel space attention mechanism CBAM improves the feature representation capability, and the use of the Transformer encoding part of the multi-head self-attention mechanism improves the model's ability to capture complex dependencies and contextual information.

[0091] S3.1.3. Define the loss function of the Deeplabv3+ semantic segmentation model, including the following steps:

[0092] Improvements are made to the original cross entropy loss function, expressed as:

[0093] ;

[0094] Among them, L W-CERepresents the weighted average cross entropy loss, which is used to measure the difference between the model prediction and the true label, W i Represents the weights of different categories, N represents the total number of samples, n represents the number of pixels, y i represents the true label of the i-th sample, Represents the probability of the i-th sample predicted by the model, where i represents the sample index;

[0095] Based on the weighted cross entropy loss function, the Dice Loss function is introduced to alleviate the problem of sample imbalance. The Dice (Dice coefficient) loss function is a function that measures the similarity between two samples. It can alleviate the negative impact of the imbalance between the background and laser ratio in the data set. The expression of the Dice Loss function is:

[0096] ;

[0097] ;

[0098] Where |x| represents the number of pixels marked as laser stripes in the annotated image, |y| represents the number of pixels marked as laser stripes in the model prediction output pixels, and |x∩y| represents the intersection between the annotated image and the predicted image, that is, the number of pixels marked as laser stripe pixels at the same time;

[0099] The cross entropy loss function of Deeplabv3+ is improved to the loss function of the Deeplabv3+ semantic segmentation model based on the combination of weighted cross entropy and Dice loss function. The weighted cross entropy loss function helps to increase the attention to small samples by setting the weights of different samples. The Dice loss function directly focuses on the overlap between the laser stripe prediction results and the true label. The expression is:

[0100] ;

[0101] Where, L DWCE Represents the improved loss function of the Deeplabv3+ semantic segmentation model.

[0102] It should be noted that by combining weighted cross entropy loss and Dice Loss, the class imbalance problem is solved, the attention to small samples is improved, and the overlap between the laser stripe prediction results and the true labels is directly focused, which improves the prediction accuracy of the model.

[0103] S3.2. Use the validation set and test set to optimize the trained Deeplabv3+ semantic segmentation model, including the following steps:

[0104] The weighted cross entropy combined with Dice Loss is used as the loss function, the batch size is set to 8, and the stochastic gradient descent (SGD) optimizer is used for training with a learning rate of 0.007. The images in the validation set are input into the model and the accuracy of the laser stripe center segmentation extraction is calculated. The model parameters are continuously optimized through an iterative process to improve the model's prediction performance. The network is trained until the model converges and the optimal parameters are obtained to obtain the trained Deeplabv3+ semantic segmentation model.

[0105] The test set is normalized and input into the trained Deeplabv3+ semantic segmentation model for effect verification, and finally an improved Deeplabv3+ semantic segmentation model is obtained.

[0106] It should be noted that by combining the weighted cross entropy with the Dice Loss loss function and using the SGD optimizer for training, the prediction performance of the model was improved, and an improved Deeplabv3+ semantic segmentation model was obtained.

[0107] S4. Use the improved Deeplabv3+ semantic segmentation model to extract the newly collected underwater laser stripe image and obtain the underwater laser stripe center.

[0108] In summary, the present invention improves the accuracy of underwater laser stripe center extraction through a series of technical means, overcomes the problems existing in traditional methods, such as narrow laser stripe width, low brightness, low contrast, etc., and improves the imaging accuracy of the underwater structured light 3D reconstruction system.

[0109] This embodiment also provides a computer device, which is suitable for the Transformer-based Deeplabv3+ underwater laser stripe center extraction method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the Transformer-based Deeplabv3+ underwater laser stripe center extraction method proposed in the above embodiment.

[0110] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0111] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the Transformer-based Deeplabv3+ underwater laser stripe center extraction method proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, disk or optical disk.

[0112] In summary, the present invention analyzes the characteristics of underwater laser stripe images, strengthens the global characteristics of laser stripes by improving the ASPP module, uses the attention mechanism to enhance the key information in the input data, constructs a Transformer-based deep feature extraction module to extract laser stripe features, and uses improved loss to optimize model parameters, which significantly improves the accuracy of underwater laser stripe extraction and effectively copes with challenges in complex underwater environments. By accurately extracting the centerline of underwater laser stripes, the present invention helps to improve the accuracy of underwater three-dimensional reconstruction and provides important support for scientific research and applications in related fields.

[0113] Example 2

[0114] Referring to Table 1, which is the second embodiment of the present invention, in order to further verify the technical solution of the present invention, experimental simulation data of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method are given.

[0115] In order to verify the effectiveness of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method, a detailed experimental design and implementation were carried out. First, a series of underwater laser stripe RGB (color model) images were acquired by the underwater multi-line laser scanner binocular camera. Subsequently, these images were preprocessed, including image cropping to remove irrelevant background information and using bilateral filtering to reduce the influence of underwater-specific noise.

[0116] An underwater laser streak dataset was constructed, which included labeling the laser streak images using Labelme software to obtain the label information required for training and testing. Considering that the dataset may have insufficient sample size, a variety of data enhancement methods were used, such as brightness enhancement, contrast enhancement, and position transformation, to ensure the generalization and richness of the dataset. The annotated json file was converted into an eight-bit color PNG format label file, where the background pixel value is 0 and the laser streak pixel value is 155. The dataset is divided into a training set and a test set in a ratio of 9:1.

[0117] Next, a Transformer-based Deeplabv3+ underwater laser stripe center extraction model was built. The core improvement of this model is to replace the Xception backbone network in the original Deeplabv3+ with MobileNet-V2, and add an improved ASPP module and Transformer encoding structure to the model. The improved ASPP module optimizes the convolution rate of the dilated convolution to adapt to the width characteristics of the laser stripes. The Transformer encoding structure enhances the model's ability to capture long-distance dependencies.

[0118] During the training process, a loss function based on weighted cross entropy combined with Dice was used to solve the problem of sample imbalance and increase the focus on small samples. The model training used the stochastic gradient descent (SGD) optimizer, with a learning rate of 0.007 and a batch size of 8. Through continuous iterative optimization, the model converged and the optimal parameters were obtained.

[0119] Finally, the trained model was evaluated using the validation set and predictions were made on the test set to verify the performance of the model. In addition, the method was compared with other classic semantic segmentation models such as Pspnet (Pyramid Scene Parsing Network), Hrnet (High Resolution Network) and the standard Deeplabv3+.

[0120] The details are shown in Table 1:

[0121] Table 1 Experimental record table

[0122]

[0123] As can be seen from Table 1, the Transformer-based Deeplabv3+ underwater laser stripe center extraction method outperforms the traditional semantic segmentation model in all evaluation indicators, especially in key indicators such as MIOU (intersection over union), MPA (pixel accuracy) and F1 score. Specifically, the MIOU of this method reached 92.82%, which is much higher than 72.58% of pspnet, and is also better than 90.71% of HRnet and 91.6% of Deeplabv3+. This shows that this method can more accurately segment the laser stripe area, especially in complex underwater environments, where this performance advantage is more obvious.

[0124] In terms of pixel accuracy MPA, this method has reached 96.93%, which is significantly higher than other models. This shows that this method is more accurate in pixel-level classification and can effectively avoid misclassification. At the same time, this method also performs well in indicators such as mprecision (average precision), mRecall (average recall), and F1 score (comprehensive precision and recall), which further proves the powerful ability of this method in dealing with underwater laser stripe segmentation tasks.

[0125] Under low brightness and low contrast conditions, the underwater laser stripes extracted by other models have more interruptions, while the underwater laser stripe segmentation results obtained by this method have better connectivity and almost no missed extraction. This means that this method can effectively cope with various challenges in underwater environments, such as light changes and image noise, while maintaining high accuracy.

[0126] In summary, the Transformer-based Deeplabv3+ underwater laser stripe center extraction method significantly improves the model's ability to extract laser stripe features through the improved ASPP module and Transformer encoding structure, effectively overcoming the limitations of traditional models in dealing with small-size features and complex backgrounds. In addition, using the improved loss function, this method can better handle the sample imbalance problem, further improving the overall performance of the model. These improvements not only improve the robustness and generalization ability of the model, but also provide strong support for underwater 3D reconstruction technology, which helps to improve the accuracy and reliability of underwater structured light detection systems.

[0127] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for extracting the center of underwater laser stripes based on Deeplabv3+ of Transformer, characterized in that: include, Acquire underwater laser stripe images and perform preprocessing; Based on the preprocessed underwater laser streak images, an underwater laser streak dataset is constructed; According to the underwater laser stripe dataset, an improved Deeplabv3+ semantic segmentation model based on Transformer is constructed; The center of underwater laser stripes is extracted using the improved Deeplabv3+ semantic segmentation model; The deep feature extraction module introduced in the Deeplabv3+ semantic segmentation model includes the improved ASPP module and Transformer encoding structure, including the following steps: The hole convolution layer is improved to combine parallel and series connection. The output of the first layer is the input of the second layer, and the output of the second layer is the input of the third layer. In this way, the third layer will have the feature information of the first and second layers. The feature information of the center of the laser stripe is superimposed layer by layer, and finally the features of the center of the laser stripe at multiple scales are gathered. The improved ASPP module introduces the channel space attention mechanism CBAM when extracting features from the CNN network and inputting the previous layer of hole convolution. The weight of the feature matrix is ​​recalculated by calculating the weights of each channel of the input image. The feature map output by the improved ASPP module is encoded with the multi-head self-attention of the Transformer part, and the compression ratio is calculated according to the input compression ratio formula. The expression is: Among them, C o represents the compression ratio, d represents the input dimension, and d f represents the dimension of the forward propagation network; Convolution is used to compress the resolution, split the two-dimensional image block into small image blocks and convert them into a one-dimensional image sequence for storage, so that the Transformer model can process the feature map output by the improved ASPP module in parallel by setting the number of heads; The input feature matrix is ​​used to calculate the attention of the query matrix, key matrix and value matrix through the linear transformation matrix. The multi-head attention is calculated using the attention of the query matrix, key matrix and value matrix, and the input feature matrix is ​​weighted to obtain the weighted matrix of the input feature. After the initial module processing is completed, it is used as the input of the next stage Transformer branch and the feature matrix weights are strengthened.

2. The method for extracting the center of underwater laser stripes based on Deeplabv3+ of Transformer as claimed in claim 1, characterized in that: The underwater laser stripe image is obtained and preprocessed, including the following steps: Acquire underwater laser stripe images through an underwater multi-line laser scanner binocular camera; For the noise at the edge of the scan and the irrelevant object areas in the background, image cropping is used to remove the uninteresting parts of the image, and bilateral filtering is used to remove underwater noise.

3. The Transformer-based Deeplabv3+ underwater laser stripe center extraction method according to claim 2, characterized in that: Based on the preprocessed underwater laser stripe image, an underwater laser stripe dataset is constructed, which includes the following steps: Use Labelme to annotate the preprocessed underwater laser streak images and construct an underwater laser streak dataset; To solve the problem of insufficient sample size of underwater laser stripe images, data enhancement is performed on the underwater laser stripe dataset. Convert the labeled json file into an eight-bit color png file. Each pixel value of the label is the type of the corresponding pixel point, expressed as: Among them, D represents the pixel value of the label image converted into an eight-bit color image, b represents the pixel value of the background, l represents the pixel value of the underwater laser stripe, f represents the index of the pixel point, B represents the background, and L represents the underwater laser stripe.

4. The method for extracting the center of underwater laser stripes based on Deeplabv3+ of Transformer as claimed in claim 3, characterized in that: The underwater laser stripe dataset is divided into training set, test set and validation set according to the set ratio. The training set is used to train the backbone network, deep feature extraction module and loss function of the Deeplabv3+ semantic segmentation model to obtain the trained Deeplabv3+ semantic segmentation model.

5. The method for extracting the center of underwater laser stripes based on Deeplabv3+ of Transformer as claimed in claim 4, characterized in that: Construct the backbone network of the Deeplabv3+ semantic segmentation model, and replace the original Deeplabv3+ DCNN backbone extraction network Xception with MobileNet-V2 using an inverted residual structure, including the following steps: First, the input feature matrix height × width × channel is increased in dimension through convolution to increase the size of the channel, and then convolution is performed through the DW convolution kernel. Finally, the convolution kernel is used to reduce the dimension, expanding the receptive field of shallow features without increasing parameters, and obtaining the improved backbone network.

6. The method for extracting the center of underwater laser stripes based on Deeplabv3+ of Transformer as claimed in claim 5, characterized in that: Defining the loss function of the Deeplabv3+ semantic segmentation model includes the following steps: The cross entropy loss function of Deeplabv3+ is improved to the loss function of the Deeplabv3+ semantic segmentation model based on the combination of weighted cross entropy and Dice loss function. The expression is: Where, L DWCE represents the improved loss function of the Deeplabv3+ semantic segmentation model, W i Represents the weights of different categories, N represents the total number of samples, n represents the number of pixels, y i represents the true label of the i-th sample, It represents the probability of the i-th sample predicted by the model, and i represents the sample index.

7. The method for extracting the center of underwater laser stripes based on Deeplabv3+ of Transformer as claimed in claim 6, characterized in that: The trained Deeplabv3+ semantic segmentation model is optimized using the validation set and the test set, and the improved Deeplabv3+ semantic segmentation model is finally obtained, which includes the following steps: The weighted cross entropy combined with Dice Loss is used as the loss function, and the stochastic gradient descent optimizer is used for training. The images in the validation set are input into the model and the accuracy of laser stripe center segmentation extraction is calculated. The model parameters are continuously optimized through an iterative process, and the network is trained until the model converges to obtain the optimal parameters, thus obtaining a trained Deeplabv3+ semantic segmentation model. The test set is normalized and input into the trained Deeplabv3+ semantic segmentation model for effect verification, and finally an improved Deeplabv3+ semantic segmentation model is obtained; The improved Deeplabv3+ semantic segmentation model is used to extract the newly acquired underwater laser stripe image and obtain the underwater laser stripe center.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the Transformer-based Deeplabv3+ underwater laser stripe center extraction method described in any one of claims 1 to 7 are implemented.

Images