Curve structure segmentation method based on double-path attention

Abstract

The invention discloses a curve structure segmentation method based on double-path attention, and the method is specifically implemented according to the following steps: 1, employing a yarn hairiness image through an industrial camera, downloading a public curve-shaped data set, and carrying out the classification arrangement and data enhancement through labeling, thereby obtaining a curve data set; 2, marking the images subjected to data cleaning, and dividing the images into a training set and a test set according to a fixed proportion; step 3, applying a coding and decoding network structure, extracting global features and local features in a double-path mode, and performing feature refinement by adopting a differential feature enhancement module; and 4, loading the divided curve data set into the model, carrying out model training, and finishing the final test of the curve detection model through the test set. The problem of fracture in curve image segmentation is solved, high-precision and complete curve segmentation is realized, and the method is suitable for an image segmentation task in a complex scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of curve image segmentation methods, and specifically relates to a curve structure segmentation method based on dual-path attention. Background Art

[0002] Curved image segmentation has important research significance in many fields. In particular, it provides key information in applications such as medical image analysis (such as blood vessel segmentation and neural network extraction), remote sensing image processing (such as road detection and boundary extraction), and industrial detection (such as crack detection and defect analysis). Curved structures usually have complex geometric shapes and are easily affected by noise and background interference. Therefore, accurately segmenting these curves is crucial to improving the accuracy of downstream tasks. Secondly, curved image segmentation can help optimize the image understanding and feature extraction process. By extracting continuous and complete curves, the geometric information in the image can be better represented, thereby enhancing the effectiveness of subsequent processing tasks.

[0003] However, curve segmentation also faces several challenges: curves often appear in complex backgrounds, and the background texture is similar to the curve, which makes segmentation more difficult and prone to missed detection and false detection. Curves of different sizes and curvatures may lead to uneven segmentation of large-sized defect boundaries, while small-sized defects are easily overlooked. The continuity and integrity of the curve are crucial in segmentation. Traditional methods often cannot effectively maintain the continuity of the curve, resulting in inaccurate segmentation results, especially when dealing with complex branches. Although deep learning methods have made some progress in curve segmentation, they still face problems such as background interference, scale transformation, and noise. The algorithm needs to be further optimized to improve accuracy and robustness. Summary of the invention

[0004] The purpose of the present invention is to provide a curve structure segmentation method based on dual-path attention, which can ensure the integrity of the curve structure segmentation.

[0005] The technical solution adopted by the present invention is a curve structure segmentation method based on dual-path attention, which is specifically implemented according to the following steps: Step 1: Use an industrial camera to take a yarn hairiness image, download a public curve dataset, and obtain a curve dataset by labeling, classification, and data enhancement; Step 2: Label the cleaned images and divide them into training sets and test sets according to a fixed ratio; Step 3: Apply the codec network structure to extract global features and local features through a dual-path method, and use a differential feature enhancement module to refine the features; Step 4: Load the divided curve data set into the model, perform model training, and complete the final test of the curve detection model through the test set.

[0006] The technical solution of the present invention is also characterized in that: In step 1, different curve images are classified according to their features, and the collected images are enhanced by translation and rotation.

[0007] In step 2, the dataset is cleaned of blurred images and images with low resolution after enhancement. The Labelme image annotation tool is used to mark the curve areas in the cleaned images to generate image labels in json format. The OpenCV library is used to generate label images with the same size as the original images, 8-bit RGB color gamut, and PNG format. By writing a script, the annotated dataset is divided into training set and test set in a ratio of 8:2 according to their corresponding defect categories. Finally, it is sorted to generate the feather dataset required for training and testing.

[0008] In step 3, a dual-path encoding and decoding structure is used. One path is a global feature extraction path to accurately locate the backbone area of ​​the curve image; the other path is a local feature extraction path to extract the detail features of the curve image; the features extracted by the two paths are effectively fused.

[0009] In step 3, the global path adopts an encoding-decoding structure. The encoder consists of two triple attentions and a residual attention based on point convolution, and uses cross-dimensional spatial attention for feature extraction; the local path uses residual attention through two point convolutions and introduces dynamic snake convolution. Finally, two 7×7 convolutions are used for local fast feature extraction to ensure that the complete curve structure is extracted.

[0010] In step 3, the differential feature enhancement module refines the intermediate features through layer-by-layer upsampling and differential calculation to enhance the detail and boundary segmentation capabilities.

[0011] In step 4, adjust the model learning parameters, adjust the data set to the output format required by the model, train the constructed neural network model, record and save the parameters and weights generated by the model during the training process, and select the weights with the best training effect for testing after the training is completed.

[0012] In step 4, the training parameters of the curve structure segmentation model are: the number of single-batch training samples is 2, the optimizer uses the Adam optimizer, and the initial learning rate is , the minimum learning rate is , the input image size is , the number of training rounds is 200.

[0013] The beneficial effects of the present invention are: The curve structure segmentation method based on dual-path attention in the present invention combines the global multi-branch attention mechanism, the local multi-layer convolution module and the differential feature enhancement module to effectively improve the accuracy of curve structure segmentation; the global attention mechanism captures cross-dimensional interactive information through the triple attention module, the local convolution module combines with the dynamic snake convolution to strengthen the extraction of local features, and the differential feature enhancement module enhances the detail and boundary segmentation capabilities through layer-by-layer upsampling and differential calculation; these technologies work together to solve the fracture problem in curve image segmentation and achieve high-precision and complete curve segmentation, which is suitable for image segmentation tasks in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is the overall structure diagram of the present invention; Figure 2 It is a diagram of the triple attention structure in the present invention; Figure 3 is the point convolution residual attention map of the present invention; Figure 4 This is a diagram of the differential feature enhancement module in the present invention. DETAILED DESCRIPTION

[0015] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] The present invention is based on the curve structure segmentation method of dual-path attention, which is specifically implemented according to the following steps: Step 1: Use an industrial camera to take a yarn hairiness image, download a public curve dataset, and obtain a curve dataset by labeling, classification, and data enhancement; Step 2: Label the cleaned images and divide them into training sets and test sets according to a fixed ratio; Step 3: Apply the codec network structure to extract global features and local features through a dual-path method, and use a differential feature enhancement module to refine the features; Step 4: Load the divided curve data set into the model, perform model training, and complete the final test of the curve detection model through the test set.

[0017] Example 1 A curve structure segmentation method based on dual-path attention, wherein in step 1, different curve images are classified according to their features, and the collected images are enhanced by translation and rotation; In step 2, the dataset is cleaned of blurred images and images with low resolution after enhancement to prevent overfitting of the model. The Labelme image annotation tool is used to mark the curve areas in the cleaned images to generate image labels in json format. The OpenCV library is used to generate label images with the same size as the original images, 8-bit RGB color gamut, and PNG format. A script is written to divide the annotated dataset into training set and test set in a ratio of 8:2 according to their corresponding defect categories. Finally, the dataset is sorted to generate the feather dataset required for training and testing.

[0018] Example 2 A curve structure segmentation method based on dual-path attention, wherein in step 1, different curve images are classified according to their features, and the collected images are enhanced by translation and rotation; In step 2, the data set is cleaned of blurred images and images with low resolution after enhancement to prevent overfitting of the model. The Labelme image annotation tool is used to mark the curve areas in the cleaned images to generate image labels in json format. The OpenCV library is used to generate label images with the same size as the original images, 8-bit RGB color gamut, and PNG format. The annotated data set is divided into a training set and a test set in a ratio of 8:2 according to the corresponding defect categories by writing a script, and finally sorted to generate the hairiness data set required for training and testing; In step 3, a dual-path encoding and decoding structure is used, one path is a global feature extraction path, which accurately locates the backbone area of ​​the curve image; the other path is a local feature extraction path, which is used to extract the detail features of the curve image; the features extracted by the two paths are effectively fused; In step 3, the global path adopts an encoder-decoder structure. The encoder consists of two triple attentions and a residual attention based on point convolution, and uses cross-dimensional spatial attention for feature extraction. The local path uses two residual attentions of point convolutions and introduces dynamic snake convolutions. Finally, two 7×7 convolutions are used for local fast feature extraction to ensure that the complete curve structure is extracted. In step 3, the differential feature enhancement module refines the intermediate features through layer-by-layer upsampling and differential calculation to enhance the detail and boundary segmentation capabilities.

[0019] Example 3 A curve structure segmentation method based on dual-path attention, wherein in step 1, different curve images are classified according to their features, and the collected images are enhanced by translation and rotation; In step 2, the data set is cleaned of blurred images and images with low resolution after enhancement to prevent overfitting of the model. The Labelme image annotation tool is used to mark the curve areas in the cleaned images to generate image labels in json format. The OpenCV library is used to generate label images with the same size as the original images, 8-bit RGB color gamut, and PNG format. The annotated data set is divided into a training set and a test set in a ratio of 8:2 according to the corresponding defect categories by writing a script, and finally sorted to generate the hairiness data set required for training and testing; In step 3, a dual-path encoding and decoding structure is used, one path is a global feature extraction path, which accurately locates the backbone area of ​​the curve image; the other path is a local feature extraction path, which is used to extract the detail features of the curve image; the features extracted by the two paths are effectively fused; In step 3, the global path adopts an encoder-decoder structure. The encoder consists of two triple attentions and a residual attention based on point convolution, and uses cross-dimensional spatial attention for feature extraction. The local path uses two residual attentions of point convolutions and introduces dynamic snake convolutions. Finally, two 7×7 convolutions are used for local fast feature extraction to ensure that the complete curve structure is extracted. In step 3, the differential feature enhancement module refines the intermediate features by upsampling and differential calculation layer by layer to enhance the detail and boundary segmentation capabilities; In step 4, the model learning parameters are adjusted, the data set is adjusted to the output format required by the model, the constructed neural network model is trained, the parameters and weights generated by the model during the training process are recorded and saved, and after the training is completed, the weight with the best training effect is selected for testing; In step 4, the training parameters of the curve structure segmentation model are: the number of single-batch training samples is 2, the optimizer uses the Adam optimizer, and the initial learning rate is , the minimum learning rate is , the input image size is , the number of training rounds is 200.

[0020] Example 4 The curve structure segmentation method based on dual-path attention is implemented in the following steps: Step 1: Use the Basler acA2500-60uc industrial camera to perform yarn image acquisition tasks, organize all image data, and generate a hairiness detection data set; Industrial cameras are selected for yarn image acquisition tasks. Considering the fine structure of the feather, high-precision size detection is required, and ordinary lenses have the problem of diffuse light spots, which may cause image blur. Therefore, this device uses Shenzhen Optics' 0.7X-5.6X variable magnification object space telecentric lens, which has the advantages of small distortion, no parallax and high resolution. In order to collect high-quality yarn feather images, before obtaining the feather images, it is also necessary to adjust the traction speed, light source brightness, image size, camera position, acquisition frame rate and exposure time to maximize the clarity of the collected yarn images. The data set contains 436 yarn feather images, of which 356 are used for model training and 80 are used for model testing. Step 2: In the above data set, the images with blurred images and low resolution after enhancement are cleaned to prevent overfitting of the model. The Labelme image annotation tool is used to mark the curve areas in the cleaned images to generate image labels in json format. The OpenCV library is used to generate label images with the same size as the original images, 8-bit RGB color gamut, and PNG format. The annotated data set is divided into a training set and a test set in a ratio of 8:2 according to the corresponding defect categories by writing a script. Finally, the hairiness data set required for training and testing is generated by sorting. Step 3. The global path adopts an encoding-decoding structure. The encoder part is composed of two triple attention modules and a residual attention module based on point convolution. Feature extraction is performed through cross-dimensional spatial attention. The triple attention module can capture the global information of the image in multiple dimensions, further enhancing the model's perception of complex curve structures. The local path further extracts local features through two residual attention modules of point convolution, and introduces dynamic snake convolution to optimize the capture of local details. Dynamic snake convolution simulates the motion trajectory of the serpentine curve and flexibly adapts to the changes in the curve structure, thereby avoiding possible breaks or blurring when processing the curve. Finally, local feature extraction is performed through two 7×7 large kernel convolutions to further refine the local information, ensure that the extracted features can cover the complete curve structure, and improve the integrity and accuracy of the segmentation results. This allows the local path to capture local details and segment the curve structure while maintaining efficient computing performance.

[0021] Example 5 The curve structure segmentation method based on dual-path attention, the overall structure of the model is as follows Figure 1 As shown in the figure, it mainly consists of three modules: global multi-branch attention mechanism, local multi-layer convolution module and differential feature enhancement module.

[0022] Global multi-branch attention mechanism module: In the curve structure segmentation task, the traditional convolutional neural network is difficult to effectively capture the global structural information of the curve due to the limitation of the local receptive field. This poses a challenge to the continuity and integrity of the curve, especially when the curve image is surrounded by a complex background or there is a large curvature change, it is difficult to accurately describe the global characteristics of the curve. To this end, the present invention introduces a multi-branch attention mechanism module, namely a triple attention module and a residual attention module based on point convolution. By extracting the curve image features in parallel through multi-branch attention and fusing the extracted features, the feature representation of different scales and directions is enhanced. Two 7*7 convolution kernels are used to increase the receptive field, thereby further extracting the global features of the image.

[0023] Example 6 Curve structure segmentation method based on dual-path attention, such as Figure 2 The triple attention module is shown in Figure 1, which is based on a structure of three parallel branches, where the first two branches are used to capture the interdependencies between the spatial dimension and the channel dimension, and the last branch is used to capture the dependencies between spatial features. Specifically, assuming the input tensor is , where C represents the number of channels and H×W represents the size of the feature map. In the first two parallel branches, the relationship between the channel and any spatial dimension is established by rotating the input tensor and applying the residual transformation. That is, the input tensor X is first rotated 90° counterclockwise along the H axis in the first parallel branch and the W axis in the second parallel branch, respectively, and recorded as the rotation tensors X1 and X2. The shapes of the two tensors are and , and then reduce the dimension of the input features to reduce the number of parameters, using Z-pool. Z-pool can connect the average and maximum pooling features in the dimension so that the layer retains rich actual tensor features, while reducing its depth to reduce the amount of calculation. The specific operation of Z-pool operation is shown in formula (1): (1) Where: Represents the input tensor; Maxpool represents maximum pooling; Avgpool represents average pooling; 0 represents the 0th dimension of the maximum pooling operation and the average pooling operation.

[0024] The reduced tensor is then passed through a 7×7 standard convolution and normalization layer, and finally the Sigmoid activation function generates the corresponding and , and add it to the rotation tensor, where and is the attention weight, which is expressed as shown in formula (2) and formula (3): (2) (3) Where: C and Z represent 7×7 convolution and Z-pool respectively; Represents a pooling operation; Represents the convolution operation; and Represent the ReLU function and the Sigmoid function respectively.

[0025] Similarly, the third branch reduces the dimension of the input tensor X through Z-pool, then passes through a 7×7 standard convolution and normalization layer, and finally generates a sigmoid activation function , It is expressed as shown in formula (4): (4) Where: C and Z represent Z-pool and 7×7 convolution respectively; Represents the convolution operation; Represents a pooling operation; and Represent ReLU and Sigmoid functions respectively.

[0026] Finally, the attention features generated by the three branches are effectively fused through an average aggregation operation, and the final output can be expressed as formula (5): (5) Where: X, X1, and X2 represent the original input tensor, the rotated input tensor along the width and channel dimensions, and the rotated input tensor along the height and channel dimensions, respectively; , and Represents the attention weight; T means that the tensor needs to be rotated to be consistent with the shape of the input tensor.

[0027] Example 7 Curved structure segmentation method based on dual-path attention, in which the local multi-layer convolution module: The difficulty of curved structure segmentation is that it has slender and fragile local structures, and the slender structures only occupy a small part of the entire image, and the pixel composition is limited. In addition, these structures are easily disturbed by complex backgrounds, so it is difficult for the model to accurately distinguish subtle changes in the target, resulting in fragmentation and breakage of the segmentation. To this end, this model introduces dynamic snake convolution, which is specifically proposed for the image segmentation task of tubular shapes, and shows good results in processing such tasks. It can flexibly fit the curved structure, so that the convolution kernel is more focused on the complex geometric features of the target.

[0028] like Figure 3As shown in the figure, the Squeeze-and-Excitation (SE) module is introduced in the residual attention branch of the point convolution, and the SE module is used to capture the inter-channel dependencies of the feature map. The SE module first "squeezes" each channel through global average pooling, and then "excites" through two fully connected layers and the ReLU activation function to generate the weight of each channel, and scales it to the [0,1] interval through the Sigmoid function. Finally, these weights are re-applied to the input feature map to enhance the feature expression of important channels. Then, a 1×1 point-by-point convolution is used in the linear layer to fuse the features of each channel and enhance the information interaction between channels. After the point-by-point convolution, batch normalization and GELU activation are also performed to maintain nonlinear features and stabilize training. Finally, the number of channels of the input feature map is aligned through the residual connection of the 1x1 convolution, and then the features after the convolution operation are added to the residual, thereby retaining the input information and improving the efficiency of gradient transfer.

[0029] In the specific implementation process, the model first divides the input feature map into 16 small blocks of the same size, and gradually extracts features from each small block. Each small block first extracts local features through a two-layer residual attention module, which not only prevents the problem of gradient disappearance during training, but also retains the original information of the image, ensuring that the two branches maintain a positive correlation when extracting curve image information. After passing through the two-layer residual attention module, two dynamic snake convolutions are introduced, and two 7×7 large-size convolution kernels are used to further enhance the ability to capture curve structures, ensuring that the model can accurately extract local features of curve images during segmentation and ensure the integrity of the structure.

[0030] Example 8 A curve structure segmentation method based on dual-path attention, in which the differential feature enhancement module: Figure 4The figure shows the differential feature enhancement module, which improves the performance of the model in the task of segmenting slender curved structures by gradually extracting and enhancing the detail differences in the feature map. First, the input image is downsampled step by step through multiple encoders to generate a multi-level feature pyramid and corresponding feature map, which provides a basis for the subsequent differential enhancement operation. On this basis, an upsampling operation is performed to generate an upsampled feature map, and then a feature difference map is obtained by point-by-point subtraction from the original feature map. These difference maps reveal the changes in features between different scales, especially capturing the changes in detail areas. Next, the differential feature map is upsampled again to generate a higher-level differential feature map, and a secondary differential feature map is generated by further subtraction. This recursive differential processing method helps the model capture the changes in local features more finely. After obtaining these differential features, they are added back to the original feature map step by step to enhance its detail expression. This fusion of differential residuals ensures the enhancement of details while retaining global structural information. Finally, the enhanced feature map is upsampled and fused layer by layer through the decoder, and the final segmentation result is obtained through the output layer. Through this differential feature enhancement module, the model can more effectively capture the slender curve structure in complex backgrounds, prevent the segmentation results from being broken, and improve the accuracy and completeness of curve segmentation.

[0031] The curve structure segmentation method based on dual-path attention in the present invention can more accurately segment curve structures under complex backgrounds, especially in low-contrast or noisy environments, and has higher segmentation accuracy than other deep learning methods; the differential feature enhancement module effectively avoids the common breakage problem in other methods, ensuring the detailed features of curve segmentation; the combination of local multi-layer convolution module and dynamic snake convolution significantly improves the ability to capture local details, especially when dealing with small size or tiny defects, and performs better than other deep learning methods; the optimized attention mechanism and convolution module improve the efficiency of feature extraction, avoid unnecessary computational redundancy, and make the segmentation process more efficient, which is particularly suitable for use in application scenarios that require real-time processing.

Claims

1. A curve structure segmentation method based on dual-path attention, characterized in that: Follow the steps below to implement it: Step 1: Use an industrial camera to take a yarn hairiness image, download a public curve dataset, and obtain a curve dataset by labeling, classification, and data enhancement; Step 2: Label the cleaned images and divide them into training sets and test sets according to a fixed ratio; Step 3: Apply the codec network structure to extract global features and local features through a dual-path method, and use a differential feature enhancement module to refine the features; Step 4: Load the divided curve data set into the model, perform model training, and complete the final test of the curve detection model through the test set.

2. The curve structure segmentation method based on dual-path attention according to claim 1 is characterized in that: In the step 1, the images are classified according to the features shown by different curve images, and the collected images are enhanced by translation and rotation.

3. The curve structure segmentation method based on dual-path attention according to claim 1 is characterized in that: In the step 2, the data set is cleaned of blurred images and images with low resolution after enhancement, the Labelme image annotation tool is used to mark the curve areas in the cleaned images to generate image labels in json format, and the OpenCV library is used to generate label images with the same size as the original images, 8-bit RGB color gamut, and PNG format. By writing a script, the annotated data set is divided into a training set and a test set according to their corresponding defect categories in a ratio of 8:2, and finally sorted to generate the feather data set required for training and testing.

4. The curve structure segmentation method based on dual-path attention according to claim 1, characterized in that: In step 3, a dual-path encoding and decoding structure is adopted, one path is a global feature extraction path, which accurately locates the backbone area of ​​the curve image; the other path is a local feature extraction path, which is used to extract the detail features of the curve image; the features extracted by the two paths are effectively fused.

5. The curve structure segmentation method based on dual-path attention according to claim 4 is characterized in that: In step 3, the global path adopts an encoding and decoding structure, the encoder is composed of two triple attentions and a residual attention based on point convolution, and cross-dimensional spatial attention is used for feature extraction; the local path passes through the residual attention of two point convolutions, and introduces dynamic snake convolution, and finally performs local fast feature extraction through two 7×7 convolutions to ensure that the complete curve structure is extracted.

6. The curve structure segmentation method based on dual-path attention according to claim 3 is characterized in that: In the step 3, the differential feature enhancement module refines the intermediate features by upsampling and differential calculation layer by layer, thereby enhancing the detail and boundary segmentation capabilities.

7. The curve structure segmentation method based on dual-path attention according to claim 1 is characterized in that: In step 4, the model learning parameters are adjusted, the data set is adjusted to the output format required by the model, the constructed neural network model is trained, the parameters and weights generated by the model during the training process are recorded and saved, and after the training is completed, the weights with the best training effect are selected for testing.

8. The curve structure segmentation method based on dual-path attention according to claim 7, characterized in that: In step 4, the training parameters of the curve structure segmentation model are: the number of single batch training samples is 2, the optimizer uses the Adam optimizer, and the initial learning rate is , the minimum learning rate is , the input image size is , the number of training rounds is 200.

Images