Transfomer-based edge detection method

Abstract

The invention belongs to the field of computer vision, and particularly relates to an edge detection technology based on Transfomer. Firstly, according to the method, a scale enhancement technology is adopted, and an input image is divided into a plurality of scales with different sizes, so that detail information in the image can be better processed; secondly, according to the method, a feature enhancement technology is adopted, and the features extracted by the network are enhanced, so that feature information is utilized more fully, and the accuracy and robustness of edge detection are improved. And finally, a feature fusion technology is introduced in the method, and features of different scales and different stages are fused, so that the feature expression capability is further enhanced, noise generation is effectively inhibited, and a clear edge is generated. Through comprehensive application of the processing technology, the edge detection method can effectively avoid the problems of loss of partial details of the edge, rough lines, disordered content, noise and illumination interference and the like when a complex background is processed, the accuracy and reliability of edge detection are improved, and meanwhile, the efficiency of edge detection is improved.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, in particular to an edge detection technology based on Transfomer. Background Art

[0002] Edge detection is a basic task in the field of computer vision and image processing. Its main goal is to detect the boundaries of objects in an image. The development of edge detection technology is closely related to the development of digital image processing technology. The development of digital image processing technology can be traced back to the 1960s, when computer technology was just emerging. At that time, people began to use computers to process digital images, mainly to solve problems in the fields of medical imaging and satellite images. Edge detection technology is an important part of digital image processing, which can help people better understand and analyze images. The earliest edge detection algorithms were based on image intensity changes, such as gradient-based methods and Laplace operator-based methods. These methods have many problems, such as sensitivity to noise and inaccurate detection results. Later, people began to use more complex algorithms to solve these problems, such as Canny edge detection algorithm, Sobel operator, Prewitt operator, etc. With the continuous development of computer technology, edge detection technology has also been continuously improved and optimized. For example, people began to use machine learning algorithms for edge detection, which can better handle complex scenes and noise. But there are still some challenges and limitations. Some of these challenges and limitations include:

[0003] 1. Sensitivity to noise and image distortion: Edge detection techniques are often very sensitive to noise and image distortion, which may lead to inaccurate detection results.

[0004] 2. Sensitivity to lighting changes and shadows: Lighting changes and shadows can cause edges in an image to change, so edge detection techniques need to be able to handle these changes.

[0005] 3. Limitation on edge width: Some edge detection algorithms can only detect edges of a certain width, which may cause some edges to be ignored or detected incorrectly.

[0006] 4. Demand for computing resources: Some edge detection algorithms require a lot of computing resources to perform edge detection, which may make them infeasible in practical applications. Summary of the invention

[0007] In order to solve the above shortcomings and difficulties, the present invention provides an edge detection method based on Transformer, which aims to solve the problems that the existing edge detection method encounters when processing complex backgrounds, such as loss of details, rough lines, confusing content, noise and light interference.

[0008] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0009] A Transformer-based edge detection method comprises the following steps:

[0010] 1) Extract basic image features using Swin Transformer;

[0011] 2) Scale enhancement and upsampling of the extracted features;

[0012] 3) Use the spatial attention mechanism to soft-weight the sampled features;

[0013] 4) The soft-weighted feature maps of each channel are weighted using the channel attention mechanism, and the weighted feature maps of multiple channels are fused to obtain the final edge detection result.

[0014] In step 1), the attention mechanism in Swin Transformer is:

[0015]

[0016]

[0017]

[0018]

[0019] Among them, z l-1 represents the initial input vector, represents the output after W-MSA, W-MSA represents the window-based multi-head attention mechanism, LN represents layer normalization, MLP represents multi-layer perceptron, express After the output of SW-MSA, z l +1 It is the basic feature finally output by SwinTransformer, and SW-MSA represents the multi-head self-attention mechanism based on movable window.

[0020] The step 2) is specifically as follows: the four basic features output by the four stages of SwinTransformer are scale-enhanced twice respectively to obtain eight enhanced feature vectors, which are F side1 、F side2 、F side3 、F side4 、F side5 、F side6 、F side7 、F side8, where the feature scale enhancement algorithm is:

[0021] F side =F1+F2+F3+F4

[0022] Among them, F side It represents the feature vector of the basic feature after scale enhancement, and F1, F2, F3, and F4 represent the feature vectors of the input vector after being processed by four dilated convolutions.

[0023] In step 3), the spatial attention mechanism is specifically:

[0024]

[0025] F s ' ide =W s (F side )·F side

[0026] Among them, W s represents the weights obtained by spatial attention calculation in the four stages, σ represents the convolution operation, and f 7×7 represents the pooling kernel size, represents the feature vector after average pooling, represents the feature vector after maximum pooling, F s ' ide Represents the soft-weighted feature map.

[0027] In step 4), the channel attention mechanism is specifically as follows:

[0028]

[0029] F fuse =Cat(F side1 ,F side2 ,F side3 ,F side4 ,F side5 ,F side6 ,F side7 ,F side8 )

[0030] F' fuse =W f ·F fuse

[0031] Among them, W f is the weight of each channel after fusion, MLP represents multi-layer perceptron, W0 and W1 represent the weight coefficients of the features passing through the two layers of the network, Cat represents the concatenation of feature vectors according to the dimension, and F fuse represents the initial fusion feature, F'fuse Represents the features of the fused features after being processed by the channel attention mechanism.

[0032] A Transfomer-based edge detection system, comprising:

[0033] Feature extraction module, used to extract basic features of images using Swin Transformer;

[0034] The scale enhancement module is used to scale and upsample the extracted features;

[0035] The feature weighting module is used to soft-weight the sampled features using the spatial attention mechanism;

[0036] The feature fusion module is used to use the channel attention mechanism to weight the soft-weighted feature map of each channel, and fuse the weighted feature maps of multiple channels to obtain the final edge detection result.

[0037] A Transfomer-based edge detection system comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement the Transfomer-based edge detection method when executing the computer program.

[0038] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the edge detection method based on Transformer is implemented.

[0039] The present invention has the following beneficial effects and advantages:

[0040] 1. It can detect the edges of objects of different sizes more accurately. The method of the present invention proposes a scale enhancement technology, the introduction of which enables different layers to pay attention to features of different scales, thereby better utilizing information of different scales. In this way, the adaptability of the model to changes in object scale can be improved, and better performance can be achieved in edge detection tasks.

[0041] 2. Solve the influence of illumination and noise on edge detection effect. The method of the present invention proposes feature enhancement technology to weight and integrate the scale-enhanced features, reduce redundant information, strengthen important information, and improve the performance and robustness of the model. Among them, the spatial attention mechanism helps the model to adaptively learn the importance of different positions, and the weighted feature map can better capture the spatial structure and semantic information of the target, solving the influence of illumination and noise on edge detection effect.

[0042] 3. Save computing resources and improve the efficiency of edge detection. In order to solve this problem, the method of the present invention proposes a channel attention mechanism to weight the fused feature vector in the channel dimension, helping the model to adaptively learn the importance of each channel, thereby greatly reducing redundant information and strengthening important information, improving the effect of feature fusion, and reducing the dependence of the method on computing resources during detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a module flow chart of the method of the present invention;

[0044] Figure 2 It is the overall model structure diagram;

[0045] Figure 3 is the scale enhancement map;

[0046] Figure 4 is the feature enhancement map;

[0047] Figure 5 It is the feature fusion graph. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0049] A Transformer-based edge detection method comprises the following steps:

[0050] 1. Use Swin Transformer to extract basic features of the image;

[0051] 2. Scale the extracted features and upsample them;

[0052] 3. Use the spatial attention mechanism to soft-weight features and enhance the weight of important information;

[0053] 4. Use the channel attention mechanism to weight the features of each channel, fuse the weighted feature maps, and obtain the final edge detection result;

[0054] The attention mechanism algorithm in the Swin Transformer is

[0055]

[0056]

[0057]

[0058]

[0059] Among them, z l-1represents the initial input vector, The output after W-MSA, W-MSA represents the window-based multi-head attention mechanism, LN represents layer normalization, and MLP represents multi-layer perceptron. express After the output of SW-MSA, z l+1 It is the basic feature finally output by SwinTransformer. SW-MSA stands for the multi-head self-attention mechanism based on movable windows. Since SwinTransformer has four stages, each stage will output 1 basic feature.

[0060] The algorithm for feature scale enhancement is

[0061] F side =F1+F2+F3+F4

[0062] The above formula is the calculation formula for scale enhancement of the feature maps of the four stages of output, where F side It represents the feature vector of the basic feature after scale enhancement. Since the features output from the four stages are scale enhanced twice, eight enhanced feature vectors can be obtained, namely, F side1 、F side2 、F side3 、F side4 、F side5 、F side6 、F side7 、F side8 , F1, F2, F3, and F4 represent the feature vectors of the input vector after being processed by four dilated convolutions.

[0063] The algorithm of the spatial attention mechanism is

[0064]

[0065] F' side =W s (F side )·F side

[0066] The above formulas are the calculation formulas for the attention weight at each stage and the weighted feature map calculation formula, where W s represents the weights obtained by spatial attention calculation in the four stages, F side It represents the feature map of the side output of each stage. There are 8 in total. Avg represents the pooling operation, Max represents the maximum pooling, and f 7×7 Represents the pooling kernel size, and σ indicates that the results of two pooling operations will be aggregated through a convolution operation.

[0067] The algorithm of the channel attention mechanism is

[0068]

[0069] F fuse =Cat(F side1 ,F side2 ,F side3 ,F side4 ,F side5 ,F side6 ,F side7 ,F side8 )

[0070] F' fuse =W f ·F fuse

[0071] Where W f is the weight of each channel after fusion, MLP stands for multi-layer perceptron, a two-layer neural network is used here, so W0 and W1 are the weight coefficients of the features passing through the two layers of the network, Cat means concatenating the feature vectors according to the dimension, F ’fuse Represents the features of the fused features after being processed by the channel attention mechanism.

[0072] Example

[0073] like Figure 1 and Figure 2 As shown, the method of the present invention comprises the following steps:

[0074] 1. Use Swin Transformer to extract the basic features of the image. When processing the image, the input image needs to be divided into small blocks of the same size (or called groups). These small blocks are flattened into one-dimensional vectors and are regarded as the input token sequence of the Transformer. This sequence is input into the Transformer for feature extraction. At the same time, in Swin Transformer, each small block has a position code to represent its position information in the original image. These position codes are input into the Transformer together with the feature vector to help the network learn global information.

[0075] 2. After extracting the basic features using Swin Transformer, the output features of the four stages need to be enhanced, such as Figure 3 The features output from each stage are input into the dilated convolution with different magnifications. The magnifications are set to [1, 2, 4, 8], the convolution kernel size is 3×3, and the number of channels is 32. The feature vectors after these four processes are added together to obtain the enhanced features.

[0076] 3. After scale enhancement, the feature maps of the four stages are changed from the original 4 to 8 feature maps of the same shape. These feature maps are spliced ​​from shallow to deep and from deep to shallow, such as Figure 4 In order to specifically find the locations of important information, a spatial attention mechanism is used to weight these locations.

[0077] 4. After feature enhancement, each side output part will generate a feature map with a size of [1, H, W]. These 8 feature maps are concatenated to obtain a feature map with a size of [8, H, W]. In order to assist the model in finding which features in which channels are more useful, a channel attention mechanism is introduced to weight the concatenated feature maps. The specific operation is to perform maximum pooling and average pooling on the channel dimension respectively, and then pass the pooling results through the MLP layer respectively, and add the results to obtain the weight of the channel dimension. Multiply the weight by the input feature, and add this feature vector according to the channel dimension to obtain the final fusion result. At this time, the size of the feature map is still [1, H, W], such as Figure 5 .

[0078] The edge detection method of the present invention can not only improve the performance and efficiency of the model, but also provide corresponding solutions for the influence of light and noise, thereby further improving the edge detection effect.

[0079] In terms of illumination and noise, the method provided by the present invention adopts a spatial attention mechanism and a channel attention mechanism, which can perform weighted processing on different parts of the image, thereby improving the detection effect of important information such as edges. In addition, the present invention also performs scale enhancement and feature enhancement processing on the image, thereby further optimizing the feature representation and expression ability of the image.

[0080] Through the comprehensive application of the above processing technologies, the edge detection method of the present invention can effectively deal with the interference of light and noise on image processing, improve the edge detection effect, and thus have better performance and application prospects in computer vision tasks such as image classification and target detection.

Claims

1. A Transformer-based edge detection method, characterized in that: The following steps are involved: 1) Extract basic image features using Swin Transformer; 2) Scale enhancement and upsampling of the extracted features; 3) Use the spatial attention mechanism to soft-weight the sampled features; 4) The soft-weighted feature maps of each channel are weighted using the channel attention mechanism, and the weighted feature maps of multiple channels are fused to obtain the final edge detection result.

2. The edge detection method based on Transformer according to claim 1, characterized in that: In step 1), the attention mechanism in Swin Transformer is: Among them, z l-1 represents the initial input vector, represents the output after W-MSA, W-MSA represents the window-based multi-head attention mechanism, LN represents layer normalization, MLP represents multi-layer perceptron, express After the output of SW-MSA, z l+1 It is the basic feature finally output by SwinTransformer, and SW-MSA represents the multi-head self-attention mechanism based on movable window.

3. The edge detection method based on Transformer according to claim 1, characterized in that: The step 2) is specifically as follows: the four basic features output by the four stages of SwinTransformer are scale-enhanced twice respectively to obtain eight enhanced feature vectors, which are F side1 、F side2 、F side3 、F side4 、F side5 、F side6 、F side7 、F side8 , Among them, the feature scale enhancement algorithm is: <h2 style=";text-align:left;direction:ltr">F<h2 style=";text-align:left;direction:ltr"> side <h2 style=";text-align:left;direction:ltr"> (F1+F2+F3+F4) Among them, F side It represents the feature vector of the basic feature after scale enhancement, and F1, F2, F3, and F4 represent the feature vectors of the input vector after being processed by four dilated convolutions.

4. The edge detection method based on Transformer according to claim 1, characterized in that: In step 3), the spatial attention mechanism is specifically: F s ' ide =W s (F side )·F side Among them, W s represents the weights obtained by spatial attention calculation in the four stages, σ represents the convolution operation, and f 7×7 represents the pooling kernel size, represents the feature vector after average pooling, represents the feature vector after maximum pooling, F′ side Represents the soft-weighted feature map.

5. The edge detection method based on Transformer according to claim 1, characterized in that: In step 4), the channel attention mechanism is specifically as follows: F fuse =Cat(F side1 ,F side2 ,F side3 ,F side4 ,F side5 ,F side6 ,F side7 ,F side8 ) F′ fuse =W f ·F fuse Among them, W f is the weight of each channel after fusion, MLP represents multi-layer perceptron, W0 and W1 represent the weight coefficients of the features passing through the two layers of the network, Cat represents the concatenation of feature vectors according to the dimension, and F fuse represents the initial fusion feature, F′ fuse Represents the features of the fused features after being processed by the channel attention mechanism.

6. A Transfomer-based edge detection system, characterized in that: include: Feature extraction module, used to extract basic features of images using Swin Transformer; The scale enhancement module is used to scale and upsample the extracted features; The feature weighting module is used to soft-weight the sampled features using the spatial attention mechanism; The feature fusion module is used to use the channel attention mechanism to weight the soft-weighted feature map of each channel, and fuse the weighted feature maps of multiple channels to obtain the final edge detection result.

7. A Transfomer-based edge detection system, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement a Transfomer-based edge detection method as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the Transformer-based edge detection method according to any one of claims 1 to 5 is implemented.

Images