An edge detection method based on biological visual dual pathways

Abstract

This invention aims to provide an edge detection method based on a dual-pathway biological vision system, comprising the following steps: A) Constructing a dual-pathway edge detection network, specifically including a first-pathway network (FPN), a second-pathway network (SPN), and a decoding network (DN); B) Inputting the original image into the first-pathway network (FPN) and the second-pathway network (SPN) respectively; finally, the three-pathway fusion module (PFM) processes the image to obtain three results, which are then input into the DN; C) After decoding by the DN, the output result is the final output contour. This invention can effectively extract edge feature information, thereby obtaining a more accurate edge map.

Description

Technical Field

[0001] This invention relates to the field of computer image processing, and more specifically to an edge detection method based on a dual-pathway biological vision. Background Technology

[0002] Edge detection aims to extract edge information of meaningful objects in an image. It is commonly used in the preprocessing stage of various intermediate and advanced computer vision tasks and is a classic problem in computer vision research. Due to the rapid development of deep learning, encoder-decoder edge detection methods based on convolutional neural networks (CNNs) have developed rapidly. Most of these methods use VGG16 or ResNet as the encoder and achieve good performance through well-designed decoders. Experiments have shown that these models achieve significant results on the Berkeley Segmentation Dataset (BSDS500).

[0003] While CNN-based encoder-decoder edge detection methods have achieved significant results, performance improvements have slowed in recent years. Furthermore, while constructing edge detection methods solely through encoder-decoder architectures can enhance performance, the lack of guidance from biological visual mechanisms increases interpretability. Summary of the Invention

[0004] The present invention aims to provide an edge detection method based on a dual-pathway biological vision, which can effectively extract edge feature information and thus obtain a more accurate edge map.

[0005] The technical solution of the present invention is as follows:

[0006] The edge detection method based on the dual-pathway biological vision includes the following steps:

[0007] A. Construct a dual-path edge detection network, as detailed below:

[0008] This includes the first path network (FPN), the second path network (SPN), and the decoding network (DN).

[0009] The first path network (FPN) includes three first path modules (FPBs) connected in sequence, and each of the three first path modules (FPBs) is composed of two or more sequentially connected Swing Transformer Layers.

[0010] The second path network SPN uses three sequentially connected second path modules SPB. Each of the three SPB modules consists of a basic 3x3 ordinary convolution and two 3x3 depth-separable convolutional layers with residual structures connected in sequence.

[0011] The first-path network FPN and the second-path network SPN interact through three second-path attention modules SPAM and three-path fusion modules PFM.

[0012] The decoding network DN consists of three feature fusion modules (FFM).

[0013] B. The original image is input into the first path network FPN and the second path network SPN respectively;

[0014] In the first-path network FPN:

[0015] The original image is first processed by Patch Embedding to cut the image into small pieces. Then, each piece is encoded and mapped into a high-dimensional vector feature. The high-dimensional vector feature and the output of the first second-path attention module SPAM are multiplied element-wise and then input into the first first-path module FPB.

[0016] The processing result of the first first-path module FPB is divided into two outputs. The first output and the processing result of the second second-path attention module SPAM are multiplied by element-wise and then input into the second first-path module FPB. The second output is input into the first-path fusion module PFM.

[0017] The processing result of the second first-path module FPB is divided into two outputs. The first output and the output of the third second-path attention module SPAM are multiplied by element-wise and then input into the third first-path module FPB. The second output is input into the second-path fusion module PFM.

[0018] The processing results of the third first-path module FPB are input into the third-path fusion module PFM;

[0019] In the second path network SPN:

[0020] The original image is input into the first second-path module SPB, and the processing results of the first second-path module SPB are input into the first second-path attention module SPAM, the first path fusion module PFM, and the second second-path module SPB, respectively.

[0021] The processing results of the second second-path module SPB are respectively input into the second second-path attention module SPAM, the second path fusion module PFM, and the third second-path module SPB.

[0022] The processing results of the third second-path module SPB are respectively input into the third second-path attention module SPAM and the third path fusion module PFM;

[0023] The processing results of the first path fusion module PFM, the second path fusion module PFM, and the third path fusion module PFM are respectively input into the decoding network DN;

[0024] After C and DN are decoded, the output result is the final output profile.

[0025] The first first path module FPB consists of 2 layers of Swing Transformer Lyer, the second first path module FPB consists of 2 layers of Swing Transformer Lyer, and the third first path module FPB consists of 18 layers of Swing Transformer Lyer.

[0026] The 3x3 depth-separable convolutional layer with residual structure includes a 3x3 depth convolutional layer, a grouping normalization and GeLU activation function module, and a 1x1 convolutional layer connected in sequence. In the 3x3 depth-separable convolutional layer with residual structure, the input result is processed by the 3x3 depth convolutional layer, the grouping normalization and GeLU activation function, and the 1x1 convolutional layer in sequence, and then added to itself to obtain the output result.

[0027] The first, second, and third SPAM second-path attention modules have the same structure, each including a 4×4 convolution with a stride of 4 and a 4×4 kernel, and a sigmoid activation function. In these SPAM second-path attention modules, the input result is processed by a 4×4 convolution with a 4 kernel, activated by the sigmoid activation function, and finally the output result is obtained after a repeat operation.

[0028] The first, second, and third path fusion modules PFM have the same structure, each including F 1,i Characteristic pathways and F 2，i Characteristic pathways;

[0029] In F 1,i In the feature path, the previous input result is processed by two sequentially connected USM Layer modules, and then by the C Layer module to obtain F. 1,i Pathway outcome;

[0030] In F 2，i In the feature path, the next input result is processed by the C Layer module to obtain F. 2，i Pathway outcome;

[0031] F 1,i Pathway outcomes and F 2，iAfter the path results are merged together by channel, they are processed by the C Layer module to obtain the output results.

[0032] In the USM Layer module, the input results are sequentially processed by 1×1 convolution, grouping and normalization, and then by the Gelu function to obtain the output results.

[0033] In the C Layer module, the input results are processed sequentially through 3×3 convolution, pixel shuffle function, and group normalization to obtain the output results.

[0034] In the aforementioned decoding network DN

[0035] The processing results of the first path fusion module PFM and the second path fusion module PFM are input into a feature fusion module FFM for fusion processing to obtain the first feature fusion result;

[0036] The processing results of the second and third path fusion modules PFM are input into a feature fusion module FFM for fusion processing to obtain the second feature fusion result.

[0037] The first feature fusion result and the second feature fusion result are input into the third feature fusion module FFM. After being fused by the third feature fusion module FFM, the result is obtained after 1×1 convolution.

[0038] The feature fusion module FFM includes D respectively. i,j Characteristic pathways and D i，j+1 Characteristic pathways;

[0039] In D i,j In the feature path, the previous input result is processed by the C Layer module to obtain D. 1,i Pathway outcome;

[0040] In D i，j+1 In the feature path, the next input result is processed sequentially through the C Layer module and the USM Layer module to obtain D. i，j+1 Pathway outcome;

[0041] D i,j Pathway outcomes and D i，j+1 After the path results are merged together by channel, they are processed by the C Layer module to obtain the output results.

[0042] In the US Layer module, the input results are sequentially processed by 1×1 convolution, grouping and normalization, and then by the Gelu function to obtain the output results.

[0043] In the C Layer module, the input results are processed sequentially through 3×3 convolution, pixel shuffle function, and group normalization to obtain the output results. Attached Figure Description

[0044] Figure 1 This is an overall structural diagram of the dual-path edge detection network in Embodiment 1 of the present invention;

[0045] Figure 2 This is a structural diagram of the FPB module in Embodiment 1 of the present invention;

[0046] Figure 3 This is a structural diagram of a 3x3 depth-separable convolutional layer with residual structure according to Embodiment 1 of the present invention;

[0047] Figure 4 This is a structural diagram of the attention module SPAM in Embodiment 1 of the present invention;

[0048] Figure 5 This is a structural diagram of the path fusion module PFM in Embodiment 1 of the present invention;

[0049] Figure 6 This is a structural diagram of the feature fusion module FFM in Embodiment 1 of the present invention. Detailed Implementation

[0050] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Example 1

[0051] This embodiment provides an edge detection method based on a dual-pathway biological vision system, comprising the following steps:

[0052] The edge detection method based on the dual-pathway biological vision includes the following steps:

[0053] A. Construct a dual-path edge detection network, as detailed below:

[0054] This includes the first path network (FPN), the second path network (SPN), and the decoding network (DN).

[0055] The first path network (FPN) includes three first path modules (FPBs) connected in sequence, and each of the three first path modules (FPBs) is composed of two or more sequentially connected Swing Transformer Layers.

[0056] The second path network SPN uses three sequentially connected second path modules SPB. Each of the three SPB modules consists of a basic 3x3 ordinary convolution and two 3x3 depth-separable convolutional layers with residual structures connected in sequence.

[0057] The first-path network FPN and the second-path network SPN interact through three second-path attention modules SPAM and three-path fusion modules PFM.

[0058] The decoding network DN consists of three feature fusion modules (FFM).

[0059] B. The original image is input into the first path network FPN and the second path network SPN respectively;

[0060] In the first-path network FPN:

[0061] The original image is first processed by Patch Embedding to cut the image into small pieces. Then, each piece is encoded and mapped into a high-dimensional vector feature. The high-dimensional vector feature and the output of the first second-path attention module SPAM are multiplied element-wise and then input into the first first-path module FPB.

[0062] The processing result of the first first-path module FPB is divided into two outputs. The first output and the processing result of the second second-path attention module SPAM are multiplied by element-wise and then input into the second first-path module FPB. The second output is input into the first-path fusion module PFM.

[0063] The processing result of the second first-path module FPB is divided into two outputs. The first output and the output of the third second-path attention module SPAM are multiplied by element-wise and then input into the third first-path module FPB. The second output is input into the second-path fusion module PFM.

[0064] The processing results of the third first-path module FPB are input into the third-path fusion module PFM;

[0065] In the second path network SPN:

[0066] The original image is input into the first second-path module SPB, and the processing results of the first second-path module SPB are input into the first second-path attention module SPAM, the first path fusion module PFM, and the second second-path module SPB, respectively.

[0067] The processing results of the second second-path module SPB are respectively input into the second second-path attention module SPAM, the second path fusion module PFM, and the third second-path module SPB.

[0068] The processing results of the third second-path module SPB are respectively input into the third second-path attention module SPAM and the third path fusion module PFM;

[0069] The processing results of the first path fusion module PFM, the second path fusion module PFM, and the third path fusion module PFM are respectively input into the decoding network DN;

[0070] After C and DN are decoded, the output result is the final output profile.

[0071] The first first path module FPB consists of 2 layers of Swing Transformer Lyer, the second first path module FPB consists of 2 layers of Swing Transformer Lyer, and the third first path module FPB consists of 18 layers of Swing Transformer Lyer.

[0072] The 3x3 depth-separable convolutional layer with residual structure includes a 3x3 depth convolutional layer, a grouping normalization and GeLU activation function module, and a 1x1 convolutional layer connected in sequence. In the 3x3 depth-separable convolutional layer with residual structure, the input result is processed by the 3x3 depth convolutional layer, the grouping normalization and GeLU activation function, and the 1x1 convolutional layer in sequence, and then added to itself to obtain the output result.

[0073] The first, second, and third SPAM second-path attention modules have the same structure, each including a 4×4 convolution with a stride of 4 and a 4×4 kernel, and a sigmoid activation function. In these SPAM second-path attention modules, the input result is processed by a 4×4 convolution with a 4 kernel, activated by the sigmoid activation function, and finally the output result is obtained after a repeat operation.

[0074] The first, second, and third path fusion modules PFM have the same structure, each including F 1,i Characteristic pathways and F 2，i Characteristic pathways;

[0075] In F 1,i In the feature path, the previous input result is processed by two sequentially connected USM Layer modules, and then by the C Layer module to obtain F. 1,i Pathway outcome;

[0076] In F 2，i In the feature path, the next input result is processed by the C Layer module to obtain F. 2，i Pathway outcome;

[0077] F 1,i Pathway outcomes and F 2，iAfter the path results are merged together by channel, they are processed by the C Layer module to obtain the output results.

[0078] In the USM Layer module, the input results are sequentially processed by 1×1 convolution, grouping and normalization, and then by the Gelu function to obtain the output results.

[0079] In the C Layer module, the input results are processed sequentially through 3×3 convolution, pixel shuffle function, and group normalization to obtain the output results.

[0080] In the aforementioned decoding network DN

[0081] The processing results of the first path fusion module PFM and the second path fusion module PFM are input into a feature fusion module FFM for fusion processing to obtain the first feature fusion result;

[0082] The processing results of the second and third path fusion modules PFM are input into a feature fusion module FFM for fusion processing to obtain the second feature fusion result.

[0083] The first feature fusion result and the second feature fusion result are input into the third feature fusion module FFM. After being fused by the third feature fusion module FFM, the result is obtained after 1×1 convolution.

[0084] The feature fusion module FFM includes D respectively. i,j Characteristic pathways and D i，j+1 Characteristic pathways;

[0085] In D i,j In the feature path, the previous input result is processed by the C Layer module to obtain D. 1,i Pathway outcome;

[0086] In D i，j+1 In the feature path, the next input result is processed sequentially through the C Layer module and the USM Layer module to obtain D. i，j+1 Pathway outcome;

[0087] D i,j Pathway outcomes and D i，j+1 After the path results are merged together by channel, they are processed by the C Layer module to obtain the output results.

[0088] In the US Layer module, the input results are sequentially processed by 1×1 convolution, grouping and normalization, and then by the Gelu function to obtain the output results.

[0089] In the C Layer module, the input results are processed sequentially through 3×3 convolution, pixel shuffle function, and group normalization to obtain the output results. Example 2

[0090] For the quantitative performance evaluation of the final edge graph, we adopted the same performance measurement standard as in Reference 1, and the specific evaluation is shown in Formula (7).

[0091] (1)

[0092] in, P Indicates accuracy. R This indicates the recall rate. F The larger the value, the better the performance.

[0093] Document 1: S. Xie and Z. Tu, "Holistically-nested edge detection," in International Conference on Computer Vision, 2015, pp. 1395-1403.

[0094] The parameters used in Reference 1 are the same as those in the original text, and they are guaranteed to be the optimal parameters for the model.

[0095] Table 1 summarizes the experimental data of Reference 1 and Example 1 on the Berkeley Segmentation Dataset (BSDS500). The experimental results show that the detection method in Example 1 outperforms the detection method in Reference 1. Furthermore, the second-path network SPN in our proposed method can improve the performance of the method in Reference 1.

[0096]

Claims

1. An edge detection method based on a dual-pathway biological vision system, characterized in that... Includes the following steps: A. Construct a dual-path edge detection network, as detailed below: This includes the first path network (FPN), the second path network (SPN), and the decoding network (DN). The first path network (FPN) includes three first path modules (FPBs) connected in sequence, and each of the three first path modules (FPBs) is composed of two or more sequentially connected Swing Transformer Layers. The second path network SPN uses three sequentially connected second path modules SPB. Each of the three second path modules SPB consists of a basic 3x3 ordinary convolution and two 3x3 depth-separable convolutional layers with residual structures connected in sequence. The first-path network FPN and the second-path network SPN interact through three second-path attention modules SPAM and three-path fusion modules PFM. The decoding network DN consists of three feature fusion modules (FFM). B. The original image is input into the first path network FPN and the second path network SPN respectively; In the first-path network FPN: The original image is first processed by Patch Embedding to cut the image into small pieces. Then, each piece is encoded and mapped into a high-dimensional vector feature. The high-dimensional vector feature and the output of the first second-path attention module SPAM are multiplied element-wise and then input into the first first-path module FPB. The processing result of the first first-path module FPB is divided into two outputs. The first output and the processing result of the second second-path attention module SPAM are multiplied by element-wise and then input into the second first-path module FPB. The second output is input into the first-path fusion module PFM. The processing result of the second first-path module FPB is divided into two outputs. The first output and the output of the third second-path attention module SPAM are multiplied by element-wise and then input into the third first-path module FPB. The second output is input into the second-path fusion module PFM. The processing results of the third first-path module FPB are input into the third-path fusion module PFM; In the second path network SPN: The original image is input into the first second-path module SPB, and the processing results of the first second-path module SPB are input into the first second-path attention module SPAM, the first path fusion module PFM, and the second second-path module SPB, respectively. The processing results of the second second-path module SPB are respectively input into the second second-path attention module SPAM, the second path fusion module PFM, and the third second-path module SPB. The processing results of the third second-path module SPB are respectively input into the third second-path attention module SPAM and the third path fusion module PFM; The processing results of the first path fusion module PFM, the second path fusion module PFM, and the third path fusion module PFM are respectively input into the decoding network DN; After C and DN are decoded, the output result is the final output profile.

2. The edge detection method based on dual-pathway biological vision as described in claim 1, characterized in that: The first first path module FPB consists of 2 layers of Swing Transformer Lyer, the second first path module FPB consists of 2 layers of Swing Transformer Lyer, and the third first path module FPB consists of 18 layers of Swing Transformer Lyer.

3. The edge detection method based on dual-pathway biological vision as described in claim 1, characterized in that: The 3x3 depth-separable convolutional layer with residual structure includes a 3x3 depth convolutional layer, a grouping normalization and GeLU activation function module, and a 1x1 convolutional layer connected in sequence. In the 3x3 depth-separable convolutional layer with residual structure, the input result is processed by the 3x3 depth convolutional layer, the grouping normalization and GeLU activation function, and the 1x1 convolutional layer in sequence, and then added to itself to obtain the output result.

4. The edge detection method based on dual-pathway biological vision as described in claim 1, characterized in that: The first, second, and third SPAM second-path attention modules have the same structure, each including a convolution with a stride of 4 and a kernel of 4×4, and a sigmoid activation function. In these SPAM second-path attention modules, the input result is processed by a convolution with a stride of 4 and a kernel of 4×4, activated by the sigmoid activation function, and finally the output result is obtained after a repeat operation.

5. The edge detection method based on dual-pathway biological vision as described in claim 1, characterized in that: The first, second, and third path fusion modules PFM have the same structure, each including F 1,i Characteristic pathways and F 2，i Characteristic pathways; In F 1,i In the feature path, the previous input result is processed by two sequentially connected USM Layer modules, and then by the C Layer module to obtain F. 1,i Pathway outcome; In F 2，i In the feature path, the next input result is processed by the C Layer module to obtain F. 2，i Pathway outcome; F 1,i Pathway outcomes and F 2，i After the path results are merged together by channel, they are processed by the C Layer module to obtain the output results.

6. The edge detection method based on dual-pathway biological vision as described in claim 5, characterized in that: In the USM Layer module, the input results are sequentially processed by 1×1 convolution, grouped normalization, and Gelu function to obtain the output results; In the C Layer module, the input results are processed sequentially through 3×3 convolution, pixel shuffle function, and group normalization to obtain the output results.

7. The edge detection method based on dual-pathway biological vision as described in claim 1, characterized in that: In the aforementioned decoding network DN The processing results of the first path fusion module PFM and the second path fusion module PFM are input into a feature fusion module FFM for fusion processing to obtain the first feature fusion result; The processing results of the second and third path fusion modules PFM are input into a feature fusion module FFM for fusion processing to obtain the second feature fusion result. The first feature fusion result and the second feature fusion result are input into the third feature fusion module FFM. After being fused by the third feature fusion module FFM, the result is obtained after 1×1 convolution.

8. The edge detection method based on dual-pathway biological vision as described in claim 7, characterized in that: The feature fusion module FFM includes D respectively. i,j Characteristic pathways and D i，j+1 Characteristic pathways; In D i,j In the feature path, the previous input result is processed by the C Layer module to obtain D. 1,i Pathway outcome; In D i，j+1 In the feature path, the next input result is processed sequentially through the C Layer module and the USM Layer module to obtain D. i，j+1 Pathway outcome; D i,i Pathway outcomes and D i，j+1 After the path results are merged together by channel, they are processed by the C Layer module to obtain the output results.

9. The edge detection method based on dual-pathway biological vision as described in claim 8, characterized in that: In the US Layer module, the input results are sequentially processed by 1×1 convolution, grouped normalization, and Gelu function to obtain the output results; In the C Layer module, the input results are processed sequentially through 3×3 convolution, pixel shuffle function, and group normalization to obtain the output results.

Images