Real-time edge detection method based on bionic bidirectional multi-scale cascade lightweight

Abstract

The present invention aims to provide a lightweight, real-time edge detection method based on a biomimetic bidirectional multi-scale cascade. The method comprises the following steps: A. constructing a neural network, wherein the neural network structure is as follows: an encoding network and a decoding network; the encoding network comprises N groups of S2D modules and N groups of D2S modules, with the number of S2D and D2S modules being the same; B. inputting an original image into the encoding network and dividing it into two paths: the first path is processed sequentially by multiple groups of S2D modules; the second path is first processed by a convolutional layer downsampling and then by multiple groups of D2S modules; the processing results of the multiple groups of S2D modules in the first path and the multiple groups of D2S modules in the second path are concatenated to obtain N processing results, which are then input into the decoding network; C. the decoding network decodes the multiple results input from the encoding network to obtain a final output contour. The present invention significantly reduces parameter size and computational cost while achieving excellent edge detection performance.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a real-time edge detection method based on bionic bidirectional multi-scale cascade lightweighting. Background Art

[0002] The goal of edge detection is to remove unnecessary information from an image, preserving only the important structural attributes. High-performance edge detectors can improve the performance of downstream tasks such as object recognition and image segmentation. Edge detection is widely used in industrial production, for example, in steel surface quality control and PCB production processes.

[0003] The existing edge detection methods are as follows:

[0004] Classic methods. Canny and Laplacian methods calculate gradients using pixel differences and then set a threshold to identify pixels with discontinuous brightness. Roberts uses template matching to detect diagonal pixels. These early methods are highly sensitive to image noise and lack an understanding of image semantics, making them unable to distinguish edges and textures.

[0005] CNN-based methods. Deep learning-based methods continue to push the performance ceiling of edge detection. The HED method uses VGG16 to extract edge features. The CED method uses sub-pixel convolution and a step-by-step upsampling strategy to address the HED method's problem of blurry edges. The RCF method fully utilizes the 13 convolutional layers of VGG16 to extract richer multi-scale features. The LRC method extends the general refinement architecture and designs edge detectors with different refinement levels to extract richer convolutional features. The DRNet method stacks multiple refinement modules and adaptively fuses the cross-entropy and dice loss functions to achieve excellent edge detection performance. Although this method has gradually improved in performance, its computational cost has also increased, and its parameter utilization efficiency is very low. This is very unfriendly to devices with low computing performance and is not conducive to the widespread application of edge detectors. Summary of the Invention

[0006] The present invention aims to provide a real-time edge detection method based on biomimetic bidirectional multi-scale cascade lightweight. This method is inspired by biological vision work and deep learning technology. Unlike most existing networks, this method does not require pre-training and greatly reduces the parameter scale and computational cost. It can also perform real-time edge detection on devices with lower computing power while having good edge detection performance.

[0007] The technical solutions of the present invention are as follows:

[0008] The bionic bidirectional multi-scale cascade lightweight real-time edge detection method comprises the following steps:

[0009] A. Construct a neural network. The neural network structure is as follows:

[0010] Includes encoding network and decoding network;

[0011] The coding network includes N groups of S2D modules and N groups of D2S modules, and the number of S2D modules and D2S modules is the same;

[0012] B. The original image is input into the encoding network and is divided into two paths. The first path is processed by multiple groups of S2D modules in sequence. Each group is numbered according to the processing order, and the processing results of each group of S2D modules are obtained respectively. x-1 ×W / 2 x-1 ×C2 x-1 / 2, where X represents the S2D module group number;

[0013] The second path is first processed by the convolution layer downsampling to make its resolution and number of channels consistent with the processing results of the last group of S2D modules, and then processed by multiple groups of D2S modules in sequence. Each group is numbered in the reverse order of the processing order, and the processing results of each group of S2D modules are obtained respectively. Y-1 ×W / 2 Y-1 ×C2 Y-1 / 2, where Y represents the S2D module group number;

[0014] The processing results of the first multiple groups of S2D modules and the processing results of the second multiple groups of D2S modules are concatenated according to the group numbers to obtain N processing results, which are input into the decoding network;

[0015] C. The decoding network decodes multiple results input from the encoding network to obtain the final output contour.

[0016] Preferably, C is 20, 28, or 36. These three values ​​correspond to: 20 (Small configuration), 28 (base configuration), 36 (Large configuration), respectively.

[0017] The processing process in the S2D module is as follows:

[0018] The input result is processed by 3×3 convolution, Norm function, ReLU function, Dropout function, and 3×3 convolution in sequence, and the processed result is multiplied by the input result to obtain the output result.

[0019] The processing process in the D2S module is as follows:

[0020] The input result is processed by 3×3 convolution, Dropout function, ReLU function, Norm function, and 3×3 convolution in sequence, and the processed result is multiplied by the input result to obtain the output result.

[0021] The decoding network includes N groups of upsampling modules and N+1 groups of downsampling modules;

[0022] The decoding network processing process is as follows:

[0023] The N processing results of the encoding network are in order from large to small resolution. The processing result with the largest resolution of the encoding network is input into the first downsampling module to obtain the first downsampling result; the first downsampling result and the processing result with the second resolution of the encoding network are spliced ​​with the Concat function, and then input into the second downsampling module to obtain the second downsampling result; the second downsampling result and the processing result with the third resolution of the encoding network are spliced ​​with the Concat function, and then input into the third downsampling module to obtain the third downsampling result; and so on, the processing result of the Nth downsampling module is obtained. After the processing result is processed by the N+1th downsampling module, a processing result with the resolution reduced by half and the number of channels doubled is obtained, which is input into the first upsampling module for processing, and the first upsampling processing result with the same resolution and number of channels as the processing result of the Nth downsampling module is output;

[0024] The first upsampling processing result and the processing result of the Nth downsampling module are concatenated by the Concat function and input into the second upsampling module for processing to obtain the second upsampling processing result; the second upsampling processing result and the processing result of the N-1th downsampling module are concatenated by the Concat function and output into the third upsampling module for processing to obtain the third upsampling processing result; and so on. Finally, the processing result of the Nth upsampling module and the processing result of the first downsampling module are concatenated by the Concat function and processed by 1×1 convolution to obtain the final output result.

[0025] Preferably, N=4-6.

[0026] The processing process in the downsampling module is as follows: the input result is processed by the Norm function, ReLU function, 3×3 convolution, Norm function, ReLU function, Dropout function, and 3×3 convolution in sequence, and the processed result is multiplied by the input result and then processed by the ReLU function to obtain the output result.

[0027] The processing process in the upsampling module is as follows:

[0028] The input result is processed by transposed convolution, and the transposed convolution is processed by Norm function, ReLU function, 3×3 convolution, Norm function, ReLU function, Dropout function, and 3×3 convolution in sequence. The processing result is multiplied by the input result and then processed by ReLU function to obtain the output result.

[0029] The method of the present invention is inspired by the biological visual pathway and proposes a bidirectional pathway structure encoder according to the working principle of the two pathways in the visual system. The encoder consists of a bottom-up S2D and a top-down D2S parallel bidirectional network, which can efficiently extract multi-scale features.

[0030] Inspired by the complex cellular information processing mechanism, the method of the present invention proposes a multi-scale interactive decoder to integrate multi-scale features, give full play to the multi-scale advantages of CNN, and generate better contour images.

[0031] Compared to existing edge detection methods of the same type, the method of the present invention uses a relatively small computational cost and parameter size, and has better edge detection performance. It is worth emphasizing that the method of the present invention has the highest inference efficiency among similar methods, which is very friendly to devices with lower computing performance.

[0032] Ablation experiments have demonstrated that these two methods effectively improve network performance. Ablation experiments were also conducted on network complexity, identifying three effective configurations. The base configuration achieves a balance between performance and inference speed, the large configuration provides a better ODS score, and the small configuration achieves the highest inference speed at a lower computational cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the structure of the neural network of Example 1 of the present invention;

[0034] Figure 2 This is a schematic diagram of the structure of the coding network according to Embodiment 1 of the present invention;

[0035] Figure 3 Schematic diagram of the structure of the decoding network of embodiment 1 of the present invention;

[0036] Figure 4 This is a comparison chart of the edge detection method of Example 1 of the present invention and the edge detection method of References 1-2. DETAILED DESCRIPTION

[0037] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0038] Example 1

[0039] The bionic bidirectional multi-scale cascade lightweight real-time edge detection method comprises the following steps:

[0040] A. Build Figure 1 The neural network shown in the figure has the following structure:

[0041] Includes encoding network and decoding network;

[0042] like Figure 2 As shown, the coding network includes 4 groups of S2D modules and 4 groups of D2S modules; Figure 3 As shown, the decoding network includes 4 groups of upsampling modules and 5 groups of downsampling modules;

[0043] B. Figure 2 As shown, the original image is input into the encoding network and is divided into two paths. The first path is processed by four groups of S2D modules in sequence to generate Result characteristics;

[0044] The second path is downsampled by 8×8 convolution and adjusted to After being processed by 4 groups of D2S modules, and Result characteristics;

[0045] The processing results of the first channel multi-group S2D module and the processing results of the second channel multi-group D2S module are paired one by one according to the results with equal resolution and number of channels, and are spliced ​​respectively to obtain 4 processing results, namely Side-output_1 (H×W×3), Side-output_2 Side-output_3 and Side-output_4 Input into the decoding network;

[0046] The processing process in the S2D module is as follows:

[0047] The input result is processed by 3×3 convolution, Norm function, ReLU function, Dropout function, and 3×3 convolution in sequence, and the processed result is multiplied by the input result to obtain the output result.

[0048] The processing process in the D2S module is as follows:

[0049] The input result is processed by 3×3 convolution, Dropout function, ReLU function, Norm function, and 3×3 convolution in sequence, and the processed result is multiplied by the input result to obtain the output result;

[0050] C. The decoding network decodes multiple results input from the encoding network to obtain the final output contour.

[0051] like Figure 3 As shown, the decoding network processing process is as follows:

[0052] The first to fourth downsampling modules receive Side-output_1 (H×W×3), Side-output_2 from the encoder respectively Side-output_3 and Side-output_4 The features are then downsampled and fused step by step. After the fifth downsampling module, we get The first four upsampling modules receive the features of the same resolution output from the downsampling modules step by step, upsample and merge the features, improving the resolution while reducing the number of channels, and finally output an H×W×1 edge image.

[0053] The processing process in the downsampling module is as follows: the input result is processed by the Norm function, ReLU function, 3×3 convolution, Norm function, ReLU function, Dropout function, and 3×3 convolution in sequence, and the processed result is multiplied by the input result and then processed by the ReLU function to obtain the output result.

[0054] The processing process in the upsampling module is as follows:

[0055] The input result is processed by transposed convolution, and the transposed convolution is processed by Norm function, ReLU function, 3×3 convolution, Norm function, ReLU function, Dropout function, and 3×3 convolution in sequence. The processing result is multiplied by the input result and then processed by ReLU function to obtain the output result.

[0056] Example 2

[0057] For the performance evaluation of the contour map output by the network, we use the same performance measurement criteria as those in References 1, 2, and 3. The specific evaluation is shown in Formula (3).

[0058]

[0059] Where P represents precision and R represents recall. A larger value of F indicates better performance.

[0060] Document 1: Z.Su et al., "Pixel difference networks for efficient edge detection," in Proceedings of the IEEE / CVF International Conference on Computer Vision, 2021, pp.5117-5127.

[0061] Document 2: Z.Luo, C.Lin, F.Li, and Y.Pan, "BLEDNet: Bio-inspired lightweight neural network for edge detection," Engineering Applications of Artificial Intelligence, vol.124, p.106530, 2023.

[0062] Figure 4 The following table shows, from left to right, three randomly selected natural images from the Berkeley Segmentation Dataset (BSDS500), the corresponding true contour maps, the optimal contour maps detected by the methods of References 1 and 2, and the optimal contour detected by the method of the present invention in Example 1. The following table compares the performance of the method of the present invention in the following modes:

[0063] Table 1 shows the performance comparison between the edge detection method provided in the embodiment and the edge detection methods in references 1 and 2. '*' indicates that in order to avoid the IO bottleneck of the high-performance GPU, this experiment uses the lower-performance RTX 3060 GPU for testing.

[0064]

[0065] The method of the present invention explores the impact of different network complexities on performance, and changes the complexity of the network by adjusting the number of channels C. The results are shown in Table 1, which measures the computational cost and performance. When the number of channels C = 28, the best ODS performance-cost balance is achieved. The inference speeds of all configurations of the method of the present invention are close to 255, because the computational costs of these configurations are very low for the GPU we use. The inference bottleneck of the method of the present invention is not the computational cost of the model, but the IO transmission speed. For this reason, the experiment of the present invention used the RTX 3060 GPU with lower performance for testing and verification, which avoided the problem of the GPU computing speed being too fast, resulting in most of the time waiting for data transmission. The experimental data in Table 1 proves that as the computational cost of the model increases, its inference speed gradually decreases.

Claims

1. A real-time edge detection method based on bionic bidirectional multi-scale cascade lightweight, characterized by The following steps are involved: A. Construct a neural network. The neural network structure is as follows: Includes encoding network and decoding network; The coding network includes N groups of S2D modules and N groups of D2S modules, and the number of S2D modules and D2S modules is the same; B. The original image is input into the encoding network and is divided into two paths. The first path is processed by multiple groups of S2D modules in sequence. Each group is numbered according to the processing order, and the processing results of each group of S2D modules are obtained respectively. x-1 ×W / 2 x-1 ×C2 x-1 / 2, where X represents the S2D module group number; The second path is first processed by the convolution layer downsampling to make its resolution and number of channels consistent with the processing results of the last group of S2D modules, and then processed by multiple groups of D2S modules in sequence. Each group is numbered in the reverse order of the processing order, and the processing results of each group of S2D modules are obtained respectively. Y-1 ×W / 2 Y-1 ×C2 Y-1 / 2, where Y represents the S2D module group number; The processing results of the first multiple groups of S2D modules and the processing results of the second multiple groups of D2S modules are concatenated according to the group numbers to obtain N processing results, which are input into the decoding network; C. The decoding network decodes multiple results input from the encoding network to obtain the final output contour; The decoding network includes N groups of upsampling modules and N+1 groups of downsampling modules; The decoding network processing process is as follows: The N processing results of the encoding network are in order from large to small resolution. The processing result with the largest resolution of the encoding network is input into the first downsampling module to obtain the first downsampling result; the first downsampling result and the processing result with the second resolution of the encoding network are spliced ​​with the Concat function, and then input into the second downsampling module to obtain the second downsampling result; the second downsampling result and the processing result with the third resolution of the encoding network are spliced ​​with the Concat function, and then input into the third downsampling module to obtain the third downsampling result; and so on, the processing result of the Nth downsampling module is obtained. After the processing result is processed by the N+1th downsampling module, a processing result with the resolution reduced by half and the number of channels doubled is obtained, which is input into the first upsampling module for processing, and the first upsampling processing result with the same resolution and number of channels as the processing result of the Nth downsampling module is output; The first upsampling processing result and the processing result of the Nth downsampling module are concatenated by the Concat function and input into the second upsampling module for processing to obtain the second upsampling processing result; the second upsampling processing result and the processing result of the N-1th downsampling module are concatenated by the Concat function and output into the third upsampling module for processing to obtain the third upsampling processing result; and so on. Finally, the processing result of the Nth upsampling module and the processing result of the first downsampling module are concatenated by the Concat function and processed by 1×1 convolution to obtain the final output result.

2. The real-time edge detection method based on biomimetic bidirectional multi-scale cascade lightweight according to claim 1, characterized in that: The processing process in the S2D module is as follows: The input result is processed by 3×3 convolution, Norm function, ReLU function, Dropout function, and 3×3 convolution in sequence, and the processed result is multiplied by the input result to obtain the output result.

3. The real-time edge detection method based on biomimetic bidirectional multi-scale cascade lightweight according to claim 1, characterized in that: The processing process in the D2S module is as follows: The input result is processed by 3×3 convolution, Dropout function, ReLU function, Norm function, and 3×3 convolution in sequence, and the processed result is multiplied by the input result to obtain the output result.

4. The real-time edge detection method based on biomimetic bidirectional multi-scale cascade lightweight according to claim 1, characterized in that: N=4-6。 5. The real-time edge detection method based on biomimetic bidirectional multi-scale cascade lightweight according to claim 1, characterized in that: The processing process in the downsampling module is as follows: the input result is processed by the Norm function, ReLU function, 3×3 convolution, Norm function, ReLU function, Dropout function, and 3×3 convolution in sequence, and the processed result is multiplied by the input result and then processed by the ReLU function to obtain the output result.

6. The real-time edge detection method based on biomimetic bidirectional multi-scale cascade lightweight according to claim 1, characterized in that: The processing process in the upsampling module is as follows: The input result is processed by transposed convolution, and the transposed convolution is processed by Norm function, ReLU function, 3×3 convolution, Norm function, ReLU function, Dropout function, and 3×3 convolution in sequence. The processing result is multiplied by the input result and then processed by ReLU function to obtain the output result.

7. The real-time edge detection method based on biomimetic bidirectional multi-scale cascade lightweight according to claim 1, characterized in that: C is 20, 28, or 36.

Images