A high-performance edge detection method based on low-rank adaptation

Abstract

The application aims to provide a high-performance edge detection method based on low-rank self-adaption, comprising the following steps: A, constructing a neural network, the neural network structure is specifically as follows: comprising an encoding network, a feature reinforcement network and a decoding network; the encoding network is based on a Vision Transformer network, wherein one reshape module is arranged before and after the 7th, 15th and 23rd Transformer Block layers; B, the original image is sequentially processed by the decoding network, the feature reinforcement network and the decoding network to obtain a final output result. The application adjusts the output distribution of a pre-training main body, simplifies the subsequent module design, greatly improves the performance of the model, effectively improves the delicacy of edge prediction and reduces the complexity of edge prediction.

Description

Technical Field

[0001] This invention relates to the field of computer image processing, and more specifically to a high-performance edge detection method based on low-rank adaptive processing. Background Technology

[0002] Traditional edge detectors can only capture low-level features in images, such as abrupt changes in brightness, contrast, and color. They cannot understand the contextual information of the image, resulting in low-quality edge lines. With the development of deep learning, researchers have proposed various visual backbone networks, including Convolutional Neural Networks (CNNs) and VisionTransformers (ViTs). CNNs initially process local features of the image, gradually capturing global features as the receptive field increases. ViTs divide the image into small patches, then map them into several sequences (tokens) through linear projection, before feeding them into a Transformer structure to extract global features. Inspired by CNNs and ViTs, researchers have developed several powerful edge detectors based on these. These deep learning-based edge detectors can capture not only low-level features but also learn high-level features, thus exhibiting strong performance in edge detection tasks. However, existing methods cannot fully utilize the performance of pre-trained backbones. This is because pre-trained backbones are not designed for edge detection, and the proxy task during pre-training is not edge detection; therefore, the output feature distribution of pre-trained backbones is not suitable for direct edge detection. To address this issue, the methods described above typically employ a series of sampling strategies and secondary computation modules to sample effective features from the pre-trained backbone output and estimate edge images. This approach increases model complexity, and the forcibly redirected features struggle to capture detailed information. Furthermore, forcibly adding modules to the pre-trained backbone inevitably disrupts existing prior knowledge, degrading model performance. Summary of the Invention

[0003] This invention aims to provide a high-performance edge detection method based on low-rank adaptation. This method adjusts the output distribution of the pre-trained backbone, simplifies the subsequent module design, greatly improves the performance of the model, effectively improves the detail of edge prediction, and reduces the complexity of edge prediction.

[0004] The technical solution of the present invention is as follows:

[0005] The high-performance edge detection method based on low-rank adaptation includes the following steps:

[0006] A. Construct a neural network. The specific structure of the neural network is as follows:

[0007] This includes encoding networks, feature enhancement networks, and decoding networks;

[0008] The encoding network is based on the Vision Transformer network and includes a linear projection module and 32 Transformer Block layers connected in sequence. A reshape module is set before and after the 7th, 15th and 23rd Transformer Block layers, respectively.

[0009] B. The original image is input into the decoding network, divided into several patches, then flattened in the H and W dimensions, and after being mapped by the linear projection module, it is sequentially input into 32 Transformer Block layers.

[0010] Specifically: the output of the 6th Transformer Block layer is divided into two paths. The first path is input into the feature enhancement network, and the second path is processed by the reshape module and then input into the 7th Transformer Block layer. The output of the 7th Transformer Block layer is processed by the reshape module to refine its granularity, and then input into the 8th Transformer Block layer and the feature enhancement network respectively.

[0011] The output of the Transformer Block layer in the 14th feature enhancement network is divided into two paths. The first path is input into the feature enhancement network, and the second path is processed by the reshape module and then input into the 15th Transformer Block layer. The processing result of the 15th Transformer Block layer is processed by the reshape module to refine its granularity, and then input into the 16th Transformer Block layer and the feature enhancement network respectively.

[0012] The output of the Transformer Block layer in the 22nd feature enhancement network is divided into two paths. The first path is input into the feature enhancement network, and the second path is processed by the reshape module and then input into the 23rd Transformer Block layer. The output of the 23rd Transformer Block layer is processed by the reshape module to refine its granularity, and then input into the 24th Transformer Block layer and the feature enhancement network respectively.

[0013] The output of the 32nd Transformer Block layer is input into the decoding network.

[0014] C. The output of the feature enhancement network is input into the decoding network;

[0015] D. The decoding network described above fuses the results of all inputs to obtain the final output result.

[0016] The processing procedure in the Transformer Block layer is as follows:

[0017] The input results are divided into two paths. The first path is processed sequentially through the Norm function, Linear layer, Linear layer, Norm function, Acti function, and Linear layer to obtain the first path result. The second path is processed by the Transformer Encoder module to obtain the second path result. The first path result and the second path result are added together to obtain the output result.

[0018] The feature enhancement network includes three feature enhancement modules. The output of the 6th TransformerBlock layer and the processing result of the 7th Transformer Block layer of the decoding network are respectively input into the first feature enhancement module. After processing by the first feature enhancement module, a first feature enhancement result is obtained. The output of the 14th TransformerBlock layer and the processing result of the 15th Transformer Block layer of the decoding network are respectively input into the second feature enhancement module. After processing by the second feature enhancement module, a third feature enhancement result is obtained. The output of the 22nd TransformerBlock layer and the processing result of the 23rd Transformer Block layer of the decoding network are respectively input into the third feature enhancement module. After processing by the third feature enhancement module, a third feature enhancement result is obtained.

[0019] The results of the first feature enhancement, the second feature enhancement, and the third feature enhancement are respectively input into the decoding network.

[0020] The processing procedure in the feature enhancement module is as follows:

[0021] The output of the previous Transformer Block layer input to the decoding network is mapped by the linear projection module to obtain the previous projection result. The output of the next Transformer Block layer input to the decoding network is mapped by the linear projection module to obtain the next projection result. The previous projection result and the next projection result are multiplied together, and then multiplied together again. The final multiplication result is then input into the decoding network after passing through the reshape module.

[0022] The decoding network contains four decoders, and the processing procedure in the decoding network is as follows:

[0023] The output of the Transformer Block layer in the 32nd feature enhancement network is input into the first decoder and processed to obtain the first decoding result. The first decoding result and the third feature enhancement result are multiplied and fused, and then input into the second decoder and processed to obtain the second decoding result. The second decoding result and the second feature enhancement result are multiplied and fused, and then input into the third decoder and processed to obtain the third decoding result. The third decoding result and the first feature enhancement result are multiplied and fused, and then input into the fourth decoder and processed to obtain the final output result.

[0024] The processing procedure in the decoder is as follows:

[0025] The input result is divided into two paths. The first path is processed by 3*3 convolution, Subpixel convolution, and 3*3 convolution in sequence to obtain the first intermediate result. The first intermediate result is divided into two paths. The first path is processed by 3*3 convolution to obtain the second intermediate result. The first path is processed by 1*1 convolution to obtain the third intermediate result.

[0026] The second path is processed by the Interpolate module to obtain the fourth intermediate result; the fourth intermediate result is multiplied and fused with the second intermediate result, and then processed by the Norm function, Acti function, and 3*3 convolution in sequence, and finally added to the third intermediate result to obtain the output result.

[0027] This invention modifies the Transformer Block layer structure and designs a low-rank adaptive method suitable for edge detection tasks. This method captures the feature distribution of the edge detection dataset, including low-level edge location information and high-level edge semantic representation. This effectively adjusts the output distribution of the pre-trained backbone, simplifies subsequent module design, and does not destroy the original prior knowledge, thus greatly improving the model's performance.

[0028] This invention employs a hybrid approach of global and local self-attention, capturing global contextual features in coarse-grained patches and local edge features in fine-grained patches, effectively improving the precision of edge prediction. Unlike existing two-stage hybrid methods, we only require one stage, reducing the complexity of edge prediction.

[0029] This invention also designs a long-term memory backtracking module, which performs cross-attention fusion of shallow, low-level positional features and global contextual features to form a backtracking token. This backtracking token is then fed into deeper, high-level features to "remind" them to backtrack to low-level positional information. This method effectively improves the accuracy of edge localization. Attached Figure Description

[0030] Figure 1This is a schematic diagram of the neural network structure in Embodiment 1 of the present invention;

[0031] Figure 2 This is a schematic diagram of the Transformer Block layer in Embodiment 1 of the present invention;

[0032] Figure 3 This is a schematic diagram of the decoder structure according to Embodiment 1 of the present invention;

[0033] Figure 4 This is a qualitative comparison diagram of contour detection between the method of this invention and existing technical documents. Detailed Implementation

[0034] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0035] Example 1

[0036] A high-performance edge detection method based on low-rank adaptive methods includes the following steps:

[0037] A. Construct a neural network. The specific structure of the neural network is as follows:

[0038] This includes encoding networks, feature enhancement networks, and decoding networks;

[0039] The encoding network is based on the Vision Transformer network and includes a linear projection module and 32 Transformer Block layers connected in sequence. A reshape module is set before and after the 7th, 15th and 23rd Transformer Block layers, respectively.

[0040] B. The original image is input into the decoding network, divided into several patches, then flattened in the H and W dimensions, and after being mapped by the linear projection module, it is sequentially input into 32 Transformer Block layers.

[0041] Specifically: the output of the 6th Transformer Block layer is divided into two paths. The first path is input into the feature enhancement network, and the second path is processed by the reshape module and then input into the 7th Transformer Block layer. The output of the 7th Transformer Block layer is processed by the reshape module to refine its granularity, and then input into the 8th Transformer Block layer and the feature enhancement network respectively.

[0042] The output of the Transformer Block layer in the 14th feature enhancement network is divided into two paths. The first path is input into the feature enhancement network, and the second path is processed by the reshape module and then input into the 15th Transformer Block layer. The processing result of the 15th Transformer Block layer is processed by the reshape module to refine its granularity, and then input into the 16th Transformer Block layer and the feature enhancement network respectively.

[0043] The output of the Transformer Block layer in the 22nd feature enhancement network is divided into two paths. The first path is input into the feature enhancement network, and the second path is processed by the reshape module and then input into the 23rd Transformer Block layer. The output of the 23rd Transformer Block layer is processed by the reshape module to refine its granularity, and then input into the 24th Transformer Block layer and the feature enhancement network respectively.

[0044] The processing procedure in the Transformer Block layer is as follows:

[0045] The input result is divided into two paths. The first path is processed sequentially through the Norm function, Linear layer, Linear layer, Norm function, Acti function, and Linear layer to obtain the first path result. The second path is processed by the Transformer Encoder module to obtain the second path result. The first path result and the second path result are added together to obtain the output result.

[0046] The output of the 32nd Transformer Block layer is input into the decoding network.

[0047] C. The processing procedure in the feature enhancement module is as follows: the output of the previous TransformerBlock layer input to the decoding network is mapped by the linear projection module to obtain the previous projection result; the output of the next Transformer Block layer input to the decoding network is mapped by the linear projection module to obtain the next projection result; the previous projection result and the next projection result are multiplied together, and then multiplied together again; the final multiplication result is then input into the decoding network after passing through the reshape module.

[0048] D. The decoding network described above fuses the results of all inputs to obtain the final output result.

[0049] The feature enhancement network comprises three feature enhancement modules. The output of the 6th TransformerBlock layer and the processing result of the 7th Transformer Block layer of the decoding network are respectively input into the first feature enhancement module. After processing by the first feature enhancement module, a first feature enhancement result is obtained. The output of the 14th TransformerBlock layer and the processing result of the 15th Transformer Block layer of the decoding network are respectively input into the second feature enhancement module. After processing by the second feature enhancement module, a second feature enhancement result is obtained. The output of the 22nd TransformerBlock layer and the processing result of the 23rd Transformer Block layer of the decoding network are respectively input into the third feature enhancement module. After processing by the third feature enhancement module, a third feature enhancement result is obtained.

[0050] The results of the first feature enhancement, the second feature enhancement, and the third feature enhancement are respectively input into the decoding network.

[0051] The decoding network contains four decoders, and the processing procedure in the decoding network is as follows:

[0052] The output of the Transformer Block layer in the 32nd feature enhancement network is input into the first decoder and processed to obtain the first decoding result. The first decoding result and the third feature enhancement result are multiplied and fused, and then input into the second decoder and processed to obtain the second decoding result. The second decoding result and the second feature enhancement result are multiplied and fused, and then input into the third decoder and processed to obtain the third decoding result. The third decoding result and the first feature enhancement result are multiplied and fused, and then input into the fourth decoder and processed to obtain the final output result.

[0053] The processing procedure in the decoder is as follows:

[0054] The input result is divided into two paths. The first path is processed by 3*3 convolution, Subpixel convolution, and 3*3 convolution in sequence to obtain the first intermediate result. The first intermediate result is divided into two paths. The first path is processed by 3*3 convolution to obtain the second intermediate result. The first path is processed by 1*1 convolution to obtain the third intermediate result.

[0055] The second path is processed by the Interpolate module to obtain the fourth intermediate result; the fourth intermediate result is multiplied and fused with the second intermediate result, and then processed by the Norm function, Acti function, and 3*3 convolution in sequence, and finally added to the third intermediate result to obtain the output result.

[0056] Example 2

[0057] The final edge graph is quantitatively evaluated using the same performance measurement standard as in Reference 1, and the specific evaluation is shown in Formula (1).

[0058]

[0059] Where P represents precision and R represents recall. The higher the value of F, the better the performance.

[0060] Document 1: M.Pu, Y.Huang, Y.Liu, Q.Guan, and H.Ling, "Edter: Edge detection with transformer," in Proceedings of the IEEE / CVF conference on computer vision and pattern recognition, 2022, pp.1402–1412.

[0061] The parameters used in Reference 1 are the same as those in the original text, and they are guaranteed to be the optimal parameters for the model.

[0062] like Figure 4 As shown, from left to right, the images are the original image, the true label of the image contour, the contour map detected by the method in Reference 1, and the contour map detected by the real-time Example 1 method of this invention.

[0063] Table 1 summarizes the experimental data of Reference 1 and Example 1 on the BSDS500 dataset. The experimental results show that Example 1 achieves better performance than the detection method in Reference 1 with fewer parameters.

[0064] Table 1 Quantitative Data

[0065] Reference 1 825M 0.824 0.841 Example 1 686M 0.832 0.847 .

Claims

1. A high-performance edge detection method based on low-rank adaptive algorithm, characterized in that, Includes the following steps: A. Construct a neural network. The specific structure of the neural network is as follows: This includes encoding networks, feature enhancement networks, and decoding networks; The encoding network is based on the Vision Transformer network and includes a linear projection module and 32 Transformer Block layers connected in sequence. A reshape module is set before and after the 7th, 15th and 23rd Transformer Block layers, respectively. B. The original image is input into the decoding network, divided into several patches, then flattened in the H and W dimensions, and after being mapped by the linear projection module, it is sequentially input into 32 Transformer Block layers. Specifically: the output of the 6th Transformer Block layer is divided into two paths. The first path is input into the feature enhancement network, and the second path is processed by the reshape module and then input into the 7th Transformer Block layer. The output of the 7th Transformer Block layer is processed by the reshape module to refine its granularity, and then input into the 8th Transformer Block layer and the feature enhancement network respectively. The output of the Transformer Block layer in the 14th feature enhancement network is divided into two paths. The first path is input into the feature enhancement network, and the second path is processed by the reshape module and then input into the 15th Transformer Block layer. The processing result of the 15th Transformer Block layer is processed by the reshape module to refine its granularity, and then input into the 16th Transformer Block layer and the feature enhancement network respectively. The output of the Transformer Block layer in the 22nd feature enhancement network is divided into two paths. The first path is input into the feature enhancement network, and the second path is processed by the reshape module and then input into the 23rd Transformer Block layer. The output of the 23rd Transformer Block layer is processed by the reshape module to refine its granularity, and then input into the 24th Transformer Block layer and the feature enhancement network respectively. The output of the 32nd Transformer Block layer is input into the decoding network. C. The output of the feature enhancement network is input into the decoding network; D. The decoding network described above fuses the results of all inputs to obtain the final output result.

2. The high-performance edge detection method based on low-rank adaptive as described in claim 1, characterized in that: The processing procedure in the Transformer Block layer is as follows: The input results are divided into two paths. The first path is processed sequentially through the Norm function, Linear layer, Linear layer, Norm function, Acti function, and Linear layer to obtain the first path result. The second path is processed by the Transformer Encoder module to obtain the second path result. The output result is obtained by adding and merging the first and second results.

3. The high-performance edge detection method based on low-rank adaptive as described in claim 1, characterized in that: The feature enhancement network includes three feature enhancement modules; the output of the 6th Transformer Block layer and the processing result of the 7th Transformer Block layer of the decoding network are respectively input into the first feature enhancement module, and after processing by the first feature enhancement module, the first feature enhancement result is obtained; The output of the 14th Transformer Block layer and the processing result of the 15th Transformer Block layer of the decoding network are respectively input into the second feature enhancement module. After processing by the second feature enhancement module, the second feature enhancement result is obtained. The output of the 22nd Transformer Block layer and the processing result of the 23rd Transformer Block layer of the decoding network are respectively input into the third feature enhancement module. After processing by the third feature enhancement module, the third feature enhancement result is obtained. The results of the first feature enhancement, the second feature enhancement, and the third feature enhancement are respectively input into the decoding network.

4. The high-performance edge detection method based on low-rank adaptive as described in claim 3, characterized in that: The processing procedure in the feature enhancement module is as follows: The output of the previous Transformer Block layer input to the decoding network is mapped by the linear projection module to obtain the previous projection result. The output of the next Transformer Block layer input to the decoding network is mapped by the linear projection module to obtain the next projection result. The previous projection result and the next projection result are multiplied together, and then multiplied together again. The final multiplication result is then input into the decoding network after passing through the reshape module.

5. The high-performance edge detection method based on low-rank adaptive as described in claim 3, characterized in that: The decoding network contains four decoders, and the processing procedure in the decoding network is as follows: The output of the Transformer Block layer in the 32nd feature enhancement network is input into the first decoder and processed to obtain the first decoding result. The first decoding result and the third feature enhancement result are multiplied and fused, and then input into the second decoder and processed to obtain the second decoding result. The second decoding result and the second feature enhancement result are multiplied and fused, and then input into the third decoder and processed to obtain the third decoding result. The third decoding result and the first feature enhancement result are multiplied and fused, and then input into the fourth decoder and processed to obtain the final output result.

6. The high-performance edge detection method based on low-rank adaptive as described in claim 5, characterized in that: The processing procedure in the decoder is as follows: The input result is divided into two paths. The first path is processed by 3*3 convolution, Subpixel convolution, and 3*3 convolution in sequence to obtain the first intermediate result. The first intermediate result is divided into two paths. The first path is processed by 3*3 convolution to obtain the second intermediate result. The first path is processed by 1*1 convolution to obtain the third intermediate result. The second path is processed by the Interpolate module to obtain the fourth intermediate result; the fourth intermediate result is multiplied and fused with the second intermediate result, and then processed by the Norm function, Acti function, and 3*3 convolution in sequence, and finally added to the third intermediate result to obtain the output result.

Images