Deep interactive fusion double-branch edge detection method and system based on CNN and Transform

Abstract

The invention discloses a double-branch edge detection method and system based on CNN and Transform deep interactive fusion, and relates to the technical field of computer vision and deep learning, and the method comprises the steps: extracting multi-scale local features through a CNN-based fine semantic edge branch by employing an OfficientNet-B2 backbone network, and carrying out the deep supervision in combination with a side output structure; meanwhile, through a global context branch based on lightweight Transform, an efficient local attention mechanism is adopted to capture a long-distance dependency relationship of the image; the core of the invention lies in that a cross-attention module is designed and realized, local features extracted by a CNN branch are used as a Key and a Value, and features of a Transform branch are used as Query, so that deep interaction and fusion of the two branches on the feature level are realized, instead of simple feature splicing; in addition, the method also adopts a uniform loss function for balancing dynamic weighted combination of binary cross entropy loss, focus loss, Dice loss and the like to perform model optimization.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and deep learning, and in particular to a dual-branch edge detection method and system based on deep interactive fusion of CNN and Transformer. Background Art

[0002] Image edge detection is a fundamental and critical task in computer vision. Its goal is to identify and locate the boundaries of objects and the outlines of scene structures in an image. Accurate edge information is an important foundation for subsequent advanced vision tasks such as image segmentation, object recognition, and 3D reconstruction.

[0003] Traditional edge detection algorithms, such as Sobel and Canny, rely primarily on image gradient information. While computationally simple, they often perform poorly when faced with complex textures, weak edges, and noise. In recent years, with the advancement of deep learning technology, methods based on convolutional neural networks (CNNs), such as Holistically Nested Edge Detection (HED), have achieved significant breakthroughs in edge detection by learning image features in an end-to-end manner. CNNs leverage their inherent translation invariance and local receptive field to effectively extract local texture and detailed features from images.

[0004] However, the local nature of CNNs limits their ability to capture long-range dependencies and understand global context. To overcome this limitation, researchers have begun to introduce the Transformer model, which has achieved great success in natural language processing. The Visual Transformer (ViT) and its variants use a self-attention mechanism to effectively establish dependencies between any two pixels in an image, thereby achieving stronger global representation capabilities.

[0005] Therefore, combining the local feature extraction advantages of CNN and the global context modeling capabilities of Transformer has become an important research direction in the current field of edge detection. Most of the current mainstream hybrid models use relatively simple fusion strategies. For example, at the end of two independent branches, the feature maps extracted from each branch are spliced ​​or added together, which is a kind of "late fusion". Although this method combines two features, the two branches are independent of each other during the feature extraction process, lacking deep and real-time information interaction. As a result, global information fails to effectively guide the extraction of local features, and vice versa. This problem of insufficient fusion limits the further improvement of model performance, especially when dealing with challenging scenes with complex backgrounds, variable target scales, and occlusions. The detection accuracy and robustness still need to be improved. Summary of the Invention

[0006] The purpose of the present invention is to solve the problems of insufficient feature interaction and shallow fusion level in the existing CNN and Transformer hybrid models in edge detection tasks, thereby providing a new technical solution that can achieve deep fusion of local details and global context and significantly improve the accuracy and robustness of edge detection.

[0007] The technical solution of the present invention is to provide a dual-branch edge detection method based on deep interactive fusion of CNN and Transformer, which includes:

[0008] S1. Convolution-based fine semantic edge feature extraction: The input digital image is processed by a fine semantic edge branch based on a convolutional neural network to extract multi-scale local features containing rich local texture and contour details. The fine semantic edge branch is connected to a side output structure after multiple different network layers to provide deep supervision of features at each layer and generate multiple side output edge prediction maps.

[0009] S2, Transformer-based global context modeling: In parallel with step S1, the input image is processed through a global context branch based on a lightweight Transformer. This process first divides the image into a series of non-overlapping image blocks, which are then serialized through multiple Transformer modules to capture long-range dependencies between elements in the image and generate global context features.

[0010] S3. Deep interaction of heterogeneous features based on cross-attention mechanism: Through a cross-attention module, deep interaction and information injection between the local features extracted by the fine semantic edge branch and the global features generated by the global context branch are achieved in the middle layer of the model. Specifically, the feature sequence generated by the global context branch is used as the query in the attention calculation, and the local features extracted by the fine semantic edge branch at at least one level are used as the key and value to perform scaled dot product attention calculation.

[0011] S4. Multi-source feature fusion and final edge prediction map generation: A memory-efficient feature fusion module is used to integrate the global contextual features enhanced by the deep interaction of the cross-attention module with the fine semantic feature map output by the fine semantic edge branch. An edge refinement network is then used to generate the final edge prediction logarithmic map without activation function processing.

[0012] S5. Model optimization and training based on a unified loss function: During the model training phase, a unified edge detection loss function consisting of a dynamically weighted combination of multiple loss components is used to jointly optimize the final edge prediction logarithmic map and the multiple side output edge prediction maps to guide the convergence of model parameters.

[0013] In any of the above technical solutions, further, step S1 specifically includes:

[0014] S11. First, the EfficientNet-B2 network is used as the backbone network for feature extraction. A series of multi-scale feature maps with different spatial resolutions and semantic levels are extracted from multiple preset intermediate feature layers of the backbone network.

[0015] S12. Each extracted multi-scale feature map is input into a side output module, which includes a feature refinement network, a spatial attention network, and an edge prediction head. The feature refinement network first further refines each multi-scale feature map. The spatial attention network emphasizes important areas in the image and suppresses unimportant areas, thereby improving the expressiveness of the feature map. The edge prediction head generates a final edge prediction map, the output of which is a logarithmic map without Sigmoid activation function processing. Each multi-scale feature map corresponds to a prediction result, providing the model with multiple edge prediction maps.

[0016] S13. After completing the processing of the above multi-scale features, the logarithmic images generated by all side output modules will be upsampled to the same size as the input image through bilinear interpolation. Each upsampled logarithmic image will be weighted one by one according to a set of learnable fusion weight parameters. The weight parameters are normalized by the softmax function, and the kth normalized weight is weight k The calculation method is:

[0017]

[0018] Where N is the number of side output modules, N ≥ k ≥ 1, w j is the weight parameter corresponding to the j-th side output feature map, which is not normalized; all weighted side output feature maps are finally spliced ​​in the channel dimension to form a feature tensor that integrates multi-scale information;

[0019] S14. Finally, all weighted logarithmic maps are fused in the channel dimension to obtain a fused feature map containing rich multi-scale information. The fused feature map is input into a fused convolutional network composed of depthwise separable convolution and standard convolution for further feature purification and dimensionality reduction, and finally a 32-channel feature map is generated, which is used for subsequent cross-attention interaction.

[0020] In any of the above technical solutions, further, step S2 specifically includes:

[0021] S21, Embed Lightweight Image Blocks: The input image is processed into multiple image blocks. Specifically, a lightweight image block embedding module converts the input image from pixel space into a one-dimensional, sequential image block embedding sequence. The module uses a series of convolutional layers with a stride of 2 to gradually reduce the spatial resolution and increase the channel dimension.

[0022] S22. Efficient Local Self-Attention Calculation: The core of the Transformer module uses an efficient local self-attention mechanism. In this mechanism, the image feature map is divided into multiple non-overlapping local windows. Within each local window, multi-head self-attention is independently calculated to capture long-range dependencies within the window.

[0023] S23, Learnable Position Encoding Injection: Add a learnable position encoding parameter matrix to the image block embedding sequence where N p is the number of image blocks, and D is the embedding dimension; this position encoding enables the model to perceive the spatial position information of each image block in the original image, and its injection method is element-by-element addition.

[0024] In any of the above technical solutions, further, step S3 includes:

[0025] S31. Generation of query, key and value: First, in the cross-attention module, the interaction of the two branch features is realized by generating three matrices: query, key and value. Specifically, the global context features from the Transformer branch are used as query. These features are first processed by multiple Transformer modules to generate a feature sequence of the image block. Where B represents the batch size, N p is the number of image blocks, D q is the embedding dimension of the query, and then, through a linear projection layer W Q Transform these features to obtain the final query matrix Q = F trans W Q ;

[0026] Next, the 32-channel fine semantic feature map F from the fine semantic edge branch cnn ∈R B×32×H×W Flatten and transpose to obtain the serialized feature F′ cnn ∈R B×(H·W)×32 , these features are passed through two independent linear projection layers W K and WV Transformation, respectively generate the key matrix K = F' cnn W K Sum value matrix V = F′ cnn W V ;

[0027] S32, scaled dot product attention calculation: After completing the query, key and value generation, multi-head scaled dot product attention calculation is performed. The core calculation formula of this process is:

[0028]

[0029] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, d k is the dimension of the key vector; through this formula, the similarity between the query and the key is calculated by dot product and normalized by the softmax function to obtain the attention weights, which are then used to weight the value matrix to generate the final attention output features;

[0030] The attention calculation process is multi-headed, that is, the process is executed in parallel on multiple attention heads, and the multiple results are concatenated and integrated through a linear projection layer to generate the final cross-attention feature sequence;

[0031] S33. Feature output after interaction: Finally, the deep interaction features calculated by the multi-head attention mechanism will be output through a linear projection layer, and the obtained feature sequence will be used for subsequent feature fusion and edge prediction.

[0032] In any of the above technical solutions, further, the overall part of the unified edge detection loss function in step S5 is as follows:

[0033]

[0034] in, is the total loss, The loss acting on the final fused prediction graph, is the loss of the k-th side output, w main and w side are the weighting coefficients of the main output and side output respectively.

[0035] 6. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer as claimed in claim 5, characterized in that the loss term as well as They are all weighted combinations of at least two of the following three loss functions: balanced binary cross entropy loss, focal loss, and Dice loss, and the combination is:

[0036]

[0037] in, Reference or λ BCE ,λ Focal ,λ Dice Is the weight coefficient of each loss. The relevant calculation methods of the three loss functions include:

[0038] Balanced binary cross entropy loss By automatically calculating weights for positive samples to alleviate the class imbalance problem, the positive sample weight w pos The calculation formula is: where N neg and N pos are the total number of pixels of negative and positive samples in the batch;

[0039] focal loss Reduce the weight of easy-to-classify samples so that the model can focus more on learning difficult-to-classify samples. The calculation formula is: Among them, p t is the model’s predicted probability for the correct category, α t is the balance factor, γ is the focusing parameter;

[0040] Dice loss The structural integrity of the edge is optimized by maximizing the overlap area between the prediction and the true value, and its calculation formula is: where p i and g i are the values ​​of the predicted probability map and the true label map at pixel i, respectively, and ∈ is a smoothing coefficient to prevent the denominator from being zero.

[0041] In any of the above technical solutions, further, the loss weight is dynamically adjusted during the training process, specifically including:

[0042] The loss weight scheduler adjusts the weight of each loss term according to a preset scheduling strategy. When using the cosine annealing scheduling strategy, the model initially focuses on balancing the pixel-level BCE loss and focal loss to quickly learn basic edge features. As training progresses, the weight of the Dice loss gradually increases, guiding the model to pay more attention to the structural integrity of edges.

[0043] The adjustment factor f of the loss weight cos Calculated by the following formula:

[0044]

[0045] Among them, total_epochs is the total number of rounds of model training, current_epoch is the current training round number; then, according to the adjustment factor f cos Adjust the weight of each loss term.

[0046] A detection system using a dual-branch edge detection method based on deep interactive fusion of CNN and Transformer in any of the above technical solutions is also provided, the system comprising:

[0047] A global context module that processes the input image into blocks and extracts global context features containing long-range dependencies;

[0048] A cross-attention interaction module that uses the features generated by the global context module as queries and the features generated by the fine semantic edge module as keys and values, enabling deep interaction and enhancement of features from both modules;

[0049] A feature fusion module that integrates the enhanced features processed by the cross-attention interaction module with the fine semantic feature map generated by the fine semantic edge module and generates the final edge prediction result;

[0050] A training module to train the system end-to-end.

[0051] The beneficial effects of the present invention are:

[0052] This paper innovatively designs a cross-attention module, enabling deep interaction between CNN local features and Transformer global features in the model's middle layers, avoiding simple post-processing. This deep interaction enables global context to effectively guide the selection of local details, while local details enrich the global representation, significantly improving the model's detection accuracy in challenging scenarios such as complex backgrounds and weak edges.

[0053] By employing innovative designs such as the lightweight EfficientNet-B2 in the CNN branch, a highly efficient local attention mechanism in the Transformer branch, and memory-efficient modules in the fusion phase, this paper reduces the total number of model parameters to 8.73M, approximately 56% of existing technologies (such as RCF). While significantly reducing model complexity, the detection accuracy surpasses existing technologies, achieving the goal of "higher accuracy and lower power consumption."

[0054] The core innovation of this invention lies in the use of a self-developed unified loss function, Unified Edge Detection Loss, which combines balanced BCE, Focal, and Dice losses to effectively solve the problem of class imbalance between edge pixels and background pixels. The loss weight scheduler further optimizes the training process, allowing the model to focus on different learning objectives at different training stages. In addition, powerful data augmentation strategies (such as elastic transformation and CLAHE) significantly improve the generalization ability and robustness of the model.

[0055] During the evaluation phase, the present invention uses a fault-tolerant matching mechanism based on distance transforms, allowing for a certain pixel deviation (e.g., 2 pixels) between the predicted edge and the true edge. This approach better aligns with human perception than traditional pixel-by-pixel precise matching. Therefore, the evaluation results (such as ODS and OIS) are more objective and have practical reference value. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The advantages of the above and additional aspects of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0057] Figure 1 1 is a schematic diagram of the overall process of a dual-branch edge detection method and system based on deep interactive fusion of CNN and Transformer according to an embodiment of the present invention;

[0058] Figure 2 1. It is a structural diagram of a dual branch edge detection model (Dual Branch Edge Model) based on a dual branch edge detection method and system based on deep interactive fusion of CNN and Transformer according to an embodiment of the present invention;

[0059] Figure 3 1 is a structural diagram of a cross-attention module (Cross-Attention Module) of a dual-branch edge detection method and system based on deep interactive fusion of CNN and Transformer according to an embodiment of the present invention;

[0060] Figure 4 2 is a structural diagram of a Memory Efficient Fusion module of a dual-branch edge detection method and system based on deep interactive fusion of CNN and Transformer according to an embodiment of the present invention;

[0061] Figure 5 1 is a structural diagram of a unified edge detection loss function (Unified Edge Detection Loss) of a dual-branch edge detection method and system based on deep interactive fusion of CNN and Transformer according to an embodiment of the present invention;

[0062] Figure 6 1. It is a schematic diagram of the principle of tolerance matching based on a dual-branch edge detection method and system with deep interactive fusion of CNN and Transformer according to an embodiment of the present invention;

[0063] Figure 7 This is a typical image showing the entire fault-tolerant matching evaluation process of a dual-branch edge detection method and system based on deep interactive fusion of CNN and Transformer according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features therein can be combined with each other without conflict.

[0065] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0066] like Figure 1 As shown, this embodiment provides a dual-branch edge detection method based on deep interactive fusion of CNN and Transformer, including:

[0067] S1. Convolution-based fine semantic edge feature extraction: First, the input digital image is processed through a fine semantic edge branch based on a convolutional neural network (CNN) to extract multi-scale local features containing rich local texture and contour details; this branch uses EfficientNet-B2 as the backbone network to extract feature maps of different levels from the image. The output features of each feature level are connected to the side output module to perform deep supervision on the features of each level; the side output module enhances the features of each level through an improved side output structure and generates multiple edge prediction maps (in the form of logits).

[0068] Step S1 specifically includes:

[0069] S11. First, the EfficientNet-B2 network is used as the backbone network for feature extraction. A series of multi-scale feature maps with different spatial resolutions and semantic levels are extracted from multiple preset intermediate feature layers of the backbone network (specifically after the 1st, 2nd, 4th, 6th, and 7th main blocks); these feature maps represent the detailed information of different levels of the image, from low-level texture features to high-level semantic information, forming a rich multi-scale feature set.

[0070] S12. Each extracted multi-scale feature map is input into an independent, improved side output module, which includes a feature refinement network, a spatial attention network and an edge prediction head; the feature refinement network first further refines each multi-scale feature map to enhance its performance in details; the spatial attention network emphasizes important areas in the image and suppresses unimportant areas in an adaptive manner, thereby improving the expressiveness of the feature map; the edge prediction head is responsible for generating the final edge prediction map, and its output is a logarithmic map (logits) without Sigmoid activation function processing; each multi-scale feature map corresponds to a prediction result, providing the model with multiple edge prediction maps.

[0071] S13. After completing the processing of the above multi-scale features, the logits generated by all side output modules will be upsampled to the same size as the input image through bilinear interpolation. Each upsampled logit will be weighted one by one according to a set of learnable fusion weight parameters. The weight parameters are normalized by the softmax function, and the kth normalized weight is weight k The calculation method is:

[0072]

[0073] Where N is the number of side output modules, N ≥ k ≥ 1, w j is the weight parameter corresponding to the j-th side output feature map and is not normalized.

[0074] All weighted side output feature maps are finally concatenated in the channel dimension to form a feature tensor that integrates multi-scale information.

[0075] S14. Finally, all weighted logarithmic maps are fused in the channel dimension to obtain a fused feature map containing rich multi-scale information. In order to further enhance the expressive power of these fused feature maps, the fused feature map is input into a fused convolutional network composed of depthwise separable convolution and standard convolution for further feature purification and dimensionality reduction, and finally a 32-channel feature map is generated, which will be used for subsequent cross-attention interaction.

[0076] Through the above process, step S1 realizes the extraction of fine semantic edge features with multi-scale information and semantic levels from the input image, and enhances the expressive ability of local features through deep supervision and feature fusion mechanism, laying the foundation for subsequent deep interaction and edge prediction.

[0077] S2. Transformer-based global context information modeling: In parallel with step S1, the input image is also processed by a global context branch based on a lightweight Transformer. First, the input image is divided into a series of non-overlapping image patches, and then serialized through multiple Transformer modules to capture long-distance dependencies between elements in the image and generate global context features. This process uses an efficient local attention mechanism to reduce computational costs while maintaining the ability to model global image information.

[0078] The purpose of this branch is to extract global context features in the image and capture the dependencies between distant pixels in the image. Step S2 specifically includes:

[0079] S21. Embed lightweight image blocks: The input image is processed into multiple image blocks. Specifically, a lightweight image block embedding module is used to convert the input image from pixel space into a one-dimensional, serialized image block embedding sequence. The module gradually reduces the spatial resolution and increases the channel dimension through a series of convolutional layers with a stride of 2.

[0080] Unlike the single large-stride convolution used in the traditional ViT (Vision Transformer) model, the method of the present invention better retains the local information of the image while reducing the dimensionality, thereby improving computational efficiency and enhancing model performance; in this way, the input image is effectively embedded in a feature space suitable for Transformer model processing.

[0081] S22. Efficient local self-attention calculation: The core of the Transformer module adopts an efficient local self-attention mechanism, in which the image feature map is divided into multiple non-overlapping local windows (for example, 7×7); within each local window, multi-head self-attention is calculated independently to capture long-range dependencies within the window.

[0082] Compared with the global self-attention mechanism, this local self-attention calculation greatly reduces the computational complexity and memory requirements. The self-attention mechanism of each window effectively models local features, enabling the model to learn rich contextual information in different local areas, thereby enhancing the global understanding of the image. This design not only improves model efficiency, but also reduces the consumption of computing resources, making it more efficient when processing large-scale images.

[0083] S23, learnable position coding injection: In the image block embedding sequence, the present invention adds a learnable position coding parameter matrix to enable the model to perceive the spatial position information of each image block in the original image; the position coding matrix is where N p is the number of image patches, and D is the embedding dimension; position encoding enables the model to learn the spatial relationship of image patches in the original image and inject it into the image patch embedding sequence by element-by-element addition.

[0084] This operation enables the model to process the position information in the image through the Transformer's self-attention mechanism without traditional convolution operations, and fully consider the spatial relationship between image blocks when modeling the global context.

[0085] Through the above steps, the branch of step S2 can not only efficiently capture the global structure of the image, but also maintain effective modeling of long-distance dependencies under limited computing resources; the core goal of this step is to obtain the global context information of the image through an efficient local self-attention mechanism and image block embedding technology, providing strong support for subsequent cross-attention interaction steps.

[0086] S3. Deep interaction of heterogeneous features based on cross-attention mechanism: A cross-attention module is used to achieve deep interaction between the local features extracted by the CNN branch and the global features generated by the Transformer branch. This process combines features from two different branches to achieve deep fusion and information injection at the feature level.

[0087] In traditional edge detection methods, local features and global features are usually fused through simple splicing or addition operations. However, the present invention uses an innovative cross-attention mechanism to enable these two features to interact at a deeper level, thereby improving the accuracy and robustness of edge detection.

[0088] Specifically, the global context features generated by the Transformer branch are used as the query (Query), and the local features of the CNN branch are used as the key (Key) and value (Value) to perform scaled dot product attention calculation. This process effectively injects global context information into local features, achieving a deep fusion of local details and global information, thereby enhancing the accuracy of edge detection. Step S3 specifically includes:

[0089] S31. Generation of query, key, and value: First, in the cross-attention module, the interaction of the two branch features is achieved by generating three matrices: query, key, and value. Specifically, the global context features from the Transformer branch are used as query. These features are first processed by multiple Transformer modules to generate a feature sequence of the image block. Where B represents the batch size, N p is the number of image blocks, D q is the embedding dimension of the query, and then, through a linear projection layer W Q Transform these features to obtain the final query matrix Q = F trans W Q .

[0090] Next, the 32-channel fine semantic feature map F from the fine semantic edge branch cnn ∈R B×32×H×W Flatten and transpose to obtain the serialized feature F′ cnn ∈R B×(H·W)×32 , these features are passed through two independent linear projection layers W K and W V Transformation, respectively generate the key matrix K = F' cnn W K Sum value matrix V = F′ cnn W V .

[0091] S32, scaled dot product attention calculation: After completing the query, key and value generation, the core operation of the cross-attention mechanism is further implemented through multi-head scaled dot product attention calculation. The core calculation formula of this process is:

[0092]

[0093] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, d k is the dimension of the key vector; through this formula, the similarity between the query and the key is calculated by the dot product and normalized by the softmax function to obtain the attention weights, which are then used to weight the value matrix to generate the final attention output features.

[0094] The attention calculation process is multi-headed, that is, the process is executed in parallel on multiple attention heads, and the multiple results are concatenated and integrated through a linear projection layer to generate the final cross-attention feature sequence.

[0095] Through this process, local features from the fine semantic edge branch are effectively injected into the global context features from the Transformer branch, achieving a deep fusion of local and global information; this cross-attention mechanism enables global features to effectively guide the extraction of local features, while local features provide detailed support for global features, thereby improving the performance of the edge detection model.

[0096] S33. Feature output after interaction: Finally, the deep interaction features calculated by the multi-head attention mechanism will be output through a linear projection layer, and the obtained feature sequence will be used for subsequent feature fusion and edge prediction; this deep interaction enhanced feature sequence not only contains local detail information, but also integrates global context information, which can provide a more comprehensive image understanding; the output features of the cross-attention module will provide support for subsequent steps, especially in the multi-source feature fusion stage, further improving the model's ability to recognize edges in complex scenes.

[0097] S4. Multi-source feature fusion and final edge prediction map generation: Through a memory-efficient feature fusion module, the global context features after deep interaction across attention modules are integrated with the fine semantic feature map output by the fine semantic edge branch. This fusion process generates the final edge prediction map through an edge refinement network. The prediction map is output in the form of logits and is not processed by the activation function, ensuring the flexibility of the prediction results so that they can be adjusted according to specific needs later.

[0098] In this step, a memory-efficient feature fusion module is used to integrate the global contextual features enhanced by deep interactions across the attention modules with the fine semantic feature map output by the fine semantic edge branch. The goal of this process is to fuse the feature information from both branches to generate a more refined and accurate edge prediction map.

[0099] S5. Model optimization and training based on a unified loss function: During the model training phase, a unified edge detection loss function consisting of a dynamically weighted combination of multiple loss components is used to jointly optimize the final edge prediction logarithmic map and multiple side output edge prediction maps. The loss function includes balanced binary cross entropy loss, focal loss, and Dice loss. The weights of the loss terms are dynamically adjusted through the softmax function to ensure that the model gradually converges to the optimal state during training.

[0100] In step S5, the training and optimization of the model are completed by adopting a unified edge detection loss function that is a dynamically weighted combination of multiple loss components; this loss function is used to guide the training of the model, ensuring the joint optimization of the final edge prediction map and multiple side output edge prediction maps, thereby improving the accuracy and robustness of the model in edge detection tasks.

[0101] The structure of the unified loss function includes multiple loss terms, each of which optimizes a different part of the model. By weightedly combining these loss terms, the model can balance the training objectives of different parts during training, thereby optimizing overall performance.

[0102] The overall part of the loss function is calculated as follows:

[0103]

[0104] in, is the total loss, The loss acting on the final fused prediction graph, is the loss of the k-th side output, w main and w side are the weighting coefficients of the main output and side output respectively.

[0105] Each loss or itself is a weighted combination of at least two loss functions selected from the following group:

[0106] Balance binary cross entropy loss, focal loss and Dice loss, the combination is:

[0107]

[0108] in, Reference or λ BCE ,λ Focal ,λ Dice Is the weight coefficient of each loss, balancing the binary cross entropy loss The class imbalance problem is alleviated by automatically calculating weights for positive samples (edge ​​pixels). The positive sample weight w pos The calculation formula is: where N neg and N pos are the total number of pixels of negative samples (background) and positive samples (edge) in the batch, respectively.

[0109] focal loss It aims to reduce the weight of easy-to-classify samples and make the model focus more on learning difficult-to-classify samples. The calculation formula is: Among them, pt is the model’s predicted probability for the correct category, α t is the balance factor and γ is the focusing parameter.

[0110] Dice loss The structural integrity of the edge is optimized by maximizing the overlap area between the prediction and the true value, and its calculation formula is: where p i and g i are the values ​​of the predicted probability map and the true label map at pixel i, respectively, and ∈ is a smoothing coefficient to prevent the denominator from being zero.

[0111] To further optimize model training, loss weights are dynamically adjusted during training. Specifically, a loss weight scheduler is used to adjust the weight of each loss term according to a preset scheduling strategy. When using a cosine annealing scheduling strategy, the model initially focuses on pixel-level balanced BCE loss and focal loss to quickly learn basic edge features. As training progresses, the weight of the Dice loss gradually increases, guiding the model to pay more attention to the structural integrity of edges.

[0112] The adjustment factor f of the loss weight cos Calculated by the following formula:

[0113]

[0114] Among them, total_epochs is the total number of rounds of model training, and current_epoch is the current training round number.

[0115] Then, according to the adjustment factor f cos Adjust the weight of each loss term. For example, the weight of BCE and Focal is the same as f cos positively correlated, and the Dice weight is (1-f cos ) is positively correlated.

[0116] This paper uses a comprehensive and scientific evaluation method to measure the performance of the model:

[0117] Evaluation metrics: Standard evaluation metrics in the field of edge detection are used, including the optimal dataset scale F1 score (ODS), the optimal image scale F1 score (OIS), and the average precision (AP).

[0118] Core evaluation method: fault-tolerant matching.

[0119] like Figure 6 As shown, unlike the traditional pixel-by-pixel precise matching, the present invention adopts an error-tolerant matching mechanism that is more in line with human visual perception.

[0120] Principle and implementation: First, the real binary edge map Mgt Perform distance transformation to obtain a distance map D gt , where the value of each non-edge pixel represents its Euclidean distance to the nearest edge pixel. For the predicted binary edge map M pred For each pixel p in D gt The value D of the corresponding position on gt If (p) is less than or equal to a preset error tolerance distance threshold τ (e.g., 2.0 pixels), the predicted pixel is considered a true positive.

[0121] Precision and Recall calculation:

[0122]

[0123] Among them, D pred It is for M pred The predicted distance map obtained after distance transformation.

[0124] This method more objectively evaluates the positioning accuracy of the model's predicted edges, avoids penalization for slight displacements, and thus more accurately reflects the model's true performance.

[0125] The present invention also provides a dual-branch edge detection system that deeply interacts and fuses CNN and Transformer.

[0126] like Figure 2 As shown, the core of the system provided by the present invention is a novel dual-branch edge detection model DualBranchEdgeModel; the model includes a CNN branch for extracting fine local features and a Transformer branch for capturing global context, and realizes deep interaction between the two branches through an innovative cross-attention module.

[0127] Branch A - Fine Grained Semantic Edge Branch (CNN branch): This branch is implemented by the Fine Grained Semantic EdgeBranch module, and its goal is to accurately extract local details and multi-scale features of the image.

[0128] Backbone network: Efficient Net-B2 is used as the backbone network for feature extraction. This network was chosen because it achieves a good balance between accuracy and efficiency.

[0129] Side output structure: Improved side output modules are connected to multiple intermediate feature layers of Efficient Net-B2 (for example, after the 1st, 2nd, 4th, 6th, and 7th main blocks). Each side output module independently processes the feature map of the current layer and generates an edge prediction map of the corresponding scale (in the form of logits, without sigmoid activation) for deep supervision during training.

[0130] Multi-scale feature fusion: The prediction maps output by each side are first upsampled to a uniform size and then weighted fused using a set of learnable weight parameters. The fused features are further processed by depthwise separable convolution to generate a 32-channel feature map multi_scale_features containing rich multi-scale information. This feature map will be used for subsequent cross-attention interaction.

[0131] Branch B-Global Context Branch (Transformer Branch): This branch is implemented by the Global Context Branch module, and its goal is to efficiently capture the long-range dependencies and global structural information of the image.

[0132] Lightweight Patch Embedding: The Lightweight Patch Embedding module converts the input image into a sequence of image patches. This module gradually reduces the spatial resolution and increases the channel dimension through a series of convolution operations with a stride of 2. This approach is more efficient than the single large-stride convolution used in ViT.

[0133] Efficient Local Attention: The core of the model consists of multiple stacked Lightweight Transformer Blocks. To reduce computational complexity, the self-attention mechanism uses Efficient Local Attention, which calculates self-attention within a non-overlapping local window (window_size, for example, 7x7) rather than performing global calculations on the entire feature map.

[0134] Learnable position encoding: A learnable position encoding parameter pos_embed is added to the image block sequence to enable the model to perceive the spatial position information of the image block.

[0135] Cross-attention interaction module: Figure 3 As shown in Figure 2, the CrossAttentionModule designed by this invention is the key to achieving deep interaction between the two branches. This module is called in the calculation process of the Transformer branch. Its calculation process can be summarized as the standard scaled dot product attention formula:

[0136]

[0137] Among them, query (Query, Q): the feature sequence of the image block after the Transformer branch itself is processed, and the linear projection (Q = feature trans W Q )get.

[0138] Key (Key, K) and Value (Value, V): Multi-scale feature map multi_scale_features after CNN branch fusion. The feature map is flattened and subjected to independent linear projection (K = feature cnn W K ,V=feature cnn W V ) get. d k is the dimension of the key vector.

[0139] This process effectively injects the rich and detailed texture and contour information of the CNN branch into the global context of the Transformer branch, achieving a deep fusion of the two modal features rather than a simple post-splicing.

[0140] The final feature fusion task is handled by the Memory Efficient Fusion module, and its detailed structure is as follows: Figure 4 As shown in the figure, this module receives a 32-channel feature map (fine semantic feature map) from the CNN branch and a 32-channel global context feature map from the Transformer branch after cross-attention enhancement. It deeply fuses the two features through a series of carefully designed processing stages, ultimately generating an edge prediction map (in the form of logits). Compared with conventional feature fusion methods, this module significantly reduces memory usage and computational costs through an optimized convolutional attention mechanism and a progressive channel dimensionality reduction strategy, while maintaining high-quality fusion results.

[0141] The workflow of the Memory Efficient Fusion module includes:

[0142] Feature projection stage: First, two independent 1×1 convolutional layers are used to project the features of the CNN branch and the Transformer branch, respectively, increasing their channel count from the original 32 to 64 channels, laying the foundation for subsequent deep fusion. At the same time, because the features of the two branches may have inconsistent spatial resolutions, the module performs bilinear interpolation on the feature map of the Transformer branch to keep its size consistent with the feature map of the CNN branch.

[0143] Feature connection stage: After completing the projection and size alignment, the two 64-channel feature maps are concatenated in the channel dimension to form a fused feature map with 128 channels (64 + 64). This simple concatenation operation retains all the original information of the two branches, but does not yet realize the interaction between them.

[0144] Convolutional attention stage: To achieve deep interaction and adaptive weighting of the two branch features, the module uses a convolution-based attention mechanism, which is one of the core innovations of this module. Specifically, the 128-channel spliced ​​features are first reduced to 32 channels through a 3×3 convolution layer (with batch normalization and ReLU activation), and then a 1×1 convolution and Sigmoid activation function are used to generate a 128-channel attention weight map. These weight maps are applied to the original spliced ​​features through element-by-element multiplication to achieve adaptive enhancement and suppression of features. Compared with the traditional multi-head attention mechanism, this convolutional attention significantly reduces computational complexity and memory requirements.

[0145] Feature fusion stage: After attention weighting, the module further fuses features through a two-layer convolutional network. The first layer of 3×3 convolution reduces the number of feature channels from 128 to 64, and the second layer of 3×3 convolution further reduces the number of channels to 32. Each convolution layer is followed by batch normalization and ReLU activation to enhance nonlinear expression capabilities and training stability. This progressive channel dimensionality reduction strategy avoids drastic compression of feature information and effectively preserves the key information required for edge detection.

[0146] Edge Refinement Stage: Finally, the module uses a three-layer edge refinement network to generate the final edge predictions. This network consists of two 3×3 convolutional layers and one 1×1 convolutional layer, gradually reducing the number of feature channels from 32 to 16, and then to 8, ultimately outputting a single-channel edge prediction map. The first two convolutional layers are equipped with batch normalization and ReLU activation, while the final convolutional layer directly outputs the raw logits values ​​without applying any activation function. This allows the output to work seamlessly with subsequent loss functions such as BCE With Logits Loss.

[0147] The design of the Memory Efficient Fusion module fully considers the balance between computational efficiency and fusion effect. By using a combination of 1×1 and 3×3 convolutions rather than larger convolution kernels, the module significantly reduces the number of parameters and computational complexity. At the same time, the introduction of the convolutional attention mechanism enables the module to adaptively focus on and integrate the most valuable features from both branches, further improving the accuracy and robustness of edge detection. Furthermore, the module outputs logits without activation function processing, which provides greater flexibility for model training and allows the threshold of edge detection results to be dynamically adjusted according to specific application scenarios during the inference phase.

[0148] To objectively and comprehensively verify the effectiveness and advancement of the method described in this paper, we conducted extensive experiments on the publicly available edge detection benchmark dataset BSDS500 and compared the performance of our method with existing advanced deep learning methods (HED and RCF). The evaluation metrics included the recognized optimal dataset-scale F1 score (ODS), optimal image-scale F1 score (OIS), average precision (AP), and the core metric for measuring model complexity—the number of model parameters. See Table 1 for specific data:

[0149] Table 1 Performance comparison between the method of the present invention and the prior art

[0150]

[0151]

[0152] The data of HED and RCF in the above table are respectively derived from their publicly published papers [1] and [2].

[0153] [1]Xie, S., & Tu, Z. (2015). Holistically-Nested Edge Detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).

[0154] [2]Liu,Y.,et al.(2017).Richer Convolutional Features for EdgeDetection.In Proceedings of the IEEE Conference on Computer Vision andPattern Recognition(CVPR).

[0155] The specific operations of the training and verification experiment of the dual-branch edge detection method provided by the present invention are as follows:

[0156] First, the data is preprocessed and enhanced. The system uniformly resizes the original input images to 260×260 pixels and performs 10x data augmentation on the training set. The data augmentation process is based on the Albumentations library and simultaneously performs a series of complex geometric and photometric transformations with a probability of 0.9, including random horizontal flips, angular rotations, elastic transformations, translation-scaling-rotation combinations, color jittering, CLAHE, random gamma transforms, Gaussian noise, etc. To filter out low-quality samples, the filter function min_edge_ratio is set to 0.002. After loading the annotation file, the ratio of edge pixels in each image is calculated. When it is lower than the preset filter function value, the sample is automatically removed to ensure the reliability and diversity of the training data.

[0157] Next, set the training strategy. This embodiment performs 50 training cycles (epochs), uses the AdamW optimizer, sets the initial learning rate to 1e-4, sets the weight decay to 1e-5, and uses the cosine annealing scheduler (CosineAnnealingLR) to smoothly adjust the learning rate, so that the model can achieve a more stable convergence effect in the later stages of training. Due to hardware memory limitations, the gradient accumulation technology is enabled at the same time, and the small batch updates of every 4 steps are accumulated once, which is equivalent to a batch size of 32, thereby improving the memory utilization efficiency while ensuring training stability.

[0158] The following conclusions can be drawn from the comparative data in Table 1, which fully demonstrates the disruptive advantages and beneficial effects of the present invention. Unlike methods such as RCF that rely on late feature splicing (late fusion), the present invention achieves early deep fusion of local and global information by introducing a cross-attention deep interaction mechanism in the middle layer of the model, thereby achieving improvements in all core accuracy indicators:

[0159] Detection accuracy is surpassed: Compared with the advanced method RCF based on pure CNN architecture, the present invention has achieved improvements in comparable core accuracy indicators. Among them, ODS increased from 0.806 to 0.813, and OIS increased from 0.823 to 0.833. In particular, in terms of average precision (AP), the present invention achieved 0.873 (according to the table, RCF does not provide AP indicator data). This directly proves the effectiveness of the cross-attention deep interaction mechanism created by the present invention, which enables the efficient fusion of the local fine features of the CNN branch and the global context information of the Transformer branch, thereby enhancing the model's ability to discriminate various types of edges.

[0160] A revolutionary breakthrough in model efficiency: While achieving higher accuracy, this invention utilizes only 8.73M model parameters, significantly lower than RCF's 15.6M. The model size of this invention is only about 56% of RCF's, with a parameter count reduction of nearly 44%. This significant efficiency advantage is due to the invention's comprehensive architectural optimization: the CNN branch utilizes the lightweight EfficientNet-B2 backbone, the Transformer branch employs an efficient local self-attention mechanism and a lightweight image block embedding module, and the final fusion stage utilizes a memory-efficient feature fusion module.

[0161] Achieving the technical goal of "higher accuracy, lower cost": Overall, this invention successfully breaks the conventional wisdom that "larger models equal better performance." Through a sophisticated design that deeply interacts with CNNs and Transformers, it achieves detection accuracy exceeding that of larger models with fewer model parameters and computing resources. This overcomes the high computational costs often associated with pursuing high accuracy in existing technologies, making it easier to deploy high-performance edge detection technology on resource-constrained platforms such as mobile devices and embedded devices, demonstrating its high practical value and broad application prospects.

[0162] like Figure 7 As shown in the figure, this example uses a representative image to illustrate the entire fault-tolerant matching evaluation process. The "Original image" shows the target to be detected and its complex background. The corresponding "True Edge" displays a manually annotated standard edge map, whose grayscale values ​​have been normalized to a range of 0–1, with the brightest point (1.000) in the image indicating the true edge location. The model's forward inference generates a "Prediction map," which reflects the model's confidence in the existence of an edge at each pixel, with the highest prediction value reaching 0.935. After binarizing the prediction map with a threshold of 0.55, the "Binarization result" displays the distribution of edge pixels determined by the model. Subsequently, in the "Fault-tolerant results" graph, the binarized results are matched against the ground-truth annotations with a 2-pixel fault tolerance. Red dots indicate correct predictions, blue dots indicate false detections, and unlabeled areas indicate missed detections. Finally, the "Distance Map" displays the Euclidean distance from each pixel to the nearest true edge in grayscale format. Regions with pixel values ​​≤ 2.0 are considered to be within the valid range for fault-tolerant matching. Through this series of visualizations, the positioning accuracy, false detection and missed detection of the model on this sample are intuitively verified.

[0163] In summary, the present invention proposes a dual-branch edge detection method based on deep interactive fusion of CNN and Transformer, including:

[0164] S1. Convolution-based fine semantic edge feature extraction: First, the input digital image is processed through a fine semantic edge branch based on a convolutional neural network (CNN) to extract multi-scale local features containing rich local texture and contour details; this branch uses EfficientNet-B2 as the backbone network to extract feature maps of different levels from the image. The output features of each feature level are connected to the side output module to perform deep supervision on the features of each level; the side output module enhances the features of each level through an improved side output structure and generates multiple edge prediction maps (in the form of logits).

[0165] S2. Transformer-based global context information modeling: In parallel with step S1, the input image is also processed by a global context branch based on a lightweight Transformer. First, the input image is divided into a series of non-overlapping image patches, and then serialized through multiple Transformer modules to capture long-distance dependencies between elements in the image and generate global context features. This process uses an efficient local attention mechanism to reduce computational costs while maintaining the ability to model global image information.

[0166] S3. Deep interaction of heterogeneous features based on cross-attention mechanism: A cross-attention module is used to achieve deep interaction between the local features extracted by the CNN branch and the global features generated by the Transformer branch. This process combines features from two different branches to achieve deep fusion and information injection at the feature level.

[0167] S4. Multi-source feature fusion and final edge prediction map generation: Through a memory-efficient feature fusion module, the global context features after deep interaction across attention modules are integrated with the fine semantic feature map output by the fine semantic edge branch. This fusion process generates the final edge prediction map through an edge refinement network. The prediction map is output in the form of logits and is not processed by the activation function, ensuring the flexibility of the prediction results so that they can be adjusted according to specific needs later.

[0168] S5. Model optimization and training based on a unified loss function: During the model training phase, a unified edge detection loss function consisting of a dynamically weighted combination of multiple loss components is used to jointly optimize the final edge prediction logarithmic map and multiple side output edge prediction maps. The loss function includes balanced binary cross entropy loss, focal loss, and Dice loss. The weights of the loss terms are dynamically adjusted through the softmax function to ensure that the model gradually converges to the optimal state during training.

[0169] In the present invention, terms such as "installed," "connected," "connected," and "fixed" should be understood broadly. For example, "connected" may refer to a fixed connection, a detachable connection, or an integral connection; and "connected" may refer to a direct connection or an indirect connection via an intermediary. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0170] The shapes of the various components in the drawings are schematic, and certain differences from their actual shapes are not excluded. The drawings are only used to illustrate the principles of the present invention and are not intended to limit the present invention.

[0171] Although the present invention has been disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely illustrative and are not intended to limit the application of the present invention. The scope of the present invention is defined by the appended claims and includes various modifications, variations, and equivalents made to the invention without departing from the scope and spirit of the present invention.

Claims

1. A dual-branch edge detection method based on deep interactive fusion of CNN and Transformer, characterized by: The method includes: S1. Convolution-based fine semantic edge feature extraction: The input digital image is processed by a fine semantic edge branch based on a convolutional neural network to extract multi-scale local features containing rich local texture and contour details. The fine semantic edge branch is connected to a side output structure after multiple different network layers to provide deep supervision of features at each layer and generate multiple side output edge prediction maps. S2, Transformer-based global context modeling: In parallel with step S1, the input image is processed through a global context branch based on a lightweight Transformer. This process first divides the image into a series of non-overlapping image blocks, which are then serialized through multiple Transformer modules to capture long-range dependencies between elements in the image and generate global context features. S3. Deep interaction of heterogeneous features based on cross-attention mechanism: Through a cross-attention module, deep interaction and information injection between the local features extracted by the fine semantic edge branch and the global features generated by the global context branch are achieved in the middle layer of the model. Specifically, the feature sequence generated by the global context branch is used as the query in the attention calculation, and the local features extracted by the fine semantic edge branch at at least one level are used as the key and value to perform scaled dot product attention calculation. S4. Multi-source feature fusion and final edge prediction map generation: A memory-efficient feature fusion module is used to integrate the global contextual features enhanced by the deep interaction of the cross-attention module with the fine semantic feature map output by the fine semantic edge branch. An edge refinement network is then used to generate the final edge prediction logarithmic map without activation function processing. S5. Model optimization and training based on a unified loss function: During the model training phase, a unified edge detection loss function consisting of a dynamically weighted combination of multiple loss components is used to jointly optimize the final edge prediction logarithmic map and the multiple side output edge prediction maps to guide the convergence of model parameters.

2. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer according to claim 1 is characterized in that The step S1 specifically includes: S11. First, the EfficientNet-B2 network is used as the backbone network for feature extraction. A series of multi-scale feature maps with different spatial resolutions and semantic levels are extracted from multiple preset intermediate feature layers of the backbone network. S12. Each extracted multi-scale feature map is input into a side output module, which includes a feature refinement network, a spatial attention network, and an edge prediction head. The feature refinement network first further refines each multi-scale feature map. The spatial attention network emphasizes important areas in the image and suppresses unimportant areas, thereby improving the expressiveness of the feature map. The edge prediction head generates a final edge prediction map, the output of which is a logarithmic map without Sigmoid activation function processing. Each multi-scale feature map corresponds to a prediction result, providing the model with multiple edge prediction maps. S13. After completing the processing of the above multi-scale features, the logarithmic images generated by all side output modules will be upsampled to the same size as the input image through bilinear interpolation. Each upsampled logarithmic image will be weighted one by one according to a set of learnable fusion weight parameters. The weight parameters are normalized by the softmax function, and the kth normalized weight is weight k The calculation method is: Where N is the number of side output modules, N ≥ k ≥ 1, w j is the weight parameter corresponding to the j-th side output feature map, which is not normalized; all weighted side output feature maps are finally spliced ​​in the channel dimension to form a feature tensor that integrates multi-scale information; S14. Finally, all weighted logarithmic maps are fused in the channel dimension to obtain a fused feature map containing rich multi-scale information. The fused feature map is input into a fused convolutional network composed of depthwise separable convolution and standard convolution for further feature purification and dimensionality reduction, and finally a 32-channel feature map is generated, which is used for subsequent cross-attention interaction.

3. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer according to claim 1 is characterized in that The step S2 specifically includes: S21, Embed Lightweight Image Blocks: The input image is processed into multiple image blocks. Specifically, a lightweight image block embedding module converts the input image from pixel space into a one-dimensional, sequential image block embedding sequence. The module uses a series of convolutional layers with a stride of 2 to gradually reduce the spatial resolution and increase the channel dimension. S22. Efficient Local Self-Attention Calculation: The core of the Transformer module uses an efficient local self-attention mechanism. In this mechanism, the image feature map is divided into multiple non-overlapping local windows. Within each local window, multi-head self-attention is independently calculated to capture long-range dependencies within the window. S23, Learnable Position Encoding Injection: Add a learnable position encoding parameter matrix to the image block embedding sequence where N p is the number of image blocks, and D is the embedding dimension; this position encoding enables the model to perceive the spatial position information of each image block in the original image, and its injection method is element-by-element addition.

4. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer according to claim 1 is characterized in that The step S3 comprises: S31. Generation of query, key and value: First, in the cross-attention module, the interaction of the two branch features is realized by generating three matrices: query, key and value. Specifically, the global context features from the Transformer branch are used as query. These features are first processed by multiple Transformer modules to generate a feature sequence of the image block. Where B represents the batch size, N p is the number of image blocks, D q is the embedding dimension of the query, and then, through a linear projection layer W Q Transform these features to obtain the final query matrix Q = F trans W Q ; Next, the 32-channel fine semantic feature map F from the fine semantic edge branch cnn ∈R B×32×H×W Flatten and transpose to obtain the serialized feature F ′ cnn ∈R B×(H·W)×32 , these features are passed through two independent linear projection layers W K and W V Transformation, respectively generate the key matrix K = F ′ cnn W K Sum value matrix V = F ′ cnn W V ; S32, scaled dot product attention calculation: After completing the query, key and value generation, multi-head scaled dot product attention calculation is performed. The core calculation formula of this process is: Among them, Q is the query matrix, K is the key matrix, V is the value matrix, d k is the dimension of the key vector; through this formula, the similarity between the query and the key is calculated by dot product and normalized by the softmax function to obtain the attention weights, which are then used to weight the value matrix to generate the final attention output features; The attention calculation process is multi-headed, that is, the process is executed in parallel on multiple attention heads, and the multiple results are concatenated and integrated through a linear projection layer to generate the final cross-attention feature sequence; S33. Feature output after interaction: Finally, the deep interaction features calculated by the multi-head attention mechanism will be output through a linear projection layer, and the obtained feature sequence will be used for subsequent feature fusion and edge prediction.

5. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer according to claim 1 is characterized in that The overall part of the unified edge detection loss function in step S5 is as follows: in, is the total loss, The loss acting on the final fused prediction graph, is the loss of the k-th side output, w main and w side are the weighting coefficients of the main output and side output respectively.

6. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer according to claim 5, characterized in that: Loss Item as well as They are all weighted combinations of at least two of the following three loss functions: balanced binary cross entropy loss, focal loss, and Dice loss. The combination is: in, Reference or λ BCE ,λ Focal ,λ Dice Is the weight coefficient of each loss. The relevant calculation methods of the three loss functions include: Balanced Binary Cross Entropy Loss The class imbalance problem is alleviated by automatically calculating weights for positive samples, and the positive sample weight w pos The calculation formula is: where N neg and N pos are the total number of pixels of negative and positive samples in the batch; focal loss Reduce the weight of easy-to-classify samples so that the model can focus more on learning difficult-to-classify samples. The calculation formula is: Among them, p t is the model’s predicted probability for the correct category, α t is the balance factor, γ is the focusing parameter; Dice loss The structural integrity of the edge is optimized by maximizing the overlap area between the prediction and the true value, and its calculation formula is: where p i and g i are the values ​​of the predicted probability map and the true label map at pixel i, respectively, and ∈ is a smoothing coefficient to prevent the denominator from being zero.

7. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer according to claim 6 is characterized in that: The loss weights are dynamically adjusted during training, including: The loss weight scheduler adjusts the weight of each loss term according to a preset scheduling strategy. When using the cosine annealing scheduling strategy, the model initially focuses on balancing the pixel-level BCE loss and focal loss to quickly learn basic edge features. As training progresses, the weight of the Dice loss gradually increases, guiding the model to pay more attention to the structural integrity of edges. The adjustment factor f of the loss weight cos Calculated by the following formula: Among them, total_epochs is the total number of rounds of model training, current_epoch is the current training round number; then, according to the adjustment factor f cos Adjust the weight of each loss term.

8. A detection system using the dual-branch edge detection method based on deep interactive fusion of CNN and Transformer as described in any one of claims 1 to 7, characterized in that: The system comprises: A global context module that processes the input image into blocks and extracts global context features containing long-range dependencies; A cross-attention interaction module that uses the features generated by the global context module as queries and the features generated by the fine semantic edge module as keys and values, enabling deep interaction and enhancement of features from both modules; A feature fusion module that integrates the enhanced features processed by the cross-attention interaction module with the fine semantic feature map generated by the fine semantic edge module and generates the final edge prediction result; A training module to train the system end-to-end.

Images