A CNN and transformer deep interaction fusion double-branch edge detection method and system

Abstract

The application discloses a kind of based on CNN and the deep interaction fusion double-branch edge detection method and system of Transformer, it is related to the technical field of computer vision and deep learning, the method includes: through a fine semantic edge branch based on CNN, utilize EfficientNet-B2 backbone network to extract multi-scale local features, and combine side output structure to carry out depth supervision;At the same time, through a global context branch based on light-weighted Transformer, high-efficiency local attention mechanism is used to capture the long-distance dependence of image;The core of the application lies in, a cross attention module is designed and realized, the local feature extracted by CNN branch is as key Key and value Value, the feature of Transformer branch is as query Query, so as to realize the deep interaction and fusion of two branches in feature level, rather than simple feature splicing;In addition, the application also adopts a unified loss function combined dynamically by balance binary cross entropy loss, focal loss and Dice loss etc. to optimize model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and deep learning, and in particular, to a CNN and Transformer deep interaction fusion double-branch edge detection method and system. BACKGROUND

[0002] Image edge detection is a fundamental and critical task in the field of computer vision, aiming to identify and locate the contours of object boundaries and scene structures in images. Accurate edge information is an important basis for subsequent advanced vision tasks such as image segmentation, object recognition, and three-dimensional reconstruction.

[0003] Traditional edge detection algorithms, such as Sobel and Canny, mainly rely on image gradient information, although they are simple to calculate, but when facing complex textures, weak edges and noise interference, their detection effect is often unsatisfactory. In recent years, with the development of deep learning technology, methods based on convolutional neural networks (CNN), such as HED (Holistically-Nested Edge Detection), have made significant breakthroughs in edge detection tasks by learning image features in an end-to-end manner. CNNs can effectively extract local texture and detail features of images using their inherent translation invariance and local receptive fields.

[0004] However, the locality of CNNs limits their ability to capture long-range dependencies and understand global context. To overcome this limitation, researchers have begun to introduce the Transformer model, which has achieved great success in natural language processing. Visual Transformer (ViT) and its variants can effectively establish dependencies between any two pixels in an image through self-attention mechanisms, thereby obtaining stronger global representation capabilities.

[0005] Therefore, combining the local feature extraction advantages of CNNs and the global context modeling capabilities of Transformers has become an important research direction in the field of edge detection. Current mainstream hybrid models mostly use simple fusion strategies, such as concatenating or adding feature maps extracted by each branch at the end of the two independent branches, which is a kind of "late fusion". This approach combines features from both branches, but the two branches are independent in feature extraction, lacking deep and real-time information interaction, which prevents global information from effectively guiding local feature extraction and vice versa. This insufficient fusion limits the further improvement of model performance, especially in challenging scenarios with complex backgrounds, varying target sizes, and occlusions, detection accuracy and robustness still need to be improved. SUMMARY

[0006] The present application aims to solve the problem of insufficient feature interaction and shallow fusion level of the CNN and Transformer hybrid model in the edge detection task in the prior art, thereby providing a new technical solution capable of realizing deep fusion of local details and global context, significantly improving edge detection accuracy and robustness.

[0007] The technical solution of the present application is to provide a CNN and Transformer deep interaction and fusion double-branch edge detection method based on the CNN and Transformer, which comprises:

[0008] S1, fine semantic edge feature extraction based on convolution: a fine semantic edge branch based on a convolutional neural network is used to process the input digital image to extract multi-scale local features containing rich local texture and contour details; the fine semantic edge branch is connected with a side output structure after multiple different network levels for deep supervision of the features of each level and generation of multiple side output edge prediction maps;

[0009] S2, global context information modeling based on Transformer: in parallel with step S1, a global context branch based on a lightweight Transformer is used to process the input image; the image is first divided into a series of non-overlapping image blocks, and then a sequence processing is performed through multiple Transformer modules to capture the long-distance dependency relationship between elements in the image to generate global context features;

[0010] S3, heterogeneous feature deep interaction based on cross-attention mechanism: a cross-attention module is used to realize deep interaction and information injection of the local features extracted by the fine semantic edge branch and the global features generated by the global context branch in the middle layer of the model; specifically, the feature sequence generated by the global context branch is used as the query Query in the attention calculation, and the local features extracted by the fine semantic edge branch at least one level are used as the key Key and the value Value for scaled dot product attention calculation;

[0011] S4, multi-source feature fusion and final edge prediction map generation: a memory-efficient feature fusion module is used to integrate the global context features enhanced by the cross-attention module and the fine semantic feature maps output by the fine semantic edge branch, and an edge refinement network is used to generate the final edge prediction log map without activation function processing;

[0012] S5, model optimization and training based on a unified loss function: in the model training stage, a unified edge detection loss function composed of multiple loss components dynamically weighted and combined is adopted to jointly optimize the final edge prediction log map and the multiple side output edge prediction maps to guide the convergence of the model parameters.

[0013] In any of the technical solutions above, further, step S1 specifically comprises:

[0014] S11, first, EfficientNet-B2 network is adopted as the backbone network for feature extraction, and a series of multi-scale feature maps with different spatial resolutions and semantic levels are extracted from multiple preset intermediate feature layers of the backbone network;

[0015] S12, each multi-scale feature map extracted is input into a side output module, which includes a feature refinement network, a spatial attention network, and an edge prediction head; the feature refinement network first performs further fine processing on each multi-scale feature map; the spatial attention network emphasizes important regions and suppresses unimportant regions in the image, improving the expression ability of the feature map; the edge prediction head generates the final edge prediction map, whose output is a log map without Sigmoid activation function processing; each multi-scale feature map corresponds to a prediction result, providing multiple edge prediction maps for the model;

[0016] S13, after the processing of the multi-scale features is completed, the log maps generated by all side output modules are upsampled to the same size as the input image through bilinear interpolation, and each upsampled log map is weighted one by one according to a set of learnable fusion weight parameters, the kth normalized weight weight k The calculation method is:

[0017]

[0018] where N is the number of side output modules, N >= k >= 1, w j is the weight parameter corresponding to the jth side output feature map, which is not normalized; all weighted side output feature maps are finally concatenated in the channel dimension to form a feature tensor that fuses multi-scale information;

[0019] S14, finally, all weighted log maps are fused in the channel dimension to obtain a fusion feature map containing rich multi-scale information, which is input into a fusion convolution network composed of depth separable convolution and standard convolution for further feature purification and dimension reduction, and finally a 32-channel feature map is generated, which is used for subsequent cross-attention interaction.

[0020] In any of the above technical solutions, step S2 further includes:

[0021] S21. Embedding lightweight image patches: The input image is processed into multiple image patches. Specifically, a lightweight image patch embedding module converts the input image from pixel space into a one-dimensional, serialized image patch embedding sequence. This module gradually reduces the spatial resolution and increases the channel dimension through a series of convolutional layers with a stride of 2.

[0022] S22. Efficient Local Self-Attention Calculation: The core of the Transformer module adopts an efficient local self-attention mechanism. In this mechanism, the image feature map is divided into multiple non-overlapping local windows. Within each local window, multi-head self-attention is calculated independently to capture long-distance dependencies within the window.

[0023] S23. Learnable positional encoding injection: A learnable positional encoding parameter matrix is ​​added to the image patch embedding sequence. Where N p Where is the number of image patches and D is the embedding dimension; this positional encoding enables the model to perceive the spatial location information of each image patch in the original image, and its injection method is element-wise addition.

[0024] In any of the above technical solutions, step S3 further includes:

[0025] S31. Generation of Query, Key, and Value: First, in the cross-attention module, the interaction between the two branches' features is achieved by generating three matrices: Query, Key, and Value. Specifically, the global context features from the Transformer branch are used as the Query. These features are first processed by multiple Transformer modules to generate feature sequences for image patches. Where B represents the batch size, N p D represents the number of image patches. q The embedding dimension of the query is then passed through a linear projection layer W. Q These features are transformed to obtain the final query matrix Q = F. trans W Q ;

[0026] Next, the 32-channel fine semantic feature map F from the fine semantic edge branch. cnn ∈R B×32×H×W Flattening and transposing yields the serialized feature F′. cnn ∈R B×(H·W)×32 These features are achieved through two independent linear projection layers W K and WV transformations, respectively, to generate Key matrix K=F' cnn W K and Value matrix V=F' cnn W V ;

[0027] S32, scaled dot-product attention calculation: after the generation of queries, keys and values, multi-head scaled dot-product attention calculation is performed, and the core calculation formula of the process is:

[0028]

[0029] where Q is the query matrix, K is the key matrix, V is the value matrix, d k is the dimension of the key vector; through this formula, the similarity between the query and the key is calculated by dot product, and the attention weight is obtained by normalizing through the softmax function, and these weights are then used to weight the value matrix, thereby generating the final attention output feature;

[0030] The attention calculation process is multi-headed, that is, the process is executed in parallel on multiple attention heads, and the multiple results obtained are spliced and integrated through a linear projection layer, thereby generating the final cross-attention feature sequence;

[0031] S33, feature output after interaction: finally, the deep interaction features calculated by the multi-head attention mechanism are output through a linear projection layer, and the feature sequence obtained will be used for subsequent feature fusion and edge prediction.

[0032] In any of the above technical solutions, further, the overall part of the unified edge detection loss function in step S5 is as follows:

[0033]

[0034] where, is the total loss, the loss acting on the final fusion prediction map, is the loss of the kth side output, w main and w side are the weighting coefficients of the main output and the side output, respectively.

[0035] 6. The CNN and Transformer deep interaction fusion double-branch edge detection method according to claim 5, wherein the loss term and are both weighted combinations of at least two selected from the following three loss functions: balanced binary cross-entropy loss, focal loss and Dice loss, and the combination method is:

[0036]

[0037] wherein, refers to or λ BCE ,λ Focal ,λ Dice are weight coefficients of each loss, and the related calculation methods of the three loss functions include:

[0038] Balanced binary cross-entropy loss By automatically calculating the weight of positive samples to alleviate the class imbalance problem, the positive sample weight w pos is calculated as follows: wherein N neg and N pos are the total number of pixels of negative samples and positive samples in the batch, respectively;

[0039] Focus loss The weight of easy-to-classify samples is reduced, so that the model focuses more on learning difficult-to-classify samples, and the calculation formula is as follows: wherein p t is the prediction probability of the model for the correct class, a t is a balance factor, and g is a focusing parameter;

[0040] Dice loss The structural integrity of the edge is optimized by maximizing the overlap area between the prediction and the true value, and the calculation formula is as follows: wherein p i and g i are the values of the prediction probability map and the real label map at pixel i, respectively, and e is a smoothing coefficient to prevent the denominator from being zero.

[0041] In any of the above technical solutions, further, the loss weight is dynamically adjusted during the training process, and the specific content includes:

[0042] The loss weight scheduler is used to adjust the weight of each loss term according to the preset scheduling strategy; when the cosine annealing scheduling strategy is used, the model will focus on the pixel-level balanced BCE loss and the focus loss at the beginning of the training to quickly learn the basic edge features; as the training progresses, the weight of the Dice loss will gradually increase, so as to guide the model to pay more attention to the structural integrity of the edge;

[0043] The adjustment factor f cos of the loss weight is calculated by the following formula:

[0044]

[0045] Wherein, total_epochs is the total number of training rounds of the model, and current_epoch is the current training round number; then, according to the adjustment factor f cos The weights of the loss terms are adjusted.

[0046] Also provided is a detection system adopting the CNN and Transformer deep interaction and fusion double-branch edge detection method in any of the above technical solutions, the system comprising:

[0047] A global context module for block processing of an input image and extracting global context features containing long-distance dependencies;

[0048] A cross-attention interaction module for taking the features generated by the global context module as queries and the features generated by the fine semantic edge module as keys and values to realize deep interaction and enhancement of the features of the two modules;

[0049] A feature fusion module for integrating the enhanced features processed by the cross-attention interaction module and the fine semantic feature map generated by the fine semantic edge module and generating a final edge prediction result;

[0050] A training module for end-to-end training of the system.

[0051] The beneficial effects of the present application are:

[0052] The cross-attention module is innovatively designed, realizing deep interaction of CNN local features and Transformer global features in the middle layer of the model and avoiding simple post-processing. This deep interaction enables the global context to effectively guide the selection of local details, while the local details enrich the global representation, thereby significantly improving the detection accuracy of the model in complex background, weak edge and other challenging scenarios.

[0053] By using the lightweight EfficientNet-B2 in the CNN branch, applying the efficient local attention mechanism in the Transformer branch, and using the memory-efficient module in the fusion stage, the total parameter amount of the model is controlled at 8.73M, which is about 56% of the prior art (such as RCF). While significantly reducing the complexity of the model, the detection accuracy exceeds that of the prior art, achieving the goal of "higher accuracy, lower consumption".

[0054] The core innovation of the present application is to adopt a self-developed unified loss function Unified Edge Detection Loss, which combines balanced BCE, Focal and Dice loss, effectively solving the class imbalance problem between edge pixels and background pixels. The loss weight scheduler further optimizes the training process, enabling the model to focus on different learning goals at different training stages. In addition, powerful data augmentation strategies such as elastic transformation and CLAHE greatly improve the generalization ability and robustness of the model.

[0055] In the evaluation stage, the present application adopts a fault-tolerant matching mechanism based on distance transformation, allowing a certain pixel deviation (e.g. 2 pixels) between the predicted edge and the real edge. This approach is more consistent with the perception standards of the human eye than traditional pixel-by-pixel accurate matching. Therefore, the evaluation results (such as ODS, OIS, etc.) are more objective and have practical reference value. BRIEF DESCRIPTION OF DRAWINGS

[0056] The above and additional aspects of the present application will become apparent and more readily appreciated from the following description of embodiments, taken in conjunction with the following drawings of which:

[0057] Figure 1 is the overall process flow diagram of the CNN and Transformer deep interaction fusion dual branch edge detection method and system based on an embodiment of the present application;

[0058] Figure 2 is the dual branch edge detection model (Dual Branch Edge Model) structure diagram of the CNN and Transformer deep interaction fusion dual branch edge detection method and system based on an embodiment of the present application;

[0059] Figure 3 is the cross-attention module (Cross-Attention Module) structure diagram of the CNN and Transformer deep interaction fusion dual branch edge detection method and system based on an embodiment of the present application;

[0060] Figure 4 is the structure diagram of the Memory Efficient Fusion module of the CNN and Transformer deep interaction fusion dual branch edge detection method and system based on an embodiment of the present application;

[0061] Figure 5 is the structure diagram of the unified edge detection loss function (Unified Edge Detection Loss) of the CNN and Transformer deep interaction fusion dual branch edge detection method and system based on an embodiment of the present application;

[0062] Figure 6 is a tolerance matching principle diagram of a CNN and Transformer deep interactive fusion double-branch edge detection method and system according to an embodiment of the present application;

[0063] Figure 7 is a typical image showing the entire tolerance matching evaluation process of the CNN and Transformer deep interactive fusion double-branch edge detection method and system according to an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to enable a more clear understanding of the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0065] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below.

[0066] As shown in Figure 1 The present embodiment provides a CNN and Transformer deep interactive fusion double-branch edge detection method, which comprises:

[0067] S1, fine semantic edge feature extraction based on convolution: first, a fine semantic edge branch based on a convolutional neural network (CNN) is used to process the input digital image, and multi-scale local features containing rich local texture and contour details are extracted; the branch uses EfficientNet-B2 as the backbone network to extract feature maps of different levels from the image, and the output features of each feature level are connected with a side output module to perform deep supervision on the features of each level; the side output module enhances and processes the features of each level through an improved side output structure, and generates multiple edge prediction maps (in logits form).

[0068] Step S1 specifically comprises:

[0069] S11, first, the EfficientNet-B2 network is used as the backbone network for feature extraction, and a series of multi-scale feature maps with different spatial resolutions and semantic levels are extracted from a plurality of preset intermediate feature layers (specifically, the first, second, fourth, sixth, and seventh main blocks) of the backbone network; these feature maps represent different levels of detail information of the image, from low-level texture features to high-level semantic information, forming a rich multi-scale feature set.

[0070] S12, each multi-scale feature map extracted is input into an independent improved side output module, which includes a feature refinement network, a spatial attention network, and an edge prediction head; the feature refinement network first performs further fine processing on each multi-scale feature map to enhance its performance in details; the spatial attention network emphasizes important areas and suppresses unimportant areas in the image in an adaptive manner, thereby improving the expression ability of the feature map; the edge prediction head is responsible for generating the final edge prediction map, and its output is a logit without Sigmoid activation function processing; each multi-scale feature map corresponds to a prediction result, providing multiple edge prediction maps for the model.

[0071] S13, after the processing of the multi-scale features is completed, the logits generated by all side output modules are upsampled to the same size as the input image through bilinear interpolation, and each upsampled logit is weighted one by one according to a set of learnable fusion weight parameters, and the kth normalized weight weight k The calculation method is:

[0072]

[0073] where N is the number of side output modules, N≥k≥1, w j is the weight parameter corresponding to the jth side output feature map, which is not normalized.

[0074] All weighted side output feature maps are finally concatenated in the channel dimension to form a feature tensor that fuses multi-scale information.

[0075] S14, finally, all weighted logits are fused in the channel dimension to obtain a fusion feature map containing rich multi-scale information, in order to further enhance the expression ability of these fusion feature maps, the fusion feature map is input into a fusion convolution network composed of depth separable convolution and standard convolution for further feature purification and dimension reduction, and finally a 32-channel feature map is generated, which will be used for subsequent cross-attention interaction.

[0076] Through the above process, step S1 realizes the extraction of fine semantic edge features with multi-scale information and semantic hierarchy from the input image, and through the deep supervision and feature fusion mechanism, the expression ability of the local features is enhanced, laying a foundation for subsequent depth interaction and edge prediction.

[0077] S2, global context information modeling based on Transformer: in parallel with step S1, the input image is also processed through a global context branch based on a lightweight Transformer; first, the input image is divided into a series of non-overlapping image blocks (patches), and then a series of Transformer modules are used for sequential processing to capture the long-distance dependency relationship between elements in the image and generate global context features; this process uses an efficient local attention mechanism to reduce the computational cost while maintaining the modeling ability of the global information of the image.

[0078] The purpose of this branch is to extract the global context features of the image and capture the dependency relationship between distant pixels in the image, and step S2 specifically includes:

[0079] S21, embedding lightweight image blocks: the input image is processed into multiple image blocks, specifically, through a lightweight image block embedding module, the input image is converted from a pixel space to a one-dimensional, sequential image block embedding sequence, and the module gradually reduces the spatial resolution and increases the channel dimension through a series of convolution layers with a step size of 2.

[0080] Unlike the single large-step convolution used in traditional ViT (Vision Transformer) models, the method of the present application better preserves the local information of the image while reducing the dimension, thereby improving the computational efficiency and enhancing the performance of the model; in this way, the input image is effectively embedded in a feature space suitable for processing by the Transformer model.

[0081] S22, efficient local self-attention calculation: the core of the Transformer module adopts an efficient local self-attention mechanism, in which the image feature map is divided into multiple non-overlapping local windows (e.g. 7x7); within each local window, multi-head self-attention is calculated independently to capture the long-distance dependency relationship within the window.

[0082] Compared with the global self-attention mechanism, this local self-attention calculation greatly reduces the computational complexity and memory requirement, and the self-attention mechanism of each window effectively models the local features, enabling the model to learn rich context information in different local regions, thereby enhancing the global understanding ability of the image; this design not only improves the efficiency of the model, but also reduces the consumption of computing resources, making it more efficient when processing large-scale images.

[0083] S23, learnable position encoding injection: in the image block embedding sequence, the application injects a learnable position encoding parameter matrix to enable the model to perceive the spatial position information of each image block in the original image; the position encoding matrix is where N p is the number of image blocks, and D is the embedding dimension; the position encoding enables the model to learn the spatial relationship of the image blocks in the original image and inject it into the image block embedding sequence through element-wise addition.

[0084] This operation enables the model to process the position information in the image through the self-attention mechanism of the Transformer without traditional convolution operations, and fully considers the spatial relationship between image blocks when modeling the global context.

[0085] Through the above steps, the branch of step S2 not only efficiently captures the global structure of the image, but also maintains effective modeling of long-distance dependencies in the case of limited computing resources; the core goal of this step is to obtain the global context information of the image through efficient local self-attention mechanism and image block embedding technology, and provide strong support for the subsequent cross-attention interaction step.

[0086] S3, heterogeneous feature depth interaction based on cross-attention mechanism: the local features extracted by the CNN branch and the global features generated by the Transformer branch are deeply interacted through a cross-attention module; this process combines features from two different branches to achieve deep fusion and information injection at the feature level.

[0087] In traditional edge detection methods, local features and global features are usually fused through simple concatenation or addition operations, while the application enables these two features to interact at a deeper level through the innovative cross-attention mechanism, thereby improving the accuracy and robustness of edge detection.

[0088] Specifically, the global context features generated by the Transformer branch are taken as the query (Query), and the local features of the CNN branch are taken as the key (Key) and value (Value) for scaled dot product attention calculation; this process effectively injects global context information into local features, realizes deep fusion of local details and global information, and enhances the accuracy of edge detection; step S3 specifically includes:

[0089] S31, Query, Key and Value generation: First, in the cross-attention module, the interaction of the two branch features is realized by generating three matrices of Query, Key and Value. Specifically, the global context features from the Transformer branch are used as Query. These features are first processed by multiple Transformer modules to generate a feature sequence of the image block where B denotes the batch size, N p is the number of image blocks, D q is the embedding dimension of Query, then the features are transformed by a linear projection layer W Q to obtain the final Query matrix Q = F trans W Q .

[0090] Next, the 32-channel fine-grained semantic feature map F cnn ∈R B×32×H×W from the fine-grained semantic edge branch is flattened and transposed to obtain the serialized features F' cnn ∈R B×(H·W)×32 These features are transformed by two independent linear projection layers W K and W V to generate the Key matrix K = F' cnn W K and the Value matrix V = F' cnn W V .

[0091] S32, scaled dot-product attention calculation: After the generation of Query, Key and Value, the core operation of the cross-attention mechanism is further realized through multi-head scaled dot-product attention calculation. The core calculation formula of this process is:

[0092]

[0093] where Q is the Query matrix, K is the Key matrix, V is the Value matrix, d k is the dimension of the Key vector. Through this formula, the similarity between Query and Key is calculated by dot product and normalized by softmax function to obtain the attention weight. These weights are then used to weight the Value matrix to generate the final attention output features.

[0094] The attention calculation process is multi-headed, that is, the process is executed in parallel on multiple attention heads, and the obtained multiple results are concatenated and integrated by a linear projection layer to generate the final cross-attention feature sequence.

[0095] Through this process, the local features from the fine semantic edge branch are effectively injected into the global context features from the Transformer branch, achieving deep fusion of local information and global information; this cross-attention mechanism enables the global features to effectively guide the extraction of local features, while the local features provide detailed support for the global features, thereby improving the performance of the edge detection model.

[0096] S33, feature output after interaction: ultimately, the deep interaction features calculated through the multi-head attention mechanism are output through a linear projection layer, and the obtained feature sequence will be used for subsequent feature fusion and edge prediction; this deep interaction enhanced feature sequence not only contains local detailed information, but also fuses global context information, and can provide more comprehensive image understanding; the output features of the cross-attention module will provide support for subsequent steps, especially in the multi-source feature fusion stage, further improving the model's ability to recognize edges in complex scenes.

[0097] S4, multi-source feature fusion and final edge prediction map generation: through a memory-efficient feature fusion module, the global context features after deep interaction through the cross-attention module are integrated with the fine semantic feature maps output by the fine semantic edge branch. This fusion process generates the final edge prediction map through an edge refinement network. The prediction map is output in logits form and has not been processed by an activation function, ensuring the flexibility of the prediction result to allow for subsequent adjustments based on specific needs.

[0098] In this step, a memory-efficient feature fusion module is used to integrate the global context features after deep interaction enhancement through the cross-attention module with the fine semantic feature maps output by the fine semantic edge branch. The goal of this process is to fuse the feature information from the two branches to generate a more fine and accurate edge prediction map.

[0099] S5, model optimization and training based on unified loss function: in the model training stage, a unified edge detection loss function composed of multiple loss components dynamically weighted and combined is used to jointly optimize the final edge prediction logit map and multiple side output edge prediction maps. The loss function includes balanced binary cross-entropy loss, focal loss, and Dice loss. The weights of the loss terms are dynamically adjusted through the soft-max (softmax) function to ensure that the model gradually converges to the optimal state during the training process.

[0100] In step S5, the training and optimization of the model is accomplished by employing a unified edge detection loss function dynamically weighted combined by multiple loss components; the loss function is used to guide the training of the model, ensuring the joint optimization of the final edge prediction map and multiple side output edge prediction maps, thereby improving the accuracy and robustness of the model in the edge detection task.

[0101] The structure of the unified loss function includes multiple loss terms, each of which optimizes different parts of the model. By weighted combining these loss terms, the model can balance the training objectives of different parts during training, thereby optimizing the overall performance.

[0102] The overall part of the loss function is calculated as follows:

[0103]

[0104] wherein, is the total loss, is the loss acting on the final fusion prediction map, is the loss of the kth side output, w main and w side are the weighting coefficients of the main output and side output, respectively.

[0105] Each loss or is itself a weighted combination of at least two loss functions selected from the following group:

[0106] Balancing binary cross-entropy loss, focal loss and Dice loss, the combination is:

[0107]

[0108] wherein, denotes or λ BCE ,λ Focal ,λ Dice are the weight coefficients of each loss, balancing binary cross-entropy loss by automatically calculating the weight for positive samples (edge pixels) to alleviate the class imbalance problem, the positive sample weight w pos is calculated as: wherein N neg and N pos are the total number of pixels of negative samples (background) and positive samples (edge) in the batch, respectively.

[0109] Focal loss aims to reduce the weight of easy-to-classify samples, so that the model focuses more on learning difficult-to-classify samples, and its calculation formula is: wherein, pt is the predicted probability of the model for the correct class, and t is the balancing factor, and

[0110] Dice loss The structural integrity of the edge is optimized by maximizing the overlapping area between the prediction and the ground truth, which is calculated as follows: where p i and g i are the values of the predicted probability map and the ground truth label map at pixel i, respectively, and is a smoothing coefficient to prevent the denominator from being zero.

[0111] In order to further optimize the model training, the loss weight is dynamically adjusted during the training process. Specifically, the loss weight scheduler is used to adjust the weight of each loss term according to the preset scheduling strategy. When using the cosine annealing scheduling strategy, the model will focus on the pixel-level balanced BCE loss and focal loss at the beginning of the training to quickly learn the basic edge features. As the training progresses, the weight of the Dice loss will gradually increase, thereby guiding the model to pay more attention to the structural integrity of the edge.

[0112] The adjustment factor f cos of the loss weight is calculated as follows:

[0113]

[0114] where total_epochs is the total number of training rounds of the model, and current_epoch is the current training round number.

[0115] Then, the weights of each loss term are adjusted according to the adjustment factor f cos . For example, the weights of BCE and Focal are positively correlated with f cos , while the weight of Dice is positively correlated with (1-f cos ).

[0116] The present application adopts a comprehensive and scientific evaluation method to measure the performance of the model:

[0117] Evaluation index: The standard evaluation index in the field of edge detection is adopted, including optimal dataset scale F1 score (ODS), optimal image scale F1 score (OIS), and average precision (AP).

[0118] Core evaluation method: fault-tolerant matching.

[0119] As shown in Figure 6 , unlike the traditional pixel-by-pixel accurate matching, the present application adopts a fault-tolerant matching mechanism that is more consistent with human visual perception.

[0120] Principle and implementation: First, the real binary edge map Mgt Perform a distance transformation to obtain a distance map D. gt , where the value of each non-edge pixel represents its Euclidean distance to the nearest edge pixel. For the predicted binary edge map M pred For each pixel p in D, if it is in D gt The value D at the corresponding position gt If (p) is less than or equal to a preset tolerance distance threshold τ (e.g., 2.0 pixels), then the predicted pixel is considered a true positive.

[0121] Precision and Recall Calculations:

[0122]

[0123] Among them, D pred It is for M pred The predicted distance map obtained after distance transformation.

[0124] This method provides a more objective assessment of the model's edge localization accuracy, avoiding penalties for slight displacements and thus more accurately reflecting the model's true performance.

[0125] The present invention also provides a dual-branch edge detection system that deeply integrates CNN and Transformer.

[0126] like Figure 2 As shown, the core of the system provided by this invention is a novel dual-branch edge detection model, DualBranchEdgeModel; the model includes a CNN branch for extracting fine local features and a Transformer branch for capturing global context, and achieves deep interaction between the two branches through an innovative cross-attention module.

[0127] Branch A - Fine Grained Semantic Edge Branch (CNN Branch): This branch is implemented by the Fine Grained Semantic EdgeBranch module, and its goal is to accurately extract local details and multi-scale features of the image.

[0128] Backbone network: Efficient Net-B2 is used as the backbone network for feature extraction. This network was chosen because it achieves a good balance between accuracy and efficiency.

[0129] Side output structure: Improved Side Output modules are connected at multiple intermediate feature layers of Efficient Net-B2 (e.g., after the 1st, 2nd, 4th, 6th, 7th main block). Each side output module independently processes the feature maps of the current level and generates an edge prediction map (in logits form, without Sigmoid activation) for the corresponding scale, which is used for deep supervision during training.

[0130] Multi-scale feature fusion: The prediction maps of each side output are first upsampled to a unified size, and then fused by a set of learnable weight parameters. The fused features are further processed by a depthwise separable convolution to generate a 32-channel feature map multi_scale_features containing rich multi-scale information, which will be used for subsequent cross-attention interaction.

[0131] Branch B - Global Context Branch (Transformer branch): This branch is implemented by the Global Context Branch module, which aims to efficiently capture long-range dependencies and global structure information of the image.

[0132] Lightweight patch embedding: The input image is converted into a sequence of image patches using the Lightweight Patch Embedding module. This module gradually reduces the spatial resolution and increases the channel dimension through a series of convolution operations with a step size of 2. Compared to the single large-step convolution used in ViT, this approach is more efficient.

[0133] Efficient local attention: The core of the model is stacked with multiple Lightweight Transformer Blocks. To reduce computational complexity, the self-attention mechanism in these blocks uses Efficient Local Attention, which calculates self-attention within non-overlapping local windows (window_size, e.g., 7x7) rather than performing global calculation on the entire feature map.

[0134] Learnable position encoding: A learnable position encoding parameter pos_embed is added to the image patch sequence, allowing the model to perceive spatial position information of the image patches.

[0135] Cross-attention interaction module: As shown in Figure 3 , the CrossAttentionModule designed by the present application is the key to realizing the deep interaction of the two branches. This module is called in the calculation process of the Transformer branch. Its calculation process can be summarized as the standard scaled dot-product attention formula:

[0136]

[0137] where Query (Q) is the image block feature sequence processed by the Transformer branch itself, projected linearly (Q = feature trans ·W Q ).

[0138] Key (K) and Value (V) are the multi-scale feature maps multi_scale_features fused by the CNN branch. The feature maps are flattened and projected linearly (K = feature cnn ·W K , V = feature cnn ·W V ) to obtain. k is the dimension of the key vector.

[0139] This process effectively injects the rich and fine texture and contour information of the CNN branch into the global context of the Transformer branch, achieving deep fusion of the two modal features rather than simple post-processing splicing.

[0140] The final feature fusion task is handled by the Memory Efficient Fusion module, whose detailed structure is shown in Figure 4 The module receives the 32-channel feature map (fine semantic feature map) from the CNN branch and the 32-channel global context feature map from the Transformer branch after cross-attention enhancement, and through a series of carefully designed processing stages, the two features are deeply fused to finally generate the edge prediction map (in logits form). Compared with conventional feature fusion methods, this module significantly reduces memory usage and computational cost through optimized convolution attention mechanisms and progressive channel dimension reduction strategies, while maintaining high-quality fusion results.

[0141] The workflow of the Memory Efficient Fusion module includes:

[0142] Feature projection stage: First, two independent 1x1 convolutional layers are used to project the features of the CNN branch and the Transformer branch, respectively, to unify their channel numbers from the original 32 channels to 64 channels, laying the foundation for subsequent deep fusion. At the same time, since the features of the two branches may have inconsistent spatial resolutions, the module performs bilinear interpolation on the feature maps of the Transformer branch to make their sizes consistent with those of the CNN branch.

[0143] Feature concatenation stage: After projection and size alignment, the two 64-channel feature maps are concatenated in the channel dimension to form a fused feature map with 128 (64+64) channels. This simple concatenation operation preserves all the original information of the two branches, but has not yet realized their interaction.

[0144] Convolutional attention stage: To realize the deep interaction and adaptive weighting of the two branch features, the module uses a convolution-based attention mechanism, which is one of the core innovations of the module. Specifically, first, a 3x3 convolution layer (with batch normalization and ReLU activation) is used to reduce the 128-channel concatenated features to 32 channels, and then a 1x1 convolution and Sigmoid activation function are used to generate 128-channel attention weight maps. These weight maps are applied to the original concatenated features through element-wise multiplication, realizing adaptive enhancement and suppression of the features. Compared with the traditional multi-head attention mechanism, this convolutional attention greatly reduces the computational complexity and memory requirements.

[0145] Feature fusion stage: After attention weighting, the module further fuses the features through a double-layer convolutional network. The first 3x3 convolution reduces the feature channels from 128 to 64, and the second 3x3 convolution further reduces the channel number to 32. Batch normalization and ReLU activation functions are used after each convolution to enhance the non-linear expression ability and training stability. This progressive channel reduction strategy avoids the drastic compression of feature information, effectively preserving the key information needed for edge detection.

[0146] Edge refinement stage: Finally, the module uses a three-layer edge refinement network to generate the final edge prediction result. The network consists of two 3x3 convolution layers and a 1x1 convolution layer, gradually reducing the feature channels from 32 to 16, then to 8, and finally outputting a single-channel edge prediction map. The first two layers of convolution are equipped with batch normalization and ReLU activation, while the last layer of convolution directly outputs the original logits value without applying any activation function, which enables the output result to seamlessly cooperate with subsequent loss functions such as BCE WithLogits Loss.

[0147] The design of the Memory Efficient Fusion module fully considers the balance between computational efficiency and fusion effect. By using a combination of 1x1 and 3x3 convolutions instead of larger convolution kernels, the module significantly reduces the number of parameters and computational complexity. At the same time, the introduction of the convolution attention mechanism enables the module to adaptively focus on and integrate the most valuable features in the two branches, further improving the accuracy and robustness of edge detection. In addition, the module outputs logits in the form of logits without activation function processing, which provides greater flexibility for model training and allows the threshold of the edge detection result to be dynamically adjusted according to specific application scenarios during the inference stage.

[0148] To objectively and comprehensively verify the effectiveness and advancement of the method described in the present application, we conducted sufficient experiments on the publicly available edge detection benchmark dataset BSDS500 and compared the performance of the method of the present application with existing advanced deep learning methods (HED, RCF). The evaluation indicators include the most recognized optimal dataset scale F1 score (ODS), the optimal image scale F1 score (OIS), the average precision (AP), and the core indicator of model complexity - the number of model parameters. For specific data, please see Table 1 below:

[0149] Table 1 Performance comparison of the method of the present application and existing technologies

[0150]

[0151]

[0152] The data of HED and RCF in the above table are respectively derived from their published papers [1] and [2].

[0153] [1] Xie, S., & Tu, Z. (2015). Holistically-Nested Edge Detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).

[0154] [2] Liu, Y., et al. (2017). Richer Convolutional Features for Edge Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[0155] The training and verification experiment of the double-branch edge detection method provided by the application is specifically operated as follows:

[0156] First, the data is preprocessed and enhanced. The system uniformly adjusts the original input image to 260x260 pixels, and implements 10 times of data enhancement on the training set. The data enhancement process is based on the Albumentations library, and a series of complex geometric and photometric transformations are simultaneously performed with a probability of 0.9, including random horizontal flip, angle rotation, elastic transformation, translation-scaling-rotation combination, color jitter, CLAHE, random gamma transformation, Gaussian noise, etc. In order to filter low-quality samples, a filtering function min_edge_ratio=0.002 is set, and after loading the annotation file, the proportion of edge pixels in each image is calculated, and when it is lower than the preset filtering function value, the sample is automatically removed to ensure the reliability and diversity of the training data.

[0157] Next, the training strategy is set. In this embodiment, a total of 50 training cycles (epochs) are performed, the AdamW optimizer is used, the initial learning rate is set to 1e-4, the weight decay is set to 1e-5, and the cosine annealing scheduler (CosineAnnealingLR) is used to smoothly adjust the learning rate, so that the model can obtain more stable convergence effect in the later training period. Due to the limitation of hardware memory, the gradient accumulation technology is enabled at the same time, which accumulates every 4 steps of small batch update once, equivalent to a batch size of 32, thereby improving the memory utilization efficiency while ensuring the stability of the training.

[0158] From the comparison data in Table 1, the following conclusions can be drawn, which fully prove the revolutionary advantages and beneficial effects of the application. Unlike methods such as RCF that rely on late fusion, the cross-attention deep interaction mechanism introduced in the middle layer of the model in the application realizes early deep fusion of local and global information, thereby improving all core precision indicators:

[0159] Detection accuracy surpasses: Compared with the advanced method RCF based on pure CNN architecture, the application improves all comparable core precision indicators. Among them, ODS is improved from 0.806 to 0.813, and OIS is improved from 0.823 to 0.833. In particular, the average precision (AP) of the application reaches 0.873 (according to the table, RCF does not provide AP indicator data). This directly proves the effectiveness of the cross-attention deep interaction mechanism unique to the application, which enables efficient fusion of local fine features of the CNN branch and global context information of the Transformer branch, thereby enhancing the model's ability to distinguish various edges.

[0160] The model efficiency achieves a revolutionary breakthrough: the model parameter quantity of the application is only 8.73M while achieving higher accuracy, which is much lower than 15.6M of RCF. The model size of the application is only about 56% of RCF, and the parameter quantity is reduced by nearly 44%. This huge efficiency advantage is due to the overall optimization of the architecture of the application: the CNN branch adopts a lightweight EfficientNet-B2 backbone, the Transformer branch adopts an efficient local self-attention mechanism and a lightweight image block embedding module, and the final fusion stage adopts a memory-efficient feature fusion module.

[0161] The technical target of "higher accuracy, lower consumption" is achieved: in summary, the application successfully breaks the traditional cognition that "the larger the model, the better the performance". It obtains detection accuracy beyond larger models with fewer model parameters and computing resources through ingenious CNN and Transformer deep interaction design. This solves the pain point that existing technologies often bring high computing cost when pursuing high accuracy, enabling high-performance edge detection technology to be more easily deployed to mobile terminals, embedded devices and other resource-constrained platforms, and has extremely high practical value and broad application prospects.

[0162] As shown in Figure 7 , the embodiment selects a representative image to demonstrate the entire fault-tolerant matching evaluation process; in the "Original image", the target to be detected and its complex background can be seen; the corresponding "True Edge" shows the standard edge map annotated by humans, whose gray value has been normalized to 0-1, and the brightest place (1.000) in the figure identifies the true edge position. The "Prediction map" obtained by forward inference of the model reflects the confidence of the model on the existence of the edge at each pixel, with the highest prediction value being 0.935. After the prediction map is subjected to a thresholding operation with a threshold of 0.55, the distribution of edge pixels determined by the model can be seen in the "Binarization result". Subsequently, in the "Fault-tolerant results" figure, the binarization result is matched with the true annotation with a fault-tolerant distance of 2 pixels, the red points represent correct prediction, the blue points are false positives, and the unmarked area is a missed detection. Finally, the "Distance Map" displays the Euclidean distance of each pixel to the nearest true edge in the form of a gray image, where the area with a pixel value ≤2.0 is considered as the effective range of fault-tolerant matching. Through this series of visualization, the positioning accuracy, false positives and missed detections of the model on this sample are intuitively verified.

[0163] In summary, the application proposes a CNN and Transformer deep interaction fusion double-branch edge detection method, which includes:

[0164] S1, fine semantic edge feature extraction based on convolution: first, the input digital image is processed by a fine semantic edge branch based on a convolutional neural network (CNN) to extract multi-scale local features containing rich local texture and contour details; the branch uses EfficientNet-B2 as the backbone network to extract feature maps of different levels from the image, and the output features of each feature level are connected with the side output module to perform deep supervision on the features of each level; the side output module enhances the features of each level through an improved side output structure and generates multiple edge prediction maps (in logits form).

[0165] S2, global context information modeling based on Transformer: in parallel with step S1, the input image is also processed by a global context branch based on a lightweight Transformer; first, the input image is divided into a series of non-overlapping image patches (Patches), and then a series of Transformer modules are used for sequential processing to capture long-range dependencies between elements in the image and generate global context features; this process uses an efficient local attention mechanism to reduce computational cost while maintaining the ability to model global information of the image.

[0166] S3, heterogeneous feature deep interaction based on cross-attention mechanism: a cross-attention module is used to realize the deep interaction between the local features extracted by the CNN branch and the global features generated by the Transformer branch; this process combines features from the two different branches to achieve deep fusion and information injection at the feature level.

[0167] S4, multi-source feature fusion and final edge prediction map generation: a memory-efficient feature fusion module is used to integrate the global context features after deep interaction by the cross-attention module and the fine semantic feature maps output by the fine semantic edge branch; the fusion process generates the final edge prediction map through an edge refinement network, which is output in logits form and has not been processed by an activation function, ensuring the flexibility of the prediction result to allow subsequent adjustments according to specific requirements.

[0168] S5, model optimization and training based on unified loss function: during model training, a unified edge detection loss function composed of multiple loss components dynamically weighted is used to jointly optimize the final edge prediction logit and multiple side output edge prediction maps; the loss function includes balanced binary cross-entropy loss, focal loss and Dice loss, and the weights of the loss terms are dynamically adjusted by a soft-max (softmax) function to ensure that the model gradually converges to an optimal state during training.

[0169] In the present application, the terms "mounting", "connecting", "connecting", "fixing" and the like should be understood in a broad sense, for example, "connecting" can be fixed connection, or detachable connection, or integrally connected; "connecting" can be directly connected, or indirectly connected through an intermediate medium. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0170] The shape of each component in the drawings is schematic, and there may be some difference from the actual shape. The drawings are only used to illustrate the principles of the present application, and are not intended to limit the present application.

[0171] Although the present application is disclosed in detail with reference to the drawings, it should be understood that the description is only exemplary and is not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims, and can include various modifications, improvements and equivalent schemes made to the application without departing from the scope and spirit of the present application.

Claims

1. A dual-branch edge detection method based on deep interactive fusion of CNN and Transformer, characterized in that, The method includes: S1. Convolution-based fine semantic edge feature extraction: Through a fine semantic edge branch based on a convolutional neural network, multi-scale local features of the input digital image are extracted; the fine semantic edge branch is connected to the side output structure after multiple different network layers to generate multiple side output edge prediction maps; S2. Global context information modeling based on Transformer: In parallel with S1, the input image is divided into a series of non-overlapping image patches through a global context branch based on a lightweight Transformer. The global context features are generated by serialization through multiple Transformer modules. S3. Deep interaction of heterogeneous features based on cross-attention mechanism: Through a cross-attention module, the feature sequence generated by the global context branch is used as the query in the attention calculation, and the local features extracted by the fine semantic edge branch at at least one level are used as the key and value for scaling dot product attention calculation. S4. Multi-source feature fusion and final edge prediction map generation: The global context features generated in S2 are integrated with the fine semantic feature map output by S1 after deep interactive enhancement in S3 through a feature fusion module, and an edge prediction logarithmic map is generated through an edge refinement network. S5. Model optimization and training based on a unified loss function: During the model training phase, a unified edge detection loss function, which is dynamically weighted and combined from multiple loss components, is used to jointly optimize the logarithmic edge prediction map and multiple side-output edge prediction maps. S1 specifically includes: S11. Using EfficientNet-B2 network as the backbone network for feature extraction, a series of multi-scale feature maps with different spatial resolutions and semantic levels are extracted from multiple preset intermediate feature layers of the backbone network. S12. Each extracted multi-scale feature map is input into a side output module. The side output module contains a feature refinement network, a spatial attention network, and an edge prediction head. The feature refinement network further refines each multi-scale feature map. The spatial attention network emphasizes important regions in the image and suppresses unimportant regions. The edge prediction head generates the final edge prediction map. Each multi-scale feature map corresponds to a prediction result. S13. The edge prediction maps generated by all side output modules are upsampled to the same size as the input image through bilinear interpolation. Each upsampled logarithmic map is weighted according to a set of learnable fusion weight parameters, which are normalized by the softmax function. S14. Fuse all weighted logarithmic maps along the channel dimension to obtain a fused feature map. Input the fused feature map into a fused convolutional network consisting of depthwise separable convolutions and standard convolutions to finally generate a 32-channel feature map.

2. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer as described in claim 1, characterized in that, The normalized weights calculated using the softmax function in step S13 are as follows: ； in, For the first Each normalized weight, It is the number of side output modules. , It corresponds to the first The weight parameters of each side output feature map are not normalized; all weighted side output feature maps are finally concatenated along the channel dimension to form a feature tensor that integrates multi-scale information.

3. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer as described in claim 1, characterized in that, Step S2 specifically includes: S21. Embedding lightweight image patches: The input image is processed into multiple image patches. Specifically, a lightweight image patch embedding module converts the input image from pixel space into a one-dimensional, serialized image patch embedding sequence. This module gradually reduces the spatial resolution and increases the channel dimension through a series of convolutional layers with a stride of 2. S22. Efficient Local Self-Attention Calculation: The core of the Transformer module adopts an efficient local self-attention mechanism. In this mechanism, the image feature map is divided into multiple non-overlapping local windows. Within each local window, multi-head self-attention is calculated independently to capture long-distance dependencies within the window. S23. Learnable positional encoding injection: A learnable positional encoding parameter matrix is ​​added to the image patch embedding sequence. ,in The number of image patches, For the embedding dimension; this positional encoding enables the model to perceive the spatial location information of each image patch in the original image, and its injection method is element-wise addition.

4. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer as described in claim 1, characterized in that, Step S3 includes: S31. Generation of Query, Key, and Value: First, in the cross-attention module, the interaction between the two branches' features is achieved by generating three matrices: Query, Key, and Value. Specifically, the global context features from the Transformer branch are used as the Query. These features are first processed by multiple Transformer modules to generate feature sequences for image patches. ,in Indicates batch size, The number of image patches, The embedding dimension of the query is then passed through a linear projection layer. These features are transformed to obtain the final query matrix. ; Next, 32-channel fine semantic feature maps from the fine semantic edge branches. Flattening and transposing yields serialized features. These features are achieved through two independent linear projection layers. and Transformations are performed to generate key matrices. Value matrix ; S32. Scaling Dot Product Attention Calculation: After completing the generation of the query, key, and value, multi-head scaling dot product attention calculation is performed. The core calculation formula for this process is: ； in, For querying the matrix, The key matrix, For value matrices, This is the dimension of the key vector; using this formula, the similarity between the query and the key is calculated by dot product and normalized by the softmax function to obtain attention weights. These weights are then used to weight the value matrix to generate the final attention output features. The attention computation process is multi-headed, meaning it is executed in parallel on multiple attention heads. The multiple results are concatenated and integrated through a linear projection layer to generate the final cross-attention feature sequence. S33. Feature output after interaction: Finally, the deep interaction features calculated through the multi-head attention mechanism will be output through a linear projection layer. The resulting feature sequence will be used for subsequent feature fusion and edge prediction.

5. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer as described in claim 1, characterized in that, The overall part of the unified edge detection loss function described in step S5 is shown below: ； in, For the total loss, The loss applied to the final fused prediction graph, For the first The loss of output on each side and These are the weighting coefficients for the main output and the side output, respectively.

6. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer as described in claim 5, characterized in that, Loss Items as well as Each loss function is a weighted combination of at least two of the following three loss functions: "balanced binary cross-entropy loss, focus loss, and Dice loss". The combination method is as follows: ； in, Reference or , These are the weighting coefficients for each loss term. The calculation methods for the three loss functions include: Balanced binary cross-entropy loss To mitigate class imbalance, positive samples are automatically weighted. The calculation formula is: ,in and These represent the total number of pixels in the negative and positive samples within the batch, respectively. Focus loss The formula for reducing the weight of easily classified samples is as follows: ,in, It is the model's predicted probability for the correct category. It is a balancing factor. It is the focusing parameter; Dice loss The structural integrity of the edges is optimized by maximizing the overlap between the prediction and the true value. The calculation formula is as follows: ,in and These are the predicted probability map and the true label map at the pixel level. The value at that location, It is a smoothing coefficient to prevent the denominator from being zero.

7. The dual-branch edge detection method based on deep interactive fusion of CNN and Transformer as described in claim 6, characterized in that, The loss weights are dynamically adjusted during training, including the following: The loss weight scheduler adjusts the weight of each loss term according to the preset scheduling strategy. When using the cosine annealing scheduling strategy, in the early stage of training, the model will focus on the pixel-level balanced BCE loss and focus loss to quickly learn basic edge features. As training progresses, the weight of the Dice loss will gradually increase, thereby guiding the model to pay more attention to the structural integrity of the edges. Adjustment factor for loss weights Calculated using the following formula: ； in, This represents the total number of rounds of model training. Number the current training epoch; then, based on the adjustment factor... Adjust the weights of each loss item.

8. A detection system employing the dual-branch edge detection method based on deep interactive fusion of CNN and Transformer as described in any one of claims 1 to 7, characterized in that, The system includes: A global context module is used to segment the input image into blocks and extract global context features containing long-range dependencies; A cross-attention interaction module is used to use features generated by the global context module as queries and features generated by the fine semantic edge module as keys and values, enabling deep interaction and enhancement of features from the two modules. A feature fusion module integrates the enhanced features processed by the cross-attention interaction module with the fine semantic feature map generated by the fine semantic edge module, and generates the final edge prediction result; A training module that performs end-to-end training on the system.

Images