A text detection method based on boundary adaptation from the perspective of a drone

Abstract

This invention relates to the field of UAV-based text detection technology, and discloses a boundary-adaptive UAV-based text detection model. First, ResNet50 is used as the backbone network, and a hybrid text attention mechanism is proposed and introduced into the feature extraction module to enhance the perception of text regions. Furthermore, this invention proposes a spatial feature fusion module for adaptively fusing text features at different scales, thereby enhancing the model's adaptability. Finally, this invention introduces a local feature extractor into the Transformer of the text detail boundary iterative optimization module, achieving accurate optimization and localization of text boundaries under complex background interference, avoiding complex post-processing steps. Extensive experiments on challenging text detection datasets and UAV-based text detection datasets validate the high robustness and state-of-the-art performance of our proposed method, laying a solid foundation for practical applications.

Description

Technical Field

[0001] This invention relates to the field of text detection technology from the perspective of unmanned aerial vehicles (UAVs), and in particular to a text detection model from the perspective of UAVs based on boundary adaptation. Background Technology

[0002] With the rapid development of drone technology, drones are increasingly widely used in daily life and professional fields, including military reconnaissance, environmental monitoring, logistics, urban planning, and disaster relief. Text detection and recognition in drone images are crucial for enhancing drones' environmental perception capabilities. This is because text often contains important geographical location information, facility names, directional signs, and advertising content. Extracting this textual information can significantly enhance drones' environmental perception capabilities in complex environments, leading to a better understanding of their surroundings and improved accuracy in autonomous decision-making and path planning. Furthermore, it helps in analyzing commercial layouts and the distribution of street advertisements in communities, which is beneficial for the development of smart cities. However, aerial images taken from a drone's perspective present challenges such as complex background information, diverse text shapes and orientations, small-scale text, and occlusion, making text detection tasks particularly difficult.

[0003] Existing text detection technologies primarily target images captured in natural scenes. However, images taken by drones present unique challenges and requirements. First, drones typically capture images from an aerial perspective, resulting in text appearing in various shapes and orientations. Second, drones often shoot at high altitudes, leading to smaller text sizes and significant background interference. Furthermore, due to the dynamic nature of drone flight, the same text area may be captured from multiple angles. Finally, variations in lighting caused by different times and weather conditions can result in shadows and highlights in the images, further increasing the complexity of text detection. Therefore, developing a text detection model specifically designed for drone perspectives is crucial. By introducing boundary adaptation techniques, the model's detection capabilities in complex scenes are effectively enhanced, meeting the needs of practical applications.

[0004] Current popular text detection methods can be broadly categorized into two types: regression-based methods and segmentation-based methods. Regression-based methods predict the boundary coordinates of text boxes without requiring additional post-processing, thus improving computational efficiency and demonstrating good performance across various text formats. However, these methods exhibit instability when handling small-scale and dense text, and struggle to achieve satisfactory results when dealing with text containing complex background information.

[0005] Segmentation-based text detection methods, such as Pan and DBNet, utilize pixel-level text region masks, typically offering higher accuracy than regression-based methods. Because they can capture the geometry of the text, these methods perform better when handling text of various shapes and orientations. Furthermore, segmentation methods can divide text regions into blocks, making them better suited for handling dense text and complex backgrounds. However, these methods often require complex post-processing steps to assign pixel groups to text regions, necessitating substantial amounts of labeled data and computational resources.

[0006] These methods typically rely on convolutional neural networks (CNNs), but CNNs often neglect long-range dependencies and global spatial relationships between text elements, making them sensitive to noisy regions within the text. However, in text detection tasks from a drone's perspective, text can appear at various scales, orientations, and shapes, and is affected by complex background interference. Therefore, global features and long-range dependencies are crucial for accurate text detection. Furthermore, commonly used CNN backbones, such as ResNet and VGG, provide coarse-grained, high-resolution features, which are useful for large-scale text detection but not for detecting small-scale text instances.

[0007] Therefore, it is necessary to provide a boundary-adaptive UAV-based text detection model to solve the above-mentioned technical problems. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a text detection model based on boundary adaptation from a UAV perspective.

[0009] The boundary-adaptive UAV-based text detection model provided by this invention comprises the following components:

[0010] A hybrid text attention mechanism (HTAM) is used to enhance the ability to perceive text regions during the feature extraction stage;

[0011] A spatial feature fusion module (SFFM) is used to adaptively fuse text features at different scales, effectively integrating low-level and high-level semantic output features to enhance the representational power of features, enrich semantic information, and ultimately improve the model's understanding of image content and perception of text at different scales.

[0012] A Text Detail Transformer (TDT) includes a Local Feature Extractor (LFE) for optimizing the iterative refinement process of text boundaries.

[0013] Preferably, the Hybrid Text Attention Mechanism (HTAM) is divided into two parts: a channel attention mechanism and a spatial attention mechanism, in order to reduce detection omissions caused by changes in viewpoint, lighting and shadows, and occlusion.

[0014] Preferably, the spatial feature fusion module (SFFM) achieves the fusion of high-level and low-level features through weighted feature maps, thereby enhancing the model's ability to detect text at different scales.

[0015] Preferably, the Text Details Transformer (TDT) improves the model's ability to extract local information from the feature map by introducing a Local Feature Extractor (LFE) in the Transformer block, thereby optimizing the refinement of text boundaries.

[0016] Preferably, the channel attention mechanism introduces an efficient channel attention aggregation mechanism (A-ECA): by performing max pooling on the input feature map, detailed texture features related to text boundaries are extracted, thereby enhancing the perception of text boundaries; at the same time, average pooling is used to capture the overall information of the image region, which facilitates the understanding of the overall image structure and background, all of which are contextual features related to the target; therefore, these two processing methods are applied to the input feature map simultaneously; after processing, the two generated feature maps are concatenated, and the spatial information of the feature maps is aggregated by merging the average pooling features and the max pooling features.

[0017] Preferably, the channel attention mechanism also introduces a local cross-channel interaction strategy and an adaptive one-dimensional convolutional structure to achieve more comprehensive cross-channel information exchange. Through network learning, it obtains different weights corresponding to different channels on the feature map, thereby providing more accurate attention information along the channel dimension.

[0018] This invention proposes a method for UAV-view text detection using the aforementioned boundary-adaptive UAV-view text detection model. The method includes the following steps:

[0019] Image features are extracted using a hybrid text attention mechanism (HTAM).

[0020] Multi-scale features are fused using the Spatial Feature Fusion Module (SFFM);

[0021] The text boundaries are iteratively optimized using the Text Details Transformer (TDT) to improve detection accuracy.

[0022] Compared with related technologies, the UAV-based text detection model based on boundary adaptation provided by this invention has the following advantages:

[0023] This invention innovatively creates a dataset for text detection from a drone's perspective.

[0024] The Hybrid Text Attention (HTAM) mechanism proposed in this invention improves the accuracy of the model under complex backgrounds, diverse perspectives, and low contrast conditions by targeting text features from the perspective of drones. This method improves training convergence and model robustness, and significantly enhances text detection performance from the perspective of drones.

[0025] To address the common problem of small-scale text in UAV-perspective text detection, this invention proposes an innovative Spatial Feature Fusion (SFFM) module. By integrating feature maps from different receptive fields, the method of this invention demonstrates significant effectiveness on UAV-perspective text detection datasets containing a large number of small-scale text instances.

[0026] By introducing a Local Feature Extractor (LFE) into the Transformer-based iterative boundary optimization module, this invention enhances its ability to extract local information from feature maps. This improvement reduces interference from complex backgrounds in UAV-view images and significantly improves the effectiveness of text boundary optimization.

[0027] Extensive experiments on private and public drone datasets demonstrate that the method of this invention achieves advanced performance and efficiency, and shows promising application prospects. Attached Figure Description

[0028] Figure 1 The following is an explanation of how the text detail boundary iteration optimization module is used in this invention to refine coarse text boundaries: (a) The backbone network and boundary proposal module generate coarse text boundaries; (b) The text detail boundary iteration optimization module is used to iteratively refine the coarse text boundaries to obtain a fine text boundary map.

[0029] Figure 2 This is a framework diagram of the UAV-based text detection model based on boundary adaptation proposed in this invention.

[0030] Figure 3 This is the backbone network diagram used for feature extraction;

[0031] Figure 4 Here is a structural diagram of HTAM;

[0032] Figure 5 Here is a structural diagram of SFFM;

[0033] Figure 6 This is a diagram of the encoder structure of TDT. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0035] This paper proposes the DADNet framework for text detection from a UAV perspective, fully leveraging the advantages of Transformer and CNN. Specifically, our model consists of three main components: a feature extraction backbone network, a coarse bounding box generation module, and a boundary refinement module in the boundary iterative optimization module. DADNet improves model performance in the following three aspects:

[0036] (1) A hybrid text attention mechanism (HTAM) is introduced, which combines spatial and channel attention mechanisms to give the model position awareness and feature selection capabilities. This mechanism reduces detection omissions caused by changes in viewpoint, lighting, shadows and occlusion, and enhances the robustness of the model and its ability to perceive text features.

[0037] (2) To address the issue of textual information at different scales in UAV-view images, a Spatial Feature Fusion (SFFM) module was designed. This module integrates features from low-level language output and high-level semantic output, achieving the fusion of high-level and low-level features. The weighted features calculated by this module are similar to a spatial attention mechanism, which facilitates feature selection and integration.

[0038] (3) A Text Details Transformer (TDT) was designed in the boundary iteration optimization module. By introducing a Local Feature Extractor (LFE) into the original Transformer block, the boundary information of the text region is further enhanced. Specifically, by using LFE to process the feature map to obtain local information, better text region boundary features can be obtained, thus positively impacting the boundary optimization of text features. Examples of results are shown below. Figure 1 As shown.

[0039] To validate the effectiveness of DADNet, we collected images from drone datasets (such as VisDrone2019 and UAVid), filtered for drone-view text data, annotated it, and created a drone text dataset for drone-view text detection. Furthermore, we evaluated the model's performance on challenging text detection datasets (such as Total-Text and CTW1500). Our model achieved state-of-the-art results on these datasets.

[0040] The UAV-based text detection model framework proposed in this invention is as follows: Figure 2 As shown, ResNet50 is used as the backbone network for feature extraction. To enhance the model's sensitivity to text features, a higher-order attention mechanism (HTAM) is introduced. Figure 3This section describes our backbone network. A spatial feature fusion module is employed based on a multi-layer feature fusion strategy to maintain spatial resolution and fully utilize information from multiple layers. The extracted features are then passed to a boundary proposal module, which generates coarse text boundary proposals for each text region. Each boundary proposal consists of N control points, representing a potential text instance. Utilizing this acquired prior information, a text boundary detail iterative module refines the text boundaries. Finally, accurate text boundaries are obtained.

[0041] The following sections will elaborate on the Hybrid Text Attention Mechanism (HTAM), the Spatial Feature Fusion Module (SFFM), and the Text Detail Transformer (TDT):

[0042] 1. Hybrid Text Attention Mechanism (HTAM):

[0043] Attention mechanisms are of great significance in deep learning. In computer vision tasks, by allowing the model to assign different attention weights to different regions of an image, the model can better focus on the regions of interest, thereby improving performance and generalization ability. This leads to improvements in performance and adaptability.

[0044] In text detection from a drone's perspective, the attention mechanism has demonstrated positive effects for the following reasons: First, the input data for this task typically includes complex natural scenes with significant interference, such as complex backgrounds, occlusions, noise, variations in natural lighting, and text viewed from different angles. By employing the attention mechanism, the model can selectively focus on important image regions related to the text, while ignoring other parts of the image. This improves the accuracy and robustness of the detection. Furthermore, text from a drone's perspective may exhibit significant variations in shape, size, font, and spatial arrangement of characters. The attention mechanism helps the model adapt to these variations, allowing it to better focus on and understand text regions.

[0045] like Figure 4As shown, the Hybrid Text Attention (HTAM) mechanism can be divided into two parts: channel attention and spatial attention. For the channel attention mechanism, an efficient channel attention aggregation mechanism (A-ECA) is introduced. By performing max pooling on the input feature map, detailed texture features related to text boundaries are extracted, thereby enhancing the perception of text boundaries. At the same time, average pooling is used to capture the overall information of the image region, facilitating the understanding of the overall image structure and background, which are contextual features related to the target. Therefore, these two processing methods are applied to the input feature map simultaneously. After processing, the two generated feature maps are concatenated, and the spatial information of the feature maps is aggregated by merging the average pooling features and max pooling features. In addition, a local cross-channel interaction strategy and an adaptive one-dimensional convolutional structure are introduced to achieve more comprehensive cross-channel information exchange. Through network learning, different weights corresponding to different channels on the feature map are obtained, thereby providing more accurate attention information along the channel dimension.

[0046] The specific implementation involves inputting feature maps. Decomposed along the channel dimension and Then, the decomposed feature maps are fed into the average pooling layer. and max pooling layer For average pooling layers, the average value within the pooling window is calculated, smoothing the feature map and highlighting global information. In contrast, max pooling layers select the maximum value within the pooling window, effectively extracting edges and other salient features. Subsequently, a feature concatenation operation merges the feature maps obtained from different pooling operations along the channel dimension, thereby integrating multiple feature representations and enhancing the model's feature expressive power. The following is... , And a formulaic representation of the feature map stitching process:

[0047]

[0048]

[0049]

[0050] In Formula 1, H and W represent the height and width of the feature map, respectively. and Represents the position in the feature map The pixel value, and This represents the connection operation. Finally, a local inter-channel interaction strategy and an adaptive one-dimensional convolutional structure are introduced. Through one-dimensional convolution, the model achieves comprehensive inter-channel information exchange, thereby enhancing the feature correlation between channels. The A-ECA calculation formula is as follows:

[0051]

[0052] In Formula 2, This represents the sigmoid function. Indicates average pooling. This indicates max pooling. Indicates inclusion A one-dimensional convolution with 3 parameters. Spatial Attention (SAM) introduces relevant parts into the bottleneck attention module, reducing spatial information loss caused by pooling layers. Pooling layers are removed to further preserve feature maps. To prevent a significant increase in parameters, we introduce depthwise separable convolution, dividing standard convolution into two steps: depthwise convolution and pointwise convolution, thereby reducing the computational complexity of the model and making the network more lightweight. This is particularly beneficial for complex models (such as attention mechanisms) and helps reduce the risk of overfitting. First, a 3×3 depthwise convolution is applied to capture the feature map. The feature map retains spatial information while preserving the relationships between channels, further enhancing the extraction of local image features. The processed feature map undergoes channel scaling via pointwise convolution. By reducing the number of channels, the computational complexity of the model is reduced, introducing lightweight feature representations and further refining features. Subsequently, another 3×3 depthwise convolution is applied to the feature map with reduced channels to further incorporate spatial relationships. Finally, pointwise convolution is used to restore the feature map. This convolutional layer corresponds to the second convolutional layer, restoring the channels. SAM achieves dynamic attention allocation in image space through channel reduction and restoration, as well as depthwise separable convolution. This operation helps the model focus more on text regions in the image, thereby improving model performance. The calculation formula for SAM is as follows:

[0053]

[0054] In Formula 3, This represents the output of channel convolution, where DW represents depthwise convolution and PW represents pointwise convolution.

[0055] The hybrid text attention mechanism combines channel attention and spatial attention. First, channel attention dynamically allocates attention across different channels, emphasizing task-specific channel information, thereby enhancing the model's abstraction and representation capabilities. This helps to better capture key features in text detection tasks, ultimately improving model performance. Second, spatial attention enables the model to focus on specific regions of the input tensor, allowing for more concentrated processing of relevant features. Through spatial attention, the model can effectively locate and identify text regions, improving the accuracy and robustness of text detection. By fully leveraging the advantages of both channel-level and spatial-level information, the hybrid text attention mechanism enables the model to better understand and process complex text images, thus improving the performance and effectiveness of text detection tasks. Figure 6This demonstrates a comparison of the effects of using a hybrid text attention mechanism versus a baseline heatmap.

[0056] 2. Spatial Feature Fusion Module (SFFM):

[0057] like Figure 5 As shown, when feature maps are input from the backbone network into the multi-level feature fusion structure, feature maps with smaller receptive fields are located at the bottom of the pyramid structure. During the backward fusion process, these feature maps with smaller receptive fields are ultimately fused. Furthermore, when the number of input channels is small, the contribution of feature information with smaller receptive fields to the fused feature information is relatively small, resulting in less processing of feature information with smaller receptive fields and reducing their influence in the final output. However, feature maps with larger receptive fields are located at the top of the pyramid. During the backward fusion process, the proportion of feature maps with larger receptive fields gradually decreases. This leads to insufficient attention paid to feature information with smaller receptive fields by the model, thus missing the detection of small-scale text in some images and failing to fully detect large-scale text in other images. Therefore, a spatial feature fusion module (SFFM) is introduced into the original multi-level feature fusion structure to appropriately fuse feature maps with smaller and larger receptive fields.

[0058] SFFM primarily aims to effectively integrate low-level and high-level semantic output features, thereby enhancing the representational power of features, enriching semantic information, and ultimately improving the model's understanding of image content and perception of text at different scales. Due to differences in attributes such as the number of channels and feature scale, simple weighting operations cannot be applied to the two sets of features. Therefore, the SFFM feature fusion module is used to fuse these two sets of features. Figure 5 As shown, the SFFM module accepts two inputs: one is low-level features. The other is advanced features. These two feature sets are first concatenated, then processed through simple convolution operations and activation functions. Next, a spatial attention mechanism is used to capture the correlations between different regions, effectively distinguishing text regions from background information in complex natural scenes. This module can better handle variations in text position and layout within images, ultimately obtaining a final feature representation that integrates information from different levels. The calculation formula is as follows:

[0059]

[0060]

[0061] In Equation 4, concat represents the concatenation operation; Conv represents the 3×3 convolution operation; Spatial attention refers to the spatial attention module, such as... Figure 5As shown, the spatial attention mechanism in SFFM can more effectively capture the correlation between different regions in the input feature map, thereby enhancing the model's perception of local structure. Based on the results of multi-layer feature fusion, high-level information is first fused with mid-level features in the original network, and then fused with low-level features to produce the final output. This design ensures that the SFFM module involves only a small number of convolutional operations and simple element-wise multiplication and addition operations without introducing additional computational overhead. Therefore, when processing text detection tasks from a UAV perspective, the model can extract text features at different scales more comprehensively and accurately. This method not only solves the performance problem but also avoids additional computational costs.

[0062] 3. Text Detail Transformer (TDT):

[0063] In previous studies, control points were primarily located at the edges of text instances. For example, in DPText-DETR, points were sampled along the outer boundary of each text region. However, these edge-sampled features often contain many background attributes, making it difficult to focus solely on the text. In subsequent TEXTBPN++, a boundary transformer module was proposed that iteratively predicts the offset of each vertex pointing to the text boundary based on a learned coarse boundary proposal. For each coarse boundary represented by a closed polygon, the multi-head attention mechanism in the boundary transformer compares global similarity, thus associating distant targets, but is relatively weak in capturing text boundaries and local structures. This weakness leads to poor adaptation to text boundaries during the coarse boundary proposal optimization in the boundary iteration optimization module. To address these challenges, we propose the Text Detail Transformer (TDT), which leverages the powerful local feature extraction capabilities of convolutional neural networks (CNNs) to construct a Local Feature Extractor (LFE) and integrates it into the transformer. Its detailed architecture is as follows: Figure 6 As shown, the input features are first split along the channel dimension, and then the split components are fed into the LFE and the global feature extractor, respectively. Here, the high-frequency mixer consists of grouped convolution operations and 1×1 convolution operations, while the global feature extractor is implemented through a multi-head attention mechanism. In terms of technical implementation, for the input feature map... It is decomposed along the channel dimension into and The formulas for the local feature extractor and the global feature extractor are as follows:

[0064]

[0065]

[0066] In Formula 5, represents a 3×3 grouped convolution, Conv represents a 1×1 convolution, and MSA refers to the multi-head self-attention mechanism in the transformer.

[0067] The boundary iteration optimization module adopts an encoder-decoder structure, where the encoder consists of three layers, our proposed Text Detail Transform (TDT) has residual connections, and the decoder is a simple multilayer perceptron (MLP), such as... Figure 6 As shown. Each encoder layer can be represented as:

[0068]

[0069] In Formula 6, The feature matrix represents the boundary proposal. Each Text Details Transformer (TDT) has a standard architecture, including parallel local feature extractors, global feature extractors, and a multilayer perceptron network (MLP).

[0070] The main advantage of the proposed TDT is its flexibility in optimizing text boundaries, enabling it to better handle ambiguous or irregular text boundaries, thereby improving the accuracy and robustness of text detection and recognition. This fusion method effectively integrates global and local information, utilizing long-distance dependencies and local details, overcoming the limitations of single methods and providing stronger feature representation capabilities for text processing tasks. It can better refine boundary information during text boundary optimization and reduce interference from background noise. Therefore, the final boundary contour can more accurately fit the text region.

[0071] The invention will be further illustrated by the following experiments:

[0072] (1) Dataset

[0073] Total-Text: This dataset contains 1,255 training images and 300 test images, providing character-level polygon annotations for text with different orientations and irregular shapes. The dataset covers a variety of scenes, including outdoor landscapes and buildings.

[0074] CTW1500: This dataset includes 1,500 natural scene images, with 1,000 used for training and 500 for testing. The dataset primarily features curved text and covers a variety of scenes, such as outdoor landscapes and city streets.

[0075] Drone-text: This drone-view text dataset contains 2,000 images, with 1,600 used for training and 400 for testing. The dataset originates from drone footage, encompassing text data from drone datasets such as Visdrone2019 and UAVid, showcasing various text images under different urban backgrounds, perspectives, and lighting conditions. These images include not only ground text but also text on shops and billboards. The dataset is labeled using PPOCRLabel and converted to the Icdar2015 text annotation format. Each line in the annotation file represents a text object; the first eight numbers are coordinate information (x1, y1, x2, y2, x3, y3, x4, y4), forming a polygon represented by four clockwise points. Unprocessed-img refers to the original image in the dataset, Annotated-img refers to the annotated image, and Label refers to the annotation file of the original data.

[0076] (2) Implementation details

[0077] In the experiments, the backbone network used ResNet50. The input image size was set to 640 × 640, and the model was trained over 660 epochs with a batch size of 12. The initial learning rate was set to 0.001, and no pre-trained model was used, decaying by 0.9 every 50 epochs. Adam was chosen as the optimizer. Data augmentation techniques such as random rotation, random flipping, and random cropping were applied. The experimental environment consisted of Python 3.8 and the PyTorch 1.7.0 framework. Training was performed on an NVIDIA RTX A6000 GPU with 48GB of memory, using an Intel(R) Xeon(R) Gold 6226R @ 2.9GHz CPU.

[0078] (3) Evaluation indicators

[0079] “R”, “P”, and “F” represent recall, precision, and F-measure, respectively. The performance of the algorithm is evaluated using precision, recall, and F-measure, which are calculated as follows:

[0080]

[0081]

[0082]

[0083] In this context, TP (True Positive) represents the true case, FP (False Positive) represents the false positive case, and FN (False Negative) represents the false negative case.

[0084] Table 1: DADNet ablation experiments on the Total-Text dataset: Hybrid text attention mechanism, spatial feature fusion module, and local feature extractor. "HTAM" represents the hybrid text attention mechanism, "SFFM" represents the spatial feature fusion module, and "LFE" represents the local feature extractor. "P", "R", and "F" correspond to precision, recall, and F-measure, respectively.

[0085]

[0086] Table 2: DADNet ablation experiments on the Drone-text dataset: Hybrid text attention mechanism, spatial feature fusion module, and local feature extractor. "HTAM" represents the hybrid text attention mechanism, "SFFM" represents the spatial feature fusion module, and "LFE" represents the local feature extractor. "P", "R", and "F" correspond to precision, recall, and F-measure, respectively.

[0087]

[0088] (4) Ablation research

[0089] We conducted ablation studies on the Total-Text and Drone-Text datasets to further validate the superior performance of text detection methods from the drone's perspective, as well as the effectiveness of HTAM, SFFM, and LFE. Detailed experimental results are shown in Tables 1 and 2.

[0090] As shown in Tables 1 and 2, the introduction of HTAM significantly improved the performance of both datasets. Specifically, HTAM improved the F-measure performance by 1.3% on the Total-Text dataset and by 0.6% on the Drone-text dataset. Building on this, combined with SFFM, this module can capture and appropriately fuse features at different levels, improving text information detection at various scales without significantly increasing computational overhead. On the Total-Text dataset, SFFM contributed a 1.6% improvement in F-measure. On the Drone-text dataset, the F-measure improvement was 2.0%. Finally, the introduction of LFE integrated all modules, achieving state-of-the-art results on both datasets. On the Total-Text dataset, precision improved by 2.7%, recall by 0.7%, and the F-measure reached 88.8%, representing a 1.7% improvement. On the Drone-text dataset, precision increased by 5.1%, recall increased by 5.6%, and the F-measure reached 76.3%, representing a 5.4% improvement.

[0091] The main function of LFE is to iteratively optimize text boundaries and reduce interference from complex backgrounds. As shown in Table 1-2, LFE improves performance on both the Total-Text and Drone-Text datasets. When LFE is introduced alone, the F-measure of both datasets improves by 0.7%, indicating its positive impact on boundary optimization. After introducing HTAM and SFFM, comparing the results with and without LFE, the Total-Text dataset shows an improvement of 0.1%, while the Drone-Text dataset shows an improvement of 3.6%. The Total-Text dataset mainly consists of routine text detection tasks, and the detection accuracy has been improved after introducing HTAM and SFFM. Since most of the text in this dataset is horizontal and unaffected by background and angle interference, boundary optimization without LFE still achieves good results. Conversely, the Drone-Text dataset contains multiple viewpoints and complex backgrounds. After HTAM and SFFM improve text detection capabilities, LFE significantly refines the coarse boundaries of the detected complex text, resulting in a significant performance improvement.

[0092] On the Total-Text dataset, the F-measure statistics of the baseline model and DADNet from epochs 300 to 660 were presented. Evaluation was performed every 5 epochs, generating a total of 72 data points. Subsequently, a frequency distribution histogram was created to analyze the model's evaluation results. It can be seen that integrating the various modules significantly improves the model's F-measure value.

[0093] (5) Comparisons with previous methods

[0094] To verify the generality of this method, we compared it with other state-of-the-art methods on three standard datasets. These datasets include two curve text benchmarks (Total-Text and CTW1500) and a dataset for detecting text from an aerial perspective. The visual results of DADNet on different datasets, along with detailed numerical results, are provided in Tables 3-5.

[0095] Table 3: Comparison of DADNet with other methods on the Total-Text dataset

[0096]

[0097] Total-Text: The Total-Text dataset is widely used in the field of arbitrary-shape text detection. It contains various text types, including multi-directional, horizontal, and curved text lines, making it ideal for validating our method's ability to detect arbitrary-shape text. When testing on this dataset, images were resized to the (640, 1024) range while maintaining their aspect ratio. Thresholds thd and this were set to 0.3 and 0.85, respectively. The results of our method on the Total-Text dataset are shown in the last column of Table 3. The comparison clearly shows that our method outperforms current mainstream models. Without using additional datasets, the F-measure reaches 88.8%, the highest performance achieved without external data. For example, our method outperforms the current best-performing MixNet (F-measure 87.8%) by 1.0%. Table 3 provides detailed numerical results.

[0098] Table 4: Comparison of DADNet with other methods on the CTW1500 dataset

[0099]

[0100] CTW1500: Compared to word-level annotations in Total-Text, curved text instances in CTW1500 are annotated using polygons with 14 vertices, presenting a greater challenge. Our method leverages SFFM and TDT to better capture long-range and local features to handle this type of situation. When testing on this dataset, images were also resized to the (640, 1024) range while maintaining their aspect ratio, with thresholds thd and this set to 0.3 and 0.855, respectively. The results are shown in Table 4. Clearly, our model achieves 88.3% accuracy, 83.9% recall, and 86.0% F-measure. Precision and F-measure are both state-of-the-art, while recall surpasses most recent models. Compared to the previous state-of-the-art method TextFuseNet, DADNet improves accuracy and F-measure by 3.3% and 0.6%, respectively. These results demonstrate the satisfactory performance of our DADNet.

[0101] Table 5: Comparison of DADNet with other methods on the Drone-Text dataset.

[0102]

[0103] Drone-text: To demonstrate the versatility of this method in drone-view scenarios, the model was trained using the Drone-text dataset. This dataset contains a variety of complex Chinese scenes, including text information such as shop signs, billboards, buildings, road signs, and traffic signs. Due to the aerial perspective of drone imagery, this dataset contains rich text information and identical text instances from different viewpoints. Furthermore, some text regions may be partially occluded or in shadow, leading to reduced model perceptibility, especially for small text instances. Additionally, the wide field of view provided by drones increases sensitivity to complex background interference. Therefore, this dataset effectively simulates the diverse scenarios encountered in drone-view settings, posing significant challenges to text detection.

[0104] We selected mainstream models including FAST, MixNet, DBNet++, TextPMs, and TextBPN++ for comparison with our model. FAST designs an ultra-simple kernel function to simulate text of arbitrary shapes. Meanwhile, the TextNet network is specifically designed for text detection. MixNet designs a novel text detection backbone network, FSNet, and employs a central transformer block to utilize the 1D manifold constraints of the scene text. TextPMs proposes an innovative probabilistic graph-based segmentation and detection method for accurate text instance detection. DBNet++ introduces an adaptive scale fusion module for scale-robust feature fusion and integrates the binarization process into the DB module of the segmentation network, optimizing both the segmentation network and the DB module to produce more accurate results. TextBPN++ systematically proposes a unified coarse-to-fine framework for detecting text of arbitrary shapes, enabling accurate and efficient text boundary localization without post-processing.

[0105] To validate the model's effectiveness, we performed detection on the Drone-text dataset using FAST, MixNet, DBNet++, TextPMs, and TextBPN++ algorithms. Detailed detection results are analyzed in Table 5. Our model achieved 80.0% precision, 73.0% recall, and 76.3% F-measure in text detection, representing state-of-the-art performance for text detection from a drone's perspective. Compared to MixNet, our model's F-measure is 4.4% higher, demonstrating significantly better performance in drone-based text detection.

[0106] Compared with related technologies, the UAV-based text detection model based on boundary adaptation provided by this invention has the following advantages:

[0107] This paper annotates a UAV-based text detection dataset and proposes a novel text detection model for UAV perspective. Our model makes three key improvements to the arbitrary-shape text detection framework: First, a hybrid text attention mechanism enhances the perception of text regions; second, a scale feature fusion module optimizes the processing of text features at different scales; and third, a text detection transformer reduces interference from complex backgrounds by integrating local features, achieving more accurate text boundary localization without complex post-processing. Extensive experiments demonstrate that our method, using a ResNet50 backbone, performs excellently on public datasets and UAV-based text detection datasets.

[0108] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1.A method for UAV perspective text detection based on boundary adaptation, characterized in that, The method comprises the following steps: S1, extracting image features by using a mixed text attention mechanism: the mixed text attention mechanism is used to strengthen the perception ability of the feature extraction stage to the text region, and the mixed text attention mechanism is divided into two parts of a channel attention mechanism and a spatial attention mechanism to reduce detection loss caused by changes in viewing angle, light and shadow and occlusion; the channel attention mechanism introduces a channel attention aggregation mechanism: detailed texture features related to the text boundary are extracted by performing a maximum pooling operation on the input feature map, thereby enhancing the perception of the text boundary; at the same time, average pooling is used to capture the overall information of the image region, which is convenient for understanding the overall image structure and background, and these are all context features related to the target; therefore, the two processing methods are applied to the input feature map at the same time; after processing, the two generated feature maps are connected, the spatial information of the aggregated feature map is aggregated by merging the average pooling feature and the maximum pooling feature; the channel attention mechanism also introduces a local cross-channel interaction strategy and an adaptive one-dimensional convolution structure to realize more comprehensive cross-channel information exchange, and through network learning, different weights corresponding to different channels on the feature map are obtained, thereby providing more accurate attention information along the channel dimension, and the calculation formula of A-ECA is as follows: ， In Equation 2, denotes a sigmoid function, denotes average pooling, denotes max pooling, denotes a one-dimensional convolution including parameters. S2, fusing multi-scale features through a spatial feature fusion module: the spatial feature fusion module is used for adaptively fusing text features of different scales, and is used for effectively integrating output features of low-level semantics and high-level semantics, so as to enhance the representation ability of features, enrich semantic information, and finally improve the understanding of the model on image content and the perception of different scale texts; the spatial feature fusion module accepts two inputs: one is low-level features , and the other is high-level features , the two parts of features are spliced first, and then processed through convolution operation and activation function; then, a spatial attention mechanism is used to capture the correlation between different regions, so as to effectively distinguish text regions and background information in a complex natural scene; the calculation formula is as follows: ， ， In the formula, concat represents a splicing operation; Conv represents a 3*3 convolution operation; and spatial attention refers to a spatial attention module; S3, iteratively optimize the text boundary using a text detail transformer to improve the accuracy of detection: the text detail transformer has a residual connection, including a local feature extractor, a global feature extractor in parallel, and a multi-layer perception network for the iterative refinement process of the text boundary optimization; first, the input features are split along the channel dimension, and then the split components are sent into the local feature extractor LFE and the global feature extractor respectively; here, the high-frequency mixer is composed of a grouping convolution operation and a 1x1 convolution operation, while the global feature extractor is implemented through a multi-head attention mechanism; for the input feature map , it is decomposed along the channel dimension into and ; the formulas of the local feature extractor and the global feature extractor are as follows: ， ， In the formula, denotes a 3x3 grouped convolution, Conv denotes a 1x1 convolution, and MSA refers to multi-head self-attention mechanism in the transformer. 2.The boundary-adaptive drone perspective text detection method of claim 1, wherein, The spatial feature fusion module realizes the fusion of high-level and low-level features through a weighted feature map, thereby enhancing the detection capability of the model for different scales of text. 3.The boundary-adaptive based drone perspective text detection method of claim 1, wherein, The text detail transformer improves the ability of the model to extract local information from the feature map by introducing a local feature extractor in the Transformer block, thereby optimizing the fine-grained text boundary.

Images