Yi character ancient book image detection method based on double-branch Transform

Abstract

The invention discloses a Yi character and ancient book image detection method based on a double-branch Transform. The method comprises the following steps: acquiring a Yi character and ancient book image data set; constructing a Yi-nationality character ancient book image detection model, wherein the Yi-nationality character ancient book image detection model comprises a feature extraction backbone network, a double-branch feature extraction and fusion module, a text kernel stretching module and a prediction and text post-processing module; training the Yi-nationality character ancient book image detection model based on the Yi-nationality character ancient book image data set; and obtaining a to-be-detected Yi character and ancient book image, inputting the to-be-detected Yi character and ancient book image into the trained Yi character and ancient book image detection model, and obtaining a Yi character and ancient book image detection model output as a detection result graph. The method has remarkable advantages in the aspects of robustness and detection precision in a complex scene, so that a new technical approach and possibility are provided for digital protection and intelligent recognition of Yi characters.

Description

Technical Field

[0001] This invention relates to the fields of image processing and optical character recognition, and more specifically, to a method for detecting images of ancient Yi script books based on a dual-branch Transformer. Background Technology

[0002] Current research on Yi script relies heavily on scanning, transcription, and manual translation. However, a large number of valuable documents have been lost due to their age, necessitating digital preservation efforts. Accurate detection of Yi script is fundamental to digitization, and its accuracy directly affects the effectiveness of subsequent identification and analysis.

[0003] Currently, text detection technology has made significant progress in processing common languages ​​such as Chinese and English. However, in Yi script detection, traditional text detection methods are difficult to apply directly due to the complexity of the characters, high similarity, irregular character spacing, and layout. Furthermore, the sheer number of Yi characters, the large intra-class differences compared to the small inter-class differences, and the frequent occurrence of connected characters and subtle variations further increase the difficulty of detection. At the same time, the lack of large-scale labeled datasets for Yi script presents an even greater challenge to detection methods that rely on deep learning model training.

[0004] In recent years, deep learning-based text detection methods have gradually become a research hotspot. Segmentation-based detection methods have the advantage of handling text of arbitrary shapes through pixel-level prediction, but they still have shortcomings when dealing with the extreme aspect ratios and dense arrangement of Yi characters. Therefore, it is urgent to design a detection method that can take into account both global semantics and local details, so as to improve the detection accuracy and robustness of Yi characters in complex scenarios, thereby laying the foundation for the digital and intelligent application of Yi characters. Summary of the Invention

[0005] In view of the shortcomings of existing technologies, this invention provides a method for detecting Yi script in ancient text images based on a dual-branch Transformer. The purpose of this invention is to overcome the problems of insufficient accuracy in complex scenes, incomplete detection of characters with extreme aspect ratios and densely arranged characters, and lack of large-scale labeled data support in existing methods for detecting Yi script in ancient text images. This invention can improve the detection accuracy of Yi script in complex backgrounds and diverse arrangement scenes. Specific technical solutions include: A dataset of images of ancient Yi script books was obtained. The dataset of images of ancient Yi script books includes clear images of ancient Yi script books and corresponding character annotation information groups. The character annotation information groups include text segmentation results completed by semi-automatic annotation and manual methods, as well as character positioning boxes. A model for detecting ancient Yi script texts is constructed. This model includes a text detection network based on a two-branch Transformer structure. The text detection network comprises a feature extraction backbone network, a two-branch feature extraction and fusion module, a text kernel stretching module, and prediction and text post-processing modules. Input images undergo multi-scale feature extraction via a feature extraction backbone composed of ResNet50 and a feature pyramid network. The two-branch feature extraction and fusion module is used to jointly model global semantics and local details to enhance feature representation capabilities. The text kernel stretching module adaptively adjusts the horizontal and vertical shrinkage ratios of characters to improve detection accuracy and kernel completeness for characters with extreme aspect ratios. The prediction and text post-processing modules generate probability maps and threshold maps, and combine differentiable binarization and post-processing strategies to restore character boundaries, thereby obtaining robust Yi script detection results. The Yi script ancient book image detection model was trained based on the Yi script ancient book image dataset; The image of the ancient Yi script book to be detected is obtained, and the image is input into the trained Yi script ancient script book image detection model. The character positioning boxes output by the model are used as the detection results.

[0006] Furthermore, the dual-branch feature extraction and fusion module includes a convolutional feedforward network and a context-aware local enhancement module; the convolutional feedforward network introduces a depthwise separable convolutional module into the traditional feedforward network structure, which is used to aggregate local spatial features while mapping channels; the context-aware local enhancement module is used to jointly represent local features and global features.

[0007] Furthermore, the convolutional feedforward network includes a first fully connected layer, a nonlinear activation layer, a depthwise separable convolutional layer, and a second fully connected layer connected in sequence. The depthwise separable convolutional layer is placed after the activation layer and is used to aggregate local spatial information during the feedforward process. The convolutional feedforward network includes two forms: one is an intra-stage convolutional feedforward network, which uses residual connections and the stride of the depthwise separable convolutional layer is 1 to maintain feature resolution; the other is a cross-stage convolutional feedforward network, in which a depthwise separable convolutional layer and a fully connected layer with a stride of 2 are set in its residual branch to complete feature downsampling and channel dimension adjustment, so as to realize feature connection between stages.

[0008] Furthermore, the context-aware local enhancement module includes global branches and local branches; The global branch is configured to: process the key vector obtained from the input feature mapping. Sum value vector Downsampling is performed to reduce feature resolution and computational complexity, thus reducing the query vector. The downsampled key vector and downsampled value vector are used for input attention calculation to obtain global context information.

[0009] In the formula, This indicates an operation that calculates the similarity between the query and the key, and then performs a weighted sum based on the value vector. This represents the average pooling operation, where the global branch is used to extract low-frequency global features and model long-distance dependencies.

[0010] Furthermore, the local branch is configured to: process the input features using an AttnConv structure based on convolutional attention to extract high-frequency local information; and process the input features... Perform a linear mapping to obtain the query vector. Key vector AND value vector The process is represented as follows:

[0011] In the formula, This represents a fully connected mapping; after a linear mapping is completed, the value vector is... Depthwise separable convolution operations are performed to aggregate local features with shared weights, resulting in an aggregated local representation. :

[0012] For query vector Key vector Combined to generate context-aware weights, using two depthwise separable convolutions. To each and Aggregate local information:

[0013] calculate and Element-wise multiplication and nonlinear mapping are performed, and then passed through a fully connected layer and a nonlinear activation function. The intermediate representation is obtained, and then passed through a fully connected layer and the hyperbolic tangent function. Normalization is performed to obtain context-aware attention weights. Its calculation is defined as:

[0014] In the formula, Represents element-wise product. Represents a non-linear activation function. The feature dimension is defined as follows: the attention weights are multiplied element-wise by the convolution transformation result of the value vector to obtain the local enhancement features. The calculation formula is as follows:

[0015] in, Represents a value vector The features after depthwise separable convolution operation, the It is used to characterize high-frequency local detail information and is fused with the global branch output as an enhanced local feature; Output features of the global branch Local branch output features Fusion involves concatenating the two outputs along the channel dimension and applying a fully connected layer along that channel dimension. The calculation formula is as follows:

[0016] In the formula, Indicates feature fusion, This indicates a fully connected layer.

[0017] Furthermore, the text kernel stretching module is configured to perform scale transformation on the text kernel using a non-uniform shrinkage strategy to improve detection accuracy and kernel integrity under extreme aspect ratio conditions. The process is as follows: In the vertical direction, according to the conventional shrinkage ratio The contraction occurs, with a smaller contraction ratio set in the horizontal direction, resulting in a contraction rate that is only a fraction of that in the vertical direction. ,in Indicates the stretch ratio, set This is used to ensure the separation of adjacent characters while avoiding excessive horizontal shrinkage; The text kernel stretching module calculates the shrinkage distance based on the Vatti cropping algorithm. Its formula is:

[0018] In the formula, Represents the area of ​​a text instance. Represents the perimeter of a text instance. For the shrinkage ratio; for Yi characters with significant differences in aspect ratio, set... To achieve the maximum safe contraction distance; During the recovery phase, the text kernel stretching module calculates the expansion distance. The calculation formula is as follows:

[0019] In the formula, Indicates the predicted area of ​​the core. Indicates the predicted perimeter of the core. Indicates the desqueezing ratio; sets the original text region by performing an expansion operation on the prediction kernel. It can maintain high recovery accuracy under text conditions with different aspect ratios.

[0020] The prediction and text post-processing module is configured to: take the feature map processed by the dual-branch feature extraction and fusion module as input, generate a prediction map and a threshold map, and calculate an approximate binary map using a differentiable binarization function.

[0021] In the formula, Represents pixels in an approximate binary image. Represents the pixels of the predicted image. This represents the threshold image pixels. The value range of this function is (0, 1). It can be fine-tuned and optimized during training, so that the network can learn a more reasonable binarization strategy. The prediction and text post-processing module utilizes the approximate binary image combined with the text kernel mapping generated by the text kernel stretching module during the inference phase to restore the text boundaries using a post-processing step. The post-processing step includes: dynamically adjusting the expansion distance of the text kernel using an iterative approach. , with initial expansion distance Starting from this point, continuously adjust until the desired result is achieved. This method can adaptively optimize the expansion ratio based on the aspect ratio of different texts, improving recovery accuracy. In practice, It is usually set to 0.5 or 1.

[0022] The loss function during training of the Yi script ancient book image detection model is: the loss function L consists of three weighted parts: shrinkage loss Threshold map loss and approximate binarization loss The weighting coefficient and The specific forms of the three losses, set at 5 and 10 respectively, are as follows:

[0023]

[0024]

[0025]

[0026] In the formula, Indicates the mask area. Let be the label of the i-th pixel. Let be the predicted probability of the i-th pixel. and These are the actual threshold and the predicted threshold, respectively. These are pixels in an approximate binary image.

[0027] The evaluation metric for training the Yi script ancient book image detection model is:

[0028]

[0029]

[0030] In the formula, Indicates the actual number of cases. Indicates the number of false positives. Represents the number of false negative examples.

[0031] Compared with the prior art, the present invention has the following advantages: This invention provides a method for detecting ancient Yi script images. By introducing depthwise separable convolution into a convolutional feedforward network and combining the global and local branches of a context-aware local enhancement module, it achieves joint modeling of global semantics and local details, thereby improving feature representation capabilities. By designing a text kernel stretching module and adopting non-uniform shrinkage and expansion strategies, it effectively solves the problems of incomplete and broken kernels for characters with extreme aspect ratios and dense arrangement, ensuring the integrity of the detection results. By predicting the probability map and threshold map output by the branch and combining differentiable binarization and post-processing strategies, it can achieve accurate restoration of character boundaries and maintain high detection accuracy and robustness in complex backgrounds. The invention brings innovative progress to the field of Yi script detection and digital protection, and promotes the development of intelligent recognition and inheritance of minority scripts. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is an overall flowchart of the Yi script ancient book image detection method based on dual-branch Transformer of the present invention.

[0034] Figure 2 This is a network architecture diagram of the Yi script ancient book image detection model in an embodiment of the present invention.

[0035] Figure 3 This is a model architecture diagram of the dual-branch feature extraction and fusion module in an embodiment of the present invention. Detailed Implementation

[0036] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0037] like Figure 1-3 As shown, this invention discloses a method for detecting ancient Yi script images based on a two-branch Transformer, characterized by the following steps: S1. Obtain a dataset of ancient Yi script images. The dataset includes clear images of ancient Yi script books and corresponding character annotation information groups. The images are derived from high-resolution scanning of ancient Yi documents and some manually collected sample data. The character annotation information groups are obtained through a combination of semi-automatic annotation and manual correction, and are used to provide the supervision information required for training and testing.

[0038] S2. Construct a Yi script ancient book image detection model. The Yi script ancient book image detection model includes a Yi script ancient book image detection network based on a two-branch Transformer structure. Its overall architecture consists of four modules: a feature extraction backbone network, a two-branch feature extraction and fusion module, a text kernel stretching module, and a prediction and text post-processing module.

[0039] Specifically, the input image undergoes multi-scale feature extraction via a feature extraction backbone composed of ResNet50 and a feature pyramid network. The feature map is then fed into a dual-branch feature extraction and fusion module to capture high-frequency and low-frequency information from the image, and outputs a feature map for use by the prediction module. In the prediction module, the feature map is further processed to obtain a probability map and an adaptive threshold map, and a predicted kernel mapping is obtained through differentiable binarization. This mapping, combined with the kernel mapping generated by the text kernel stretching module, is used to calculate the loss function. During the inference phase, the model directly extracts the text kernel from the predicted kernel mapping and efficiently and accurately recovers it through post-processing operations, thereby obtaining the Yi script detection result image.

[0040] Furthermore, the dual-branch feature extraction and fusion module mainly includes a convolutional feedforward network and a context-aware local enhancement module. The convolutional feedforward network is an improved module that introduces depthwise separable convolutions into the traditional feedforward network structure. This module can aggregate feature information from the local spatial neighborhood while completing channel mapping, thereby not only achieving nonlinear transformation of features in the channel dimension but also improving the model's ability to model local details and spatial structure. The context-aware local enhancement module is used to jointly represent local and global features.

[0041] The convolutional feedforward network scheme is as follows: To integrate local information into the feedforward network, a convolutional feedforward network replaces the traditional feedforward network. The main difference between the convolutional feedforward network and commonly used feedforward networks is that the convolutional feedforward network uses depthwise convolution after the activation function, which allows it to aggregate local information. This invention employs two types of convolutional feedforward networks. The first is an intra-level convolutional feedforward network that directly utilizes skip connections. The second is a convolutional feedforward network connecting two stages, in which depthwise separable convolutions and fully connected layers are used for downsampling and dimensionality expansion, respectively, in the skip connections.

[0042] Furthermore, the context-aware local enhancement module consists of a global branch and a local branch. The global branch downsamples the feature map and combines it with a self-attention mechanism to model long-distance dependencies, thereby extracting global semantic information of the image and improving the model's ability to perceive the overall structure. The local branch introduces convolution operations to aggregate local neighborhood features, thereby enhancing the model's ability to represent character edges and fine-grained strokes.

[0043] The global branching scheme is as follows: For the key vector obtained from the input feature mapping... Sum value vector Downsampling is performed to reduce feature resolution and computational complexity. Then, the query vector... The downsampled key vector and downsampled value vector are input into the attention calculation process to obtain global context information, defined as:

[0044] in This indicates an operation that calculates the similarity between the query and the key, and then performs a weighted sum based on the value vector. This indicates the average pooling operation.

[0045] The local branching scheme is as follows: Input features are processed using an AttnConv structure based on convolutional attention to extract high-frequency local information; the input features... Perform a linear mapping to obtain the query vector. Key vector AND value vector Defined as:

[0046] in This represents a fully connected mapping; after a linear mapping is completed, the value vector is... Depthwise separable convolution operations are performed to aggregate local features with shared weights, resulting in an aggregated local representation. Defined as:

[0047] Furthermore, regarding the query vector Key vector Combined to generate context-aware weights, using two depthwise separable convolutions. To each and The local information is aggregated and defined as follows:

[0048] Subsequently, calculation and Element-wise multiplication and nonlinear mapping are performed, and then passed through a fully connected layer and a nonlinear activation function. The intermediate representation is obtained, and then passed through a fully connected layer and the hyperbolic tangent function. Normalization is performed to obtain context-aware attention weights. Defined as:

[0049] in Represents element-wise product. Represents a non-linear activation function. The feature dimension is defined as follows: Finally, the attention weights are multiplied element-wise by the convolution transformation result of the value vector to obtain the local enhancement features. Defined as:

[0050] in Represents a value vector Features after depthwise separable convolution operation. It is used to characterize high-frequency local detail information and is fused with the global branch output as an enhanced local feature.

[0051] Finally, output features from the global branch. Local branch output features The two outputs are combined and connected along the channel dimension by applying a fully connected layer along the channel dimension, defined as:

[0052] in Indicates feature fusion, This indicates a fully connected layer.

[0053] The text kernel stretching module is configured to perform scale transformation on the text kernel using a non-uniform shrinkage strategy to improve detection accuracy and kernel integrity under extreme aspect ratio conditions; specifically, it performs scaling transformation on the text kernel according to a conventional shrinkage ratio in the vertical direction. The contraction occurs, with a smaller contraction ratio set in the horizontal direction, resulting in a contraction rate that is only a fraction of that in the vertical direction. ,in Indicates the stretch ratio, set This is used to ensure the separation of adjacent characters while avoiding excessive horizontal shrinkage; Calculate the shrinkage distance based on the Vatti clipping algorithm. Defined as:

[0054] in Represents the area of ​​a text instance. Represents the perimeter of a text instance. For the shrinkage ratio; for Yi characters with significant differences in aspect ratio, set... To achieve the maximum safe contraction distance.

[0055] During the recovery phase, the expansion distance is further calculated. Defined as:

[0056] in Indicates the predicted area of ​​the core. Indicates the predicted perimeter of the core. Indicates the desqueezing ratio; sets the original text region by performing an expansion operation on the prediction kernel. It can maintain high recovery accuracy under text conditions with different aspect ratios.

[0057] The prediction and text post-processing modules are configured to take the feature map processed by the dual-branch feature extraction and fusion module as input, generate a prediction map and a threshold map, and calculate an approximate binary map using a differentiable binarization function.

[0058] In the formula, Represents pixels in an approximate binary image. Represents the pixels of the predicted image. This represents the threshold image pixels. The function takes values ​​in the range (0, 1) and can be fine-tuned and optimized during training, allowing the network to learn a more reasonable binarization strategy.

[0059] During the inference phase, the prediction and post-processing modules utilize the approximate binary image combined with the text kernel mapping generated by the text kernel stretching module to recover the text boundaries through a post-processing step. This post-processing step includes: dynamically adjusting the expansion distance of the text kernel using an iterative approach. , with initial expansion distance Starting from this point, continuously adjust until the desired result is achieved. This method can adaptively optimize the expansion ratio based on the aspect ratio of different texts, improving recovery accuracy. In practice, It is usually set to 0.5 or 1.

[0060] In this embodiment, the loss function L of the Yi script ancient book image detection model consists of three weighted parts: shrinkage loss. Threshold map loss and approximate binarization loss The weighting coefficient and Let's set them to 5 and 10 respectively. This is a commonly used loss function in text detection, and the formula is:

[0061]

[0062]

[0063]

[0064] in Indicates the mask area; Let be the label of the i-th pixel. Let be the predicted probability of the i-th pixel; and These are the actual threshold and the predicted threshold, respectively. These are pixels in an approximate binary image.

[0065] In this embodiment of the application, the evaluation index of the Yi script ancient book image detection model is:

[0066]

[0067]

[0068] in Indicates the actual number of cases. Indicates the number of false positives. Represents the number of false negative examples.

[0069] S3. Train the Yi script ancient book image detection model based on the Yi script ancient book image dataset.

[0070] S4. Input the group of Yi script ancient book images to be detected into the trained Yi script ancient book image detection model, and obtain the output of the Yi script ancient book image detection model as the Yi script detection result image.

[0071] The solution and effects of the present invention will be further explained below based on specific application examples.

[0072] This embodiment discloses a method for detecting ancient Yi script images based on a two-branch Transformer, with detailed steps as follows: Step 1: Collect image data of ancient Yi script books The self-built ancient Yi script dataset is configured to include images of ancient documents obtained through high-definition scanning and digitization, along with their corresponding character annotation information. The character annotation information is obtained through a combination of semi-automatic segmentation and manual correction, and is used to provide character segmentation results and character positioning boxes, thereby ensuring the accuracy and completeness of the annotations. The dataset covers complex character structures and different layouts, and features strong character diversity, irregular character spacing, and dense text. It is divided into training and testing sets in an 8:2 ratio to ensure the effectiveness of model training and evaluation in complex scenarios.

[0073] Step 2: Construct an image dataset of ancient Yi script books A dataset containing 200 images of ancient Yi script books was constructed, each containing approximately 200 Yi characters. The dataset was divided into a training set and a test set, with 160 images randomly selected as the training set and the remaining 40 images used as the test set.

[0074] Step 3: Design a network for detecting images of ancient Yi script books, such as... Figure 2 As shown, its overall architecture consists of four modules: a feature extraction backbone network, a dual-branch feature extraction and fusion module, a text kernel stretching module, and a prediction and text post-processing module.

[0075] The input image undergoes multi-scale feature extraction via a feature extraction backbone composed of ResNet50 and a feature pyramid network. The dual-branch feature extraction and fusion module is used to jointly model global semantics and local details to enhance feature representation capabilities. The text kernel stretching module adaptively adjusts the shrinkage ratio of characters in the horizontal and vertical directions to improve the detection accuracy and kernel integrity of characters with extreme aspect ratios. The prediction and text post-processing module generates probability maps and threshold maps, and combines differentiable binarization and post-processing strategies to restore character boundaries, thereby obtaining robust Yi script detection results.

[0076] The dual-branch feature extraction and fusion module comprises a convolutional feedforward network and a context-aware local enhancement module. The convolutional feedforward network introduces depthwise separable convolutions into the traditional feedforward network structure, aggregating local spatial features while mapping channels, thereby enhancing the ability to model local information. The context-aware local enhancement module is used to jointly represent local and global features.

[0077] The convolutional feedforward network comprises a first fully connected layer, a nonlinear activation layer, a depthwise separable convolutional layer, and a second fully connected layer connected in sequence. The depthwise separable convolutional layer is placed after the activation layer and is used to aggregate local spatial information during the feedforward process. The convolutional feedforward network includes two forms: one is an intra-stage convolutional feedforward network, which uses residual connections and the stride of the depthwise separable convolutional layer is 1 to maintain feature resolution; the other is a cross-stage convolutional feedforward network, in which a depthwise separable convolutional layer with a stride of 2 and a fully connected layer are set in its residual branch to complete feature downsampling and channel dimension adjustment, thereby achieving feature connection between stages.

[0078] The context-aware local enhancement module includes a global branch and a local branch. The global branch downsamples the feature map and combines it with a self-attention mechanism to model long-distance dependencies, thereby extracting global semantic information of the image and improving the model's ability to perceive the overall structure. The local branch introduces convolution operations to aggregate local neighborhood features, thereby enhancing the model's ability to represent character edges and fine-grained strokes.

[0079] The text kernel stretching module employs a non-uniform shrinkage strategy to scale the text kernel, thereby improving detection accuracy and kernel integrity under extreme aspect ratio conditions. Specifically, it applies a conventional shrinkage ratio in the vertical direction. The contraction occurs, with a smaller contraction ratio set in the horizontal direction, resulting in a contraction rate that is only a fraction of that in the vertical direction. ,in Indicates the stretch ratio, set This is used to prevent excessive horizontal shrinkage while ensuring the separation of adjacent characters.

[0080] The prediction and text post-processing module takes the feature map processed by the dual-branch feature extraction and fusion module as input to generate a prediction map and a threshold map, and calculates an approximate binary map using a differentiable binarization function. During the inference phase, the approximate binary map is combined with the text kernel mapping generated by the text kernel stretching module to recover the text boundaries using a post-processing step. This post-processing step includes: dynamically adjusting the expansion distance of the text kernel using an iterative approach. , with initial expansion distance Starting from this point, continuously adjust until the desired result is achieved. This method can adaptively optimize the expansion ratio based on the aspect ratio of different texts, improving recovery accuracy. In practice, It is usually set to 0.5 or 1.

[0081] Step 4: Training and Testing of the Yi Script Ancient Book Image Detection Network In this embodiment of the invention, the Yi script ancient book image detection network designed in the Yi script ancient book image detection method based on a two-branch Transformer is used for training. The training dataset is a collection of Yi script ancient book images, with 160 images randomly selected for training. In this embodiment, the Adam optimizer is used, with the initial learning rate set to 1e-4 and the weight decay coefficient set to 1e-4. The learning rate decays to 0.1 at 50% and 80% of the total iterations. Each batch processes a set of 4 samples, and the entire network training process uses the MIMOCR1.0.0rc5 framework on an NVIDIA GeForce RTX3090 Ti GPU. After training, 40 randomly selected images are used as the test set to verify the effectiveness of the method of the present invention. In addition, six deep learning-based methods are compared with the present invention. The present invention uses three evaluation metrics: precision, recall, and harmonic mean (F-measure) to objectively evaluate the performance of the test results, as shown in Table 1. The best and second-best results are represented in red and blue, respectively, and "Ours" indicates our method.

[0082] Table 1 Comparison of image detection results between this method and other methods.

[0083] The proposed method for detecting Yi script ancient text images based on a dual-branch Transformer effectively addresses the detection difficulties caused by complex character shapes, dense character formation, and background interference in Yi script ancient text images, achieving precise localization and boundary restoration of character regions. This method not only overcomes the limitations of traditional text detection methods in balancing global semantics and local details but also demonstrates superior performance in detecting characters with extreme aspect ratios, resulting in more complete and accurate detection results. Compared with existing technologies, this invention exhibits significant advantages in robustness and detection accuracy in complex scenarios, thus providing new technical approaches and possibilities for the digital preservation and intelligent recognition of Yi script.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting ancient Yi script texts based on a two-branch Transformer, characterized in that, Includes the following steps: A dataset of images of ancient Yi script books was obtained. The dataset of images of ancient Yi script books includes clear images of ancient Yi script books and corresponding character annotation information groups. The character annotation information groups include text segmentation results completed by semi-automatic annotation and manual methods, as well as character positioning boxes. A model for detecting ancient Yi script texts is constructed. This model includes a text detection network based on a two-branch Transformer structure. The text detection network comprises a feature extraction backbone network, a two-branch feature extraction and fusion module, a text kernel stretching module, and prediction and text post-processing modules. Input images undergo multi-scale feature extraction via a feature extraction backbone composed of ResNet50 and a feature pyramid network. The two-branch feature extraction and fusion module is used to jointly model global semantics and local details to enhance feature representation capabilities. The text kernel stretching module adaptively adjusts the horizontal and vertical shrinkage ratios of characters to improve detection accuracy and kernel completeness for characters with extreme aspect ratios. The prediction and text post-processing modules generate probability maps and threshold maps, and combine differentiable binarization and post-processing strategies to restore character boundaries, thereby obtaining robust Yi script detection results. The Yi script ancient book image detection model was trained based on the Yi script ancient book image dataset; The image of the ancient Yi script book to be detected is obtained, and the image is input into the trained Yi script ancient script book image detection model. The character positioning boxes output by the model are used as the detection results.

2. The method for detecting ancient Yi script images based on a dual-branch Transformer according to claim 1, characterized in that, The dual-branch feature extraction and fusion module includes a convolutional feedforward network and a context-aware local enhancement module. The convolutional feedforward network introduces a depthwise separable convolutional module into the traditional feedforward network structure to aggregate local spatial features while mapping channels. The context-aware local enhancement module is used to jointly represent local and global features.

3. The method for detecting ancient Yi script images based on a dual-branch Transformer according to claim 2, characterized in that, The convolutional feedforward network includes a first fully connected layer, a nonlinear activation layer, a depthwise separable convolutional layer, and a second fully connected layer connected in sequence. The depthwise separable convolutional layer is placed after the activation layer and is used to aggregate local spatial information during the feedforward process. The convolutional feedforward network includes two forms: one is an intra-stage convolutional feedforward network, which uses residual connections and the stride of the depthwise separable convolutional layer is 1 to maintain feature resolution; the other is a cross-stage convolutional feedforward network, in which a depthwise separable convolutional layer and a fully connected layer with a stride of 2 are set in its residual branch to complete feature downsampling and channel dimension adjustment to achieve feature connection between stages.

4. The method for detecting ancient Yi script images based on a dual-branch Transformer according to claim 2, characterized in that, The context-aware local enhancement module includes global branches and local branches; The global branch is configured to: process the key vector obtained from the input feature mapping. Sum value vector Downsampling is performed to reduce feature resolution and computational complexity, thus reducing the query vector. The downsampled key vector and downsampled value vector are used for input attention calculation to obtain global context information. In the formula, This indicates an operation that calculates the similarity between the query and the key, and then performs a weighted sum based on the value vector. This represents the average pooling operation, where the global branch is used to extract low-frequency global features and model long-distance dependencies.

5. The method for detecting ancient Yi script images based on a dual-branch Transformer according to claim 4, characterized in that, The local branch is configured to: process the input features using an AttnConv structure based on convolutional attention to extract high-frequency local information; and process the input features... Perform a linear mapping to obtain the query vector. Key vector AND value vector The process is represented as follows: In the formula, Indicates a fully connected mapping; After completing the linear mapping, the value vector Depthwise separable convolution operations are performed to aggregate local features with shared weights, resulting in an aggregated local representation. : For query vector Key vector Combined to generate context-aware weights, using two depthwise separable convolutions. To each and Aggregate local information: calculate and Element-wise multiplication and nonlinear mapping are performed, and then passed through a fully connected layer and a nonlinear activation function. The intermediate representation is obtained, and then passed through a fully connected layer and the hyperbolic tangent function. Normalization is performed to obtain context-aware attention weights. Its calculation is defined as: In the formula, Represents element-wise product. Represents a non-linear activation function. The feature dimension is defined as follows: the attention weights are multiplied element-wise by the convolution transformation result of the value vector to obtain the local enhancement features. The calculation formula is as follows: in, Represents a value vector The features after depthwise separable convolution operation, the It is used to characterize high-frequency local detail information and is fused with the global branch output as an enhanced local feature; Output features of the global branch Local branch output features Fusion involves concatenating the two outputs along the channel dimension and applying a fully connected layer along that channel dimension. The calculation formula is as follows: In the formula, Indicates feature fusion, This indicates a fully connected layer.

6. The method for detecting ancient Yi script images based on a dual-branch Transformer according to claim 1, characterized in that, The text kernel stretching module is configured to perform scale transformation on the text kernel using a non-uniform shrinkage strategy to improve detection accuracy and kernel integrity under extreme aspect ratio conditions. The process is as follows: In the vertical direction, according to the conventional shrinkage ratio When shrinkage occurs, and the shrinkage ratio is set in the horizontal direction, its shrinkage rate is only one-third of that in the vertical direction. ,in Indicates the stretch ratio, set This is used to ensure the separation of adjacent characters while avoiding excessive horizontal shrinkage; The text kernel stretching module calculates the shrinkage distance based on the Vatti cropping algorithm. Its formula is: In the formula, Represents the area of ​​a text instance. Represents the perimeter of a text instance. For the shrinkage ratio; for Yi characters with significant differences in aspect ratio, set... To achieve the maximum safe contraction distance; During the recovery phase, the text kernel stretching module calculates the expansion distance. The calculation formula is as follows: In the formula, Indicates the predicted area of ​​the core. Indicates the predicted perimeter of the core. Indicates the desqueezing ratio; sets the original text region by performing an expansion operation on the prediction kernel. It can maintain high recovery accuracy under text conditions with different aspect ratios.

7. The method for detecting ancient Yi script images based on a dual-branch Transformer according to claim 1, characterized in that, The prediction and text post-processing module is configured to: take the feature map processed by the dual-branch feature extraction and fusion module as input, generate a prediction map and a threshold map, and calculate an approximate binary map using a differentiable binarization function. In the formula, Represents pixels in an approximate binary image. Represents the pixels of the predicted image. This represents the threshold image pixels. The value range of this function is (0, 1). It can be fine-tuned and optimized during training, so that the network can learn a more reasonable binarization strategy. The prediction and text post-processing module utilizes the approximate binary image combined with the text kernel mapping generated by the text kernel stretching module during the inference phase to restore the text boundaries using a post-processing step. The post-processing step includes: dynamically adjusting the expansion distance of the text kernel using an iterative approach. , with initial expansion distance Starting from this point, continuously adjust until the desired result is achieved. .

8. The method for detecting ancient Yi script images based on a dual-branch Transformer according to claim 1, characterized in that, The loss function during training of the Yi script ancient book image detection model is: the loss function L consists of three weighted parts: shrinkage loss Threshold map loss and approximate binarization loss The weighting coefficient and The specific forms of the three losses, set at 5 and 10 respectively, are as follows: In the formula, Indicates the mask area. Let be the label of the i-th pixel. Let be the predicted probability of the i-th pixel. and These are the actual threshold and the predicted threshold, respectively. These are pixels in an approximate binary image.

9. The method for detecting ancient Yi script images based on a dual-branch Transformer according to claim 1, characterized in that, The evaluation metric for training the Yi script ancient book image detection model is: In the formula, Indicates the actual number of cases. Indicates the number of false positives. Represents the number of false negative examples.

Images