A license text detection method and system based on spatial calibration and feature fusion

Abstract

The application provides a license text detection method and system based on spatial calibration and feature fusion, and belongs to the technical field of image processing. The method inputs a license image into a network based on PP-HGNet to extract multi-scale features; performs enhancement processing on the multi-scale feature maps based on spatial calibration, extracts global context, performs shape self-calibration and local detail fusion to strengthen the spatial structure information of the text; performs fusion processing on the enhanced features based on multi-scale hollow convolution, realizes adaptive weighted fusion of multi-scale features by combining multi-branch hollow convolution with a double attention mechanism, and finally generates a binary graph of the text region through differentiable binarization. The application effectively solves the problems of dense text boundary confusion, long text breakage and uneven multi-scale text feature capture in the license, significantly improves the detection accuracy and efficiency, and is suitable for automatic extraction of license information.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more particularly to a method and system for detecting certificate text based on spatial calibration and feature fusion. Background Technology

[0002] Document text detection is a crucial component of Optical Character Recognition (OCR) systems, widely used for the automated extraction and verification of information from various documents such as ID cards, driver's licenses, and business licenses. With the development of deep learning technology, neural network-based text detection methods have become a mainstream research direction. These methods are mainly categorized into three types: regression-based text detection, segmentation-based text detection, and Transformer-based text detection.

[0003] Regression-based text detection methods locate and detect text regions by optimizing anchor point design. For example, TextBoxes++ uses quadrilateral anchor boxes to specifically adapt to the shape characteristics of multi-directional text, and uses bounding box regression to locate tilted and rotated text. However, when faced with multi-scale and elongated text in certificates and documents, this type of method struggles to design suitable text boxes for accurate detection, and the detection speed is slow, failing to meet the practical requirements of efficiency for certificate and document text detection.

[0004] Segmentation-based text detection methods transform text detection into a pixel-level classification task, achieving text localization by accurately characterizing the pixel distribution of text regions. For example, methods like PSENet, DBNet, and DBNet++, through progressive scale scaling or differentiable binarization operations, can efficiently handle text targets of arbitrary shapes, offering advantages in simplicity and efficiency. However, in document and certificate text detection tasks, these methods are susceptible to interference from complex backgrounds, prone to missed detections in multi-scale text, and struggle to accurately segment dense text lines, thus limiting overall detection performance.

[0005] Transformer-based text detection methods leverage the self-attention mechanism of the Transformer architecture to directly capture long-distance dependencies between text regions, achieving end-to-end text detection and contextual information fusion. For example, methods like DPText-DETR improve text detection accuracy through global modeling capabilities. However, these methods suffer from slow training convergence and high computational costs; their computational complexity increases quadratically with the number of image pixels, resulting in a significant speed gap compared to segmentation-based methods on document datasets. Furthermore, their multi-scale feature fusion capabilities are insufficient, failing to adequately meet the practical needs of document text detection.

[0006] In summary, existing methods for detecting certificate text still have significant shortcomings in handling issues such as blurred boundaries in dense multi-line text, fragmented long text, and unbalanced capture of multi-scale text features. There is an urgent need for a text detection method that can balance detection accuracy and efficiency, and effectively enhance the fusion of text spatial structure features and multi-scale context. Summary of the Invention

[0007] To address the technical problems of existing certificate text detection methods, such as boundary confusion, fragmentation of long texts, and unbalanced feature capture across multiple scales when processing dense multi-line texts, and the difficulty in balancing detection accuracy and efficiency, this invention provides a certificate text detection method based on spatial calibration and feature fusion. This invention primarily utilizes a hierarchical feature extraction network based on PP-HGNet to extract multi-scale features, and enhances these multi-scale feature maps using spatial calibration to strengthen the text's spatial structure information and boundary discriminability. Simultaneously, it fuses the enhanced features using multi-scale dilated convolution to adapt to the receptive field requirements of texts at different scales. Finally, it generates a binary map of the text region through differentiable binarization, thereby accurately extracting the structural information and contextual features of the certificate text, effectively improving the detection capability for dense multi-line texts, long texts, and multi-scale texts, achieving high-precision and efficient certificate text detection.

[0008] The technical means employed in this invention are as follows: A certificate text detection method based on spatial calibration and feature fusion includes: S1. Obtain the image of the certificate to be detected. X The image of the certificate to be tested X Input the feature extraction network based on PP-HGNet, and perform multi-scale feature extraction through feature initialization and hierarchical HG Block stacking to obtain feature maps of different scales; S2. Enhance the feature maps of multiple different scales based on spatial calibration. Through global context extraction, shape self-calibration and local detail fusion, obtain multiple calibration feature maps of different scales. Then, stitch the multiple calibration feature maps of different scales together in the channel dimension to output a feature map that integrates multi-scale information. S3. The feature map integrating multi-scale information is fused based on multi-scale dilated convolution. The receptive field is expanded by multi-branch dilated convolution with customized hierarchical dilation rate. Adaptive weighted fusion is performed by combining channel attention and spatial attention to output an enhanced fused feature map. S4. Perform text region detection processing on the enhanced fusion feature map, generate text region probability map and threshold map through parallel branching, and obtain text region binary map through differentiable binarization operation to realize certificate text detection.

[0009] Further, step S1 includes: S11. Obtain the image of the certificate to be inspected. The images of certificates to be tested Perform feature initialization, after After convolutional downsampling, it splits into two branches. The first branch passes through two... Convolution completes feature refinement; the second branch goes through one... The max pooling layer concatenates the two outputs and then passes them through a... After convolution and double downsampling, the output size is The initial feature map; S12. Input the initial feature map into the backbone network. The backbone network contains four HG Block stages, wherein: The first HG Block stage contains one HG Block module, with the initial feature map as input and an output size of [size missing]. First feature map ; The second HG Block stage contains one HG Block module, with the first feature map as input. The output size is Second feature map ; The third HG Block stage consists of three HG Block modules, with the second feature map as the input. The output size is The third feature map ; The fourth HG Block stage contains one HG Block module, with the third feature map as input. The output size is The fourth feature map .

[0010] Furthermore, in step S12, the HG Block module, as the core feature extraction unit, employs dense... The structural design and extraction process of convolution and residual connections include: In 5 groups Convolution is a parallel branch, and each convolution is paired with Batch Normalization (BN) and ReLU activation functions. Information integration of local texture and global structure is achieved through multi-branch feature fusion. The feature fusion process is as follows:

[0011]

[0012] in, The input feature map for the HG Block. Indicates the first Group convolution, Output for each convolutional branch, This is a channel-level splicing operation. The fused feature map output by HG Block; pass After convolution completes channel compression, a residual connection is established with the input feature map.

[0013] Further, step S2 includes: S21. Apply horizontal and vertical pooling to the first feature map respectively. Second feature map Third feature map and the fourth feature map Dimensional aggregation is performed to generate vectors in both horizontal and vertical directions. Then, rectangular attention regions are modeled using broadcast addition to capture the global structure of the entire row and column of text. S22. By optimizing the matching degree between the attention region and the actual shape of the text through spatial calibration, the background interference is significantly reduced, the adjacent boundaries of dense text are distinguished, and the calibrated attention map is obtained. The spatial calibration process is as follows:

[0014] in, This represents the initial rectangular attention map. This represents the Sigmoid function. Represents the ReLU activation function. This represents the calibrated attention map; S23. Apply depthwise separable convolution to the first feature map. Second feature map Third feature map and the fourth feature map Local detail features are extracted, and the calibrated attention map is weighted and fused with the corresponding local detail features. Residual connections and a multilayer perceptron are then used for feature refinement, outputting multiple single-scale calibration feature maps, including a first single-scale calibration feature map. Second single-scale calibration feature map Third single-scale calibration feature map and the fourth single-scale calibration feature map ; S24. Calculate the first single-scale calibration feature map. Second single-scale calibration feature map Third single-scale calibration feature map and the fourth single-scale calibration feature map The feature maps are concatenated along the channel dimension to output a feature map that integrates multi-scale information. .

[0015] Further, step S23 includes: S231. Apply depthwise separable convolution to the first feature map. Second feature map Third feature map and the fourth feature map Extract local detail features, preserving text strokes and fine edge textures; S232. By using the Hadamard product, the calibrated attention map is fused with local detail features element by element in a weighted manner, allowing global structural information to accurately guide the effective area of ​​local details. S233. Residual connections and MLP layers are introduced. Residual connections prevent the loss of feature information during the fusion process. MLP layers further refine the feature dimensions and improve the feature reuse capability through nonlinear transformation, and finally output multiple single-scale calibration feature maps.

[0016] Further, step S3 includes: S31. The feature map integrating multi-scale information Three hierarchical dilated convolutional branches are input in parallel to adapt to the needs of small-scale, large-scale and wide-space text detection, and three branch feature maps are output. S32. Concatenate the three branch feature maps along the channel dimension to obtain a multi-branch fused feature map. ; S33, Multi-branch fusion feature map Channel attention weighting and spatial attention weighting are performed, and the features after dual attention weighting are convolutionally processed to unify the channel dimensions, outputting an enhanced fusion feature map. .

[0017] Further, in step S31, the three hierarchical dilated convolution branches include a small dilation rate branch, a large dilation rate branch, and a wide-spacing branch, wherein: The small void ratio branch uses void ratios of 1 and 3. By combining convolutions sequentially, the receptive field is appropriately expanded while preserving details, focusing on preserving the stroke details and edge features of small texts and avoiding the loss of details; The high void ratio branch uses void ratios of 6 and 9. Convolutional sequences are combined to cover the entire text area with a larger receptive field; Wide-spaced branches use a hole ratio of 3 for wide-spaced text. Convolution combined with interval attention mechanism strengthens the correlation between characters within the same text line and weakens the feature fragmentation problem caused by wide spacing. The core of interval attention is to use bar convolution in the line direction to weighted aggregate the features in the text line direction and identify characters belonging to the same text line under wide spacing.

[0018] Further, step S33 includes: S331, Channel Attention Module for Multi-Branch Fusion Feature Map Global average pooling is performed to compress spatial dimension features into channel dimension global statistical information, and then... Convolution and Sigmoid activation generate channel weight maps This enables feature weighting along the channel dimension. S332, Spatial Attention Module for Multi-Branch Fusion Feature Maps conduct Convolutional dimensionality reduction and Sigmoid activation generate a spatial weight map. Complete the feature weighting of the spatial dimension; S333, Combine the channel-weighted features and the spatial-weighted features through... Convolution unifies the channel dimensions and outputs an enhanced fusion feature map that combines detail and semantics. .

[0019] Further, step S4 includes: S41. The enhanced fusion feature map After convolution and upsampling operations, the number of channels in the feature map is adjusted, and two parallel branches generate probability maps of the same size as the original image. and threshold map ; S42, in the probability graph The last layer uses Sigmoid activation, with output values ​​between 0 and 1, representing the probability that each pixel belongs to the text region; S43, in the threshold map The final layer uses Sigmoid activation, with output values ​​between 0 and 1, representing the threshold for each pixel, used in conjunction with the probability map. The probability values ​​are compared to determine whether the pixel belongs to the text region; S44. Through differentiable binarization, based on the probability graph... With threshold map Difference calculation yields a binary image of the text region. The formula is as follows:

[0020] in, This indicates the coordinate position in the feature map. Represents a probability graph In coordinates Pixel value at that location, Threshold map In coordinates Pixel value at that location, Indicates the magnification factor. Binary graph representing text region Those seated The output value at that location.

[0021] This invention also provides a certificate text detection system based on spatial calibration and feature fusion, implemented using the aforementioned certificate text detection method based on spatial calibration and feature fusion. The system includes: a feature extraction module based on PP-HGNet, a feature enhancement module based on spatial calibration, a feature fusion module based on multi-scale dilated convolution, and a text region detection module, wherein: The feature extraction module based on PP-HGNet is used to acquire the certificate image to be detected. X The image of the certificate to be tested X Input the feature extraction network based on PP-HGNet, and perform multi-scale feature extraction through feature initialization and hierarchical HG Block stacking to obtain feature maps of different scales; The spatial calibration-based feature enhancement module is used to enhance the feature maps of multiple different scales based on spatial calibration. Through global context extraction, shape self-calibration and local detail fusion, multiple calibration feature maps of different scales are obtained, and the multiple calibration feature maps of different scales are stitched together in the channel dimension to output a feature map integrating multi-scale information. The feature fusion module based on multi-scale dilated convolution is used to fuse the feature map integrating multi-scale information based on multi-scale dilated convolution. It expands the receptive field by multi-branch dilated convolution with customized hierarchical dilation rate, and combines channel attention and spatial attention for adaptive weighted fusion to output an enhanced fused feature map. The text region detection module is used to perform text region detection processing on the enhanced fusion feature map. It generates a text region probability map and a threshold map through parallel branches, and obtains a text region binary map through a differentiable binarization operation, thereby realizing certificate text detection.

[0022] Compared with the prior art, the present invention has the following advantages: 1. This invention designs a feature extraction network based on PP-HGNet. Through a hierarchical feature extraction structure with dense convolution and residual connections to HG Block, it achieves efficient capture and lightweight processing of multi-scale text features. While ensuring high-precision feature extraction capabilities, it also takes into account detection efficiency, effectively overcoming the problems of limited receptive field and weak representation ability of traditional backbone networks.

[0023] 2. This invention adopts a feature enhancement processing method based on spatial calibration. Through the synergistic effect of horizontal and vertical bidirectional pooling, shape self-calibrating strip convolution, and local detail fusion mechanism, it achieves accurate differentiation of adjacent boundaries of dense text and effective correction of morphological distortion of long text, significantly enhancing the horizontal spatial continuity and vertical boundary differentiation of text features.

[0024] 3. This invention employs feature fusion processing based on multi-scale dilated convolution. Through the joint optimization of multi-branch dilated convolution with customized hierarchical dilation rate and channel space dual attention mechanism, it achieves accurate matching of receptive fields of text at different scales and effective association of wide-space text features, achieving efficient fusion of shallow details and deep semantics, and effectively avoiding missed detection of small texts and wide-space character fragmentation.

[0025] 4. This invention employs a differentiable binarization text region detection processing method. Through the parallel generation of adaptive probability maps and threshold maps, as well as differentiable binarization operations, it achieves refined segmentation of text region boundaries. Experimental results on the CertDoc dataset and the MTWI_2018 dataset show that the F1 score of the method of this invention reaches 90.8% and 79.7%, respectively, which is 1.6% and 1.5% higher than the mainstream PP-OCRv5 method, respectively, significantly improving the accuracy and recall of text detection.

[0026] In summary, by applying the technical solution of this invention, the problem of unbalanced multi-scale text feature capture caused by the limited receptive field and weak representation ability of the backbone network in existing technologies is solved by using the PP-HGNet hierarchical feature extraction network; the problems of boundary confusion and fragmentation of long texts in existing technologies are solved by spatial calibration feature enhancement processing; and the problems of missed detection of small texts, feature fragmentation of wide-interval texts, and insufficient context fusion in existing technologies are solved by multi-scale dilated convolutional feature fusion processing. Therefore, the technical solution of this invention solves the problems of difficulty in balancing accuracy and efficiency in certificate text detection, unbalanced multi-scale text feature capture, inaccurate boundary segmentation of dense texts, and fragmentation of long texts in existing technologies. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a diagram of the overall network structure for text detection in this invention.

[0029] Figure 2 This is a diagram of the feature extraction network structure based on PP-HGNet in this invention.

[0030] Figure 3 This is a network structure diagram of the HG block of the present invention.

[0031] Figure 4 This is a network structure diagram of the feature enhancement module based on spatial calibration in this invention.

[0032] Figure 5 This is a network structure diagram of the feature fusion module based on multi-scale dilated convolution in this invention.

[0033] Figure 6 This is a network structure diagram of the text region detection module of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] likeFigure 1 As shown, this invention provides a method for detecting certificate text based on spatial calibration and feature fusion, including: S1. Obtain the image of the certificate to be detected. X The image of the certificate to be tested X Input the feature extraction network based on PP-HGNet, and perform multi-scale feature extraction through feature initialization and hierarchical HG Block stacking to obtain feature maps of different scales; S2. Enhance the feature maps of multiple different scales based on spatial calibration. Through global context extraction, shape self-calibration and local detail fusion, obtain multiple calibration feature maps of different scales. Then, stitch the multiple calibration feature maps of different scales together in the channel dimension to output a feature map that integrates multi-scale information. S3. The feature map integrating multi-scale information is fused based on multi-scale dilated convolution. The receptive field is expanded by multi-branch dilated convolution with customized hierarchical dilation rate. Adaptive weighted fusion is performed by combining channel attention and spatial attention to output an enhanced fused feature map. S4. Perform text region detection processing on the enhanced fusion feature map, generate text region probability map and threshold map through parallel branching, and obtain text region binary map through differentiable binarization operation to realize certificate text detection.

[0037] In a specific implementation, as a preferred embodiment of the present invention, step S1 includes: S11. Obtain the image of the certificate to be inspected. The images of certificates to be tested Perform feature initialization, after After convolutional downsampling, it splits into two branches. The first branch passes through two... Convolution completes feature refinement; the second branch goes through one... The max pooling layer concatenates the two outputs and then passes them through a... After convolution and double downsampling, the output size is The initial feature map; S12. Input the initial feature map into the backbone network. The backbone network contains four HG Block stages, wherein: The first HG Block stage contains one HG Block module, with the initial feature map as input and an output size of [size missing]. First feature map ; The second HG Block stage contains one HG Block module, with the first feature map as input. The output size is Second feature map ; The third HG Block stage consists of three HG Block modules, with the second feature map as the input. The output size is The third feature map ; The fourth HG Block stage contains one HG Block module, with the third feature map as input. The output size is The fourth feature map .

[0038] In this embodiment, to address the shortcomings of traditional ResNet50 as a backbone network, such as small receptive field and weak representation ability, this invention employs a feature extraction network based on PP-HGNet, with the overall structure as follows: Figure 2 As shown, it can effectively address the problem of unbalanced multi-scale text feature capture in certificates and licenses.

[0039] In a specific implementation, as a preferred embodiment of the present invention, in step S12, the HG Block module serves as the core feature extraction unit, employing dense... The structural design of convolution and residual connections, and the network structure diagram are as follows: Figure 3 As shown, the extraction process includes: In 5 groups Convolution is a parallel branch, and each convolution is paired with Batch Normalization (BN) and ReLU activation functions. Information integration of local texture and global structure is achieved through multi-branch feature fusion. The feature fusion process is as follows:

[0040]

[0041] in, The input feature map for the HG Block. Indicates the first Group convolution, Output for each convolutional branch, This is a channel-level splicing operation. The fused feature map output by HG Block; pass After convolution completes channel compression, a residual connection is established with the input feature map. This avoids the gradient vanishing problem of deep networks and preserves the text position information in shallow features, providing a complete feature foundation for the subsequent spatial calibration module.

[0042] In a specific implementation, as a preferred embodiment of the present invention, step S2 includes: S21. Apply horizontal and vertical pooling to the first feature map respectively. Second feature map Third feature map and the fourth feature map Dimensional aggregation is performed to generate vectors in both horizontal and vertical directions. Then, rectangular attention regions are modeled using broadcast addition to capture the global structure of the entire row and column of text. S22. By optimizing the matching degree between the attention region and the actual shape of the text through spatial calibration, the background interference is significantly reduced, the adjacent boundaries of dense text are distinguished, and the calibrated attention map is obtained. The spatial calibration process is as follows:

[0043] in, This represents the initial rectangular attention map. This represents the Sigmoid function. Represents the ReLU activation function. This represents the calibrated attention map; S23. Apply depthwise separable convolution to the first feature map. Second feature map Third feature map and the fourth feature map Local detail features are extracted, and the calibrated attention map is weighted and fused with the corresponding local detail features. Residual connections and a multilayer perceptron are then used for feature refinement, outputting multiple single-scale calibration feature maps, including a first single-scale calibration feature map. Second single-scale calibration feature map Third single-scale calibration feature map and the fourth single-scale calibration feature map ; S24. Calculate the first single-scale calibration feature map. Second single-scale calibration feature map Third single-scale calibration feature map and the fourth single-scale calibration feature map The feature maps are concatenated along the channel dimension to output a feature map that integrates multi-scale information. .

[0044] In this embodiment, addressing the detection challenges of traditional detection methods when handling multi-line dense text and long texts, a feature enhancement module based on spatial calibration enhances the horizontal spatial continuity and vertical boundary discriminativeness of text features through context extraction, shape calibration, and fusion of local details. The network structure is as follows: Figure 4 As shown.

[0045] In a specific implementation, as a preferred embodiment of the present invention, step S23 includes: S231. Apply depthwise separable convolution to the first feature map. Second feature map Third feature map and the fourth feature map Extract local detail features, preserving text strokes and fine edge textures; S232. By using the Hadamard product, the calibrated attention map is fused with local detail features element by element in a weighted manner, allowing global structural information to accurately guide the effective area of ​​local details. S233. Residual connections and MLP layers are introduced. Residual connections prevent the loss of feature information during the fusion process. MLP layers further refine the feature dimensions and improve the feature reuse capability through nonlinear transformation, and finally output multiple single-scale calibration feature maps.

[0046] In a specific implementation, as a preferred embodiment of the present invention, step S3 includes: S31. The feature map integrating multi-scale information Three hierarchical dilated convolutional branches are input in parallel to adapt to the needs of small-scale, large-scale and wide-space text detection, and three branch feature maps are output. S32. Concatenate the three branch feature maps along the channel dimension to obtain a multi-branch fused feature map. ; S33, Multi-branch fusion feature map Channel attention weighting and spatial attention weighting are performed, and the features after dual attention weighting are convolutionally processed to unify the channel dimensions, outputting an enhanced fusion feature map. .

[0047] In this embodiment, feature fusion based on multi-scale dilated convolution expands the receptive field and accurately captures spacing information through multi-branch dilated convolution with customized hierarchical dilation rates. Combined with a dual attention mechanism of channel and spatial attention, adaptive allocation of feature weights is achieved, ultimately resulting in efficient fusion of shallow details and deep semantics. The network structure is as follows: Figure 5 As shown.

[0048] In a specific implementation, as a preferred embodiment of the present invention, in step S31, the three hierarchical dilated convolution branches include a small dilation rate branch, a large dilation rate branch, and a wide-spacing branch, wherein: The small void ratio branch uses void ratios of 1 and 3. By combining convolutions sequentially, the receptive field is appropriately expanded while preserving details, focusing on preserving the stroke details and edge features of small texts and avoiding the loss of details; The high void ratio branch uses void ratios of 6 and 9. Convolutional sequences are combined to cover the entire text area with a larger receptive field; Wide-spaced branches use a hole ratio of 3 for wide-spaced text. Convolution combined with interval attention mechanism strengthens the correlation between characters within the same text line and weakens the feature fragmentation problem caused by wide spacing. The core of interval attention is to use bar convolution in the line direction to weighted aggregate the features in the text line direction and identify characters belonging to the same text line under wide spacing.

[0049] In a specific implementation, as a preferred embodiment of the present invention, step S33 includes: S331, Channel Attention Module for Multi-Branch Fusion Feature Map Global average pooling is performed to compress spatial dimension features into channel dimension global statistical information, and then... Convolution and Sigmoid activation generate channel weight maps This enables feature weighting at the channel dimension, dynamically increasing the weight of effective feature channels such as text edges, and suppressing interference from redundant channels such as background. S332, Spatial Attention Module for Multi-Branch Fusion Feature Maps conduct Convolutional dimensionality reduction and Sigmoid activation generate a spatial weight map. This completes the feature weighting in the spatial dimension, further enhancing the spatial resolution of features and improving the feature recognition of text regions; S333, Combine the channel-weighted features and the spatial-weighted features through... Convolution unifies the channel dimensions and outputs an enhanced fusion feature map that combines detail and semantics. .

[0050] In specific implementation, as a preferred embodiment of the present invention, such as Figure 6 As shown, step S4 includes: S41. The enhanced fusion feature map After convolution and upsampling operations, the number of channels in the feature map is adjusted, and two parallel branches generate probability maps of the same size as the original image. and threshold map ; S42, in the probability graph The last layer uses Sigmoid activation, with output values ​​between 0 and 1, representing the probability that each pixel belongs to the text region; S43, in the threshold map The final layer uses Sigmoid activation, with output values ​​between 0 and 1, representing the threshold for each pixel, used in conjunction with the probability map. The probability values ​​are compared to determine whether the pixel belongs to the text region; S44. Through differentiable binarization, based on the probability graph... With threshold map Difference calculation yields a binary image of the text region. The formula is as follows:

[0051] in, This indicates the coordinate position in the feature map. Represents a probability graph In coordinates Pixel value at that location, Threshold map In coordinates Pixel value at that location, Indicates the magnification factor. Binary graph representing text region Those seated The output value at that location.

[0052] This invention also provides a certificate text detection system based on spatial calibration and feature fusion, implemented using the aforementioned certificate text detection method based on spatial calibration and feature fusion. The system includes: a feature extraction module based on PP-HGNet, a feature enhancement module based on spatial calibration, a feature fusion module based on multi-scale dilated convolution, and a text region detection module, wherein: The feature extraction module based on PP-HGNet is used to acquire the certificate image to be detected. X The image of the certificate to be tested X Input the feature extraction network based on PP-HGNet, and perform multi-scale feature extraction through feature initialization and hierarchical HG Block stacking to obtain feature maps of different scales; In this embodiment, the PP-HGNet-based feature extraction network enhances multi-scale feature capture and context modeling capabilities through HG Block stacking and hierarchical feature fusion. It maintains high-precision feature extraction while also offering lightweight advantages, making it suitable for complex document text detection needs. The input is a document image. X This network completes initial feature extraction and size compression through feature initialization. The HG Block serves as the core feature extraction unit, employing dense... The structural design of convolution and residual connections enables hierarchical feature learning from shallow to deep, which not only enhances the ability to capture fine-grained text features but also effectively preserves multi-scale text information, ultimately outputting feature maps of four different scales. , , and .

[0053] The spatial calibration-based feature enhancement module is used to enhance the feature maps of multiple different scales based on spatial calibration. Through global context extraction, shape self-calibration and local detail fusion, multiple calibration feature maps of different scales are obtained, and the multiple calibration feature maps of different scales are stitched together in the channel dimension to output a feature map integrating multi-scale information. In this embodiment, the feature enhancement module based on spatial calibration is a core optimization unit addressing the pain points of detecting multi-line dense text and long text, improving the spatial resolution capability of text features. The input is a feature map. , , and This module extracts global contextual information through horizontal and vertical bidirectional pooling to model the overall structural features of the text. Then, it performs shape self-calibration on the initial attention map using horizontally and vertically decoupled large-kernel bar convolutions, accurately distinguishing adjacent boundaries in dense text and correcting morphological distortion in long texts. Finally, it combines local detail extraction through depthwise separable convolutions with residual connections to enhance both global structure and local details, outputting four spatially calibrated feature maps. , , and .

[0054] The feature fusion module based on multi-scale dilated convolution is used to fuse the feature map integrating multi-scale information based on multi-scale dilated convolution. It expands the receptive field by multi-branch dilated convolution with customized hierarchical dilation rate, and combines channel attention and spatial attention for adaptive weighted fusion to output an enhanced fused feature map. In this embodiment, a feature fusion module based on multi-scale dilated convolution is used to address the issues of missed detection of small texts and fragmented text features with wide intervals, achieving deep fusion of text features at different scales. The input is a feature map. , , and Feature map obtained by Concat This module precisely matches the receptive field requirements of text at different scales through multi-branch dilated convolutions with customized hierarchical dilation rates. It then dynamically adjusts feature channel weights through channel attention and enhances the spatial discriminability of text regions through spatial attention. Adaptive weighted fusion of multi-branch features ultimately outputs an enhanced feature map that combines shallow details with deep semantics. .

[0055] The text region detection module is used to perform text region detection processing on the enhanced fusion feature map. It generates a text region probability map and a threshold map through parallel branches, and obtains a text region binary map through a differentiable binarization operation, thereby realizing certificate text detection.

[0056] In this embodiment, the text region detection module is used to convert the fused features into a binary map of the text region. The input is the feature map. This module introduces two parallel branches, which generate a probability map and a threshold map with the same size as the original image through convolution and upsampling operations, respectively; then, through differentiable binarization, a binary map of the text region is calculated based on the probability map and the threshold map.

[0057] The embodiments of the present invention are described simply because they correspond to those in the embodiments above. For any similarities, please refer to the descriptions in the embodiments above, which will not be elaborated here.

[0058] Example To verify the superiority of the method proposed in this patent, comparative experiments were conducted on the CertDoc and MTWI datasets to analyze its superiority over mainstream text detection methods. The relevant experimental results are shown in Tables 1 and 2. The results demonstrate that the method proposed in this patent outperforms other methods in terms of Precision, Recall, and F1 scores on both the CertDoc and MTWI datasets. Specifically, the F1 scores on the CertDoc and MTWI_2018 datasets reached 90.8% and 79.7%, respectively, exceeding the best-performing PP-OCRv5 method by 1.6% and 1.5%, respectively. This indicates that the text detection method proposed in this patent is better suited for document text detection tasks.

[0059] Table 1. Performance comparison of text detection methods on the CertDoc dataset.

[0060] Table 2. Performance comparison of text detection methods on the MTWI dataset.

[0061] To visually demonstrate the performance of the method, a portion of the detection results from the CertDoc dataset were selected for comparison. The results show that the method of this invention can accurately segment dense text regions, completely capture multi-scale text in certificates, and successfully associate scattered characters in wide-spaced text.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting certificate text based on spatial calibration and feature fusion, characterized in that, include: S1. Obtain the image of the certificate to be detected. X The image of the certificate to be tested X Input the feature extraction network based on PP-HGNet, and perform multi-scale feature extraction through feature initialization and hierarchical HG Block stacking to obtain feature maps of different scales; S2. Enhance the feature maps of multiple different scales based on spatial calibration. Through global context extraction, shape self-calibration and local detail fusion, obtain multiple calibration feature maps of different scales. Then, stitch the multiple calibration feature maps of different scales together in the channel dimension to output a feature map that integrates multi-scale information. S3. The feature map integrating multi-scale information is fused based on multi-scale dilated convolution. The receptive field is expanded by multi-branch dilated convolution with customized hierarchical dilation rate. Adaptive weighted fusion is performed by combining channel attention and spatial attention to output an enhanced fused feature map. S4. Perform text region detection processing on the enhanced fusion feature map, generate text region probability map and threshold map through parallel branching, and obtain text region binary map through differentiable binarization operation to realize certificate text detection.

2. The certificate text detection method based on spatial calibration and feature fusion according to claim 1, characterized in that, Step S1 includes: S11. Obtain the image of the certificate to be inspected. The images of certificates to be tested Perform feature initialization, after After convolutional downsampling, it splits into two branches. The first branch passes through two... Convolution completes feature refinement; the second branch goes through one... The max pooling layer concatenates the two outputs and then passes them through a... After convolution and double downsampling, the output size is The initial feature map; S12. Input the initial feature map into the backbone network. The backbone network contains four HG Block stages, wherein: The first HG Block stage contains one HG Block module, with the initial feature map as input and an output size of [size missing]. First feature map ; The second HG Block stage contains one HG Block module, with the first feature map as input. The output size is Second feature map ; The third HG Block stage consists of three HG Block modules, with the second feature map as the input. The output size is The third feature map ; The fourth HG Block stage contains one HG Block module, with the third feature map as input. The output size is The fourth feature map .

3. The certificate text detection method based on spatial calibration and feature fusion according to claim 1, characterized in that, In step S12, the HG Block module serves as the core feature extraction unit, employing dense... The structural design and extraction process of convolution and residual connections include: In 5 groups Convolution is a parallel branch, and each convolution is paired with Batch Normalization (BN) and ReLU activation functions. Information integration of local texture and global structure is achieved through multi-branch feature fusion. The feature fusion process is as follows: in, The input feature map for the HG Block. Indicates the first Group convolution, Output for each convolutional branch, This is a channel-level splicing operation. The fused feature map output by HG Block; pass After convolution completes channel compression, a residual connection is established with the input feature map.

4. The certificate text detection method based on spatial calibration and feature fusion according to claim 1, characterized in that, Step S2 includes: S21. Apply horizontal and vertical pooling to the first feature map respectively. Second feature map Third feature map and the fourth feature map Dimensional aggregation is performed to generate vectors in both horizontal and vertical directions. Then, rectangular attention regions are modeled using broadcast addition to capture the global structure of the entire row and column of text. S22. By optimizing the matching degree between the attention region and the actual shape of the text through spatial calibration, the background interference is significantly reduced, the adjacent boundaries of dense text are distinguished, and the calibrated attention map is obtained. The spatial calibration process is as follows: in, This represents the initial rectangular attention map. This represents the Sigmoid function. Represents the ReLU activation function. This represents the calibrated attention map; S23. Apply depthwise separable convolution to the first feature map. Second feature map Third feature map and the fourth feature map Local detail features are extracted, and the calibrated attention map is weighted and fused with the corresponding local detail features. Residual connections and a multilayer perceptron are then used for feature refinement, outputting multiple single-scale calibration feature maps, including a first single-scale calibration feature map. Second single-scale calibration feature map Third single-scale calibration feature map and the fourth single-scale calibration feature map ; S24. Calculate the first single-scale calibration feature map. Second single-scale calibration feature map Third single-scale calibration feature map and the fourth single-scale calibration feature map The feature maps are concatenated along the channel dimension to output a feature map that integrates multi-scale information. .

5. The certificate text detection method based on spatial calibration and feature fusion according to claim 4, characterized in that, Step S23 includes: S231. Apply depthwise separable convolution to the first feature map. Second feature map Third feature map and the fourth feature map Extract local detail features, preserving text strokes and fine edge textures; S232. By using the Hadamard product, the calibrated attention map is fused with local detail features element by element in a weighted manner, allowing global structural information to accurately guide the effective area of ​​local details. S233. Residual connections and MLP layers are introduced. Residual connections prevent the loss of feature information during the fusion process. MLP layers further refine the feature dimensions and improve the feature reuse capability through nonlinear transformation, and finally output multiple single-scale calibration feature maps.

6. The certificate text detection method based on spatial calibration and feature fusion according to claim 1, characterized in that, Step S3 includes: S31. The feature map integrating multi-scale information Three hierarchical dilated convolutional branches are input in parallel to adapt to the needs of small-scale, large-scale and wide-space text detection, and three branch feature maps are output. S32. Concatenate the three branch feature maps along the channel dimension to obtain a multi-branch fused feature map. ; S33, Multi-branch fusion feature map Channel attention weighting and spatial attention weighting are performed, and the features after dual attention weighting are convolutionally processed to unify the channel dimensions, outputting an enhanced fusion feature map. .

7. The certificate text detection method based on spatial calibration and feature fusion according to claim 6, characterized in that, In step S31, the three hierarchical dilated convolution branches include a small dilation rate branch, a large dilation rate branch, and a wide-spacing branch, wherein: The small void ratio branch uses void ratios of 1 and 3. By combining convolutions sequentially, the receptive field is appropriately expanded while preserving details, focusing on preserving the stroke details and edge features of small texts and avoiding the loss of details; The high void ratio branch uses void ratios of 6 and 9. Convolutional sequences are combined to cover the entire text area with a larger receptive field; Wide-spaced branches use a hole ratio of 3 for wide-spaced text. Convolution combined with interval attention mechanism strengthens the correlation between characters within the same text line and weakens the feature fragmentation problem caused by wide spacing. The core of interval attention is to use bar convolution in the line direction to weighted aggregate the features in the text line direction and identify characters belonging to the same text line under wide spacing.

8. The certificate text detection method based on spatial calibration and feature fusion according to claim 6, characterized in that, Step S33 includes: S331, Channel Attention Module for Multi-Branch Fusion Feature Map Global average pooling is performed to compress spatial dimension features into channel dimension global statistical information, and then... Convolution and Sigmoid activation generate channel weight maps This enables feature weighting along the channel dimension. S332, Spatial Attention Module for Multi-Branch Fusion Feature Maps conduct Convolutional dimensionality reduction and Sigmoid activation generate a spatial weight map. Complete the feature weighting of the spatial dimension; S333, Combine the channel-weighted features and the spatial-weighted features through... Convolution unifies the channel dimensions and outputs an enhanced fusion feature map that combines detail and semantics. .

9. The certificate text detection method based on spatial calibration and feature fusion according to claim 1, characterized in that, Step S4 includes: S41. The enhanced fusion feature map After convolution and upsampling operations, the number of channels in the feature map is adjusted, and two parallel branches generate probability maps of the same size as the original image. and threshold map ; S42, in the probability graph The last layer uses Sigmoid activation, with output values ​​between 0 and 1, representing the probability that each pixel belongs to the text region; S43, in the threshold map The final layer uses Sigmoid activation, with output values ​​between 0 and 1, representing the threshold for each pixel, used in conjunction with the probability map. The probability values ​​are compared to determine whether the pixel belongs to the text region; S44. Through differentiable binarization, based on the probability graph... With threshold map Difference calculation yields a binary image of the text region. The formula is as follows: in, This indicates the coordinate position in the feature map. Represents a probability graph In coordinates Pixel value at that location, Threshold map In coordinates Pixel value at that location, Indicates the magnification factor. Binary graph representing text region Those seated The output value at that location.

10. A certificate text detection system based on spatial calibration and feature fusion, implemented according to the certificate text detection method based on spatial calibration and feature fusion as described in any one of claims 1-9, characterized in that, include: The system comprises a feature extraction module based on PP-HGNet, a feature enhancement module based on spatial calibration, a feature fusion module based on multi-scale dilated convolution, and a text region detection module, among which: The feature extraction module based on PP-HGNet is used to acquire the certificate image to be detected. X The image of the certificate to be tested X Input the feature extraction network based on PP-HGNet, and perform multi-scale feature extraction through feature initialization and hierarchical HG Block stacking to obtain feature maps of different scales; The spatial calibration-based feature enhancement module is used to enhance the feature maps of multiple different scales based on spatial calibration. Through global context extraction, shape self-calibration and local detail fusion, multiple calibration feature maps of different scales are obtained, and the multiple calibration feature maps of different scales are stitched together in the channel dimension to output a feature map integrating multi-scale information. The feature fusion module based on multi-scale dilated convolution is used to fuse the feature map integrating multi-scale information based on multi-scale dilated convolution. It expands the receptive field by multi-branch dilated convolution with customized hierarchical dilation rate, and combines channel attention and spatial attention for adaptive weighted fusion to output an enhanced fused feature map. The text region detection module is used to perform text region detection processing on the enhanced fusion feature map. It generates a text region probability map and a threshold map through parallel branches, and obtains a text region binary map through a differentiable binarization operation, thereby realizing certificate text detection.

Images