Ancient character detection method based on multi-scale target perception and adaptive query selection

Abstract

The application discloses a hieroglyphic detection method based on multi-scale target perception and adaptive query selection, and relates to the technical field of hieroglyphic detection. The method comprises the following steps: constructing a hieroglyphic detection model, wherein the hieroglyphic detection model comprises a backbone network, a multi-scale target perception device, a cross-scale feature fusion module based on CNN, a multi-scale adaptive query selector and a Transformer decoder connected in sequence; training the hieroglyphic detection model by using a hieroglyphic image training set to obtain a trained hieroglyphic detection model; and inputting a hieroglyphic image to be detected into the trained hieroglyphic detection model to obtain a hieroglyphic detection result. The application solves the problems that an existing end-to-end detection method has insufficient representation ability for small and fuzzy characters in cultural relic images and initial target query is inaccurate under a complex background, and effectively improves the accuracy and robustness of hieroglyphic detection.

Description

Technical Field

[0001] This invention relates to the field of ancient script detection technology, specifically to an ancient script detection method based on multi-scale target perception and adaptive query selection. Background Technology

[0002] Detecting ancient characters in artifact images is crucial for interpreting information about ancient societies. In recent years, object detection methods, represented by the YOLO series, have made significant progress, but they typically rely on Non-Maximum Suppression (NMS), which slows down inference. The Transformer series of real-time detection methods offers an end-to-end detection paradigm that does not require NMS, demonstrating excellent performance in both speed and accuracy.

[0003] However, real-time detection Transformer methods face two major challenges when applied to ancient character detection tasks: First, artifact images often contain numerous small and blurry characters. To reduce computational costs, real-time detection Transformer methods primarily rely on low-resolution feature maps for feature interaction, resulting in insufficient representation of fine-grained details and difficulty in accurately locating character boundaries. Second, complex background noise in artifact images is similar to and intertwined with character strokes. Real-time detection Transformer methods, relying solely on a global query selection mechanism based on classification confidence, are prone to incorrectly selecting background noise as the initial target query, affecting detection accuracy. Therefore, there is an urgent need in this field for an ancient character detection method that can both enhance multi-scale feature representation capabilities and improve the accuracy of the initial target query. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides an ancient character detection method based on multi-scale target perception and adaptive query selection.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: The ancient character detection method based on multi-scale target perception and adaptive query selection includes the following steps: An ancient character detection model is constructed, comprising a backbone network, a multi-scale target perceptron, a CNN-based cross-scale feature fusion module, a multi-scale adaptive query selector, and a Transformer decoder, connected sequentially. The backbone network extracts multi-scale feature maps; the multi-scale target perceptron enhances the multi-scale feature maps through target perception region attention; the CNN-based cross-scale feature fusion module fuses the enhanced multi-scale feature maps; the multi-scale adaptive query selector performs multi-scale adaptive query selection on the fused enhanced multi-scale feature maps using an adaptive weight assignor to generate an initial target query; and the Transformer decoder generates the ancient character detection result based on the initial target query. The ancient character detection model is trained using a training set of ancient character images to obtain the trained ancient character detection model; The ancient character image to be detected is input into the trained ancient character detection model to obtain the ancient character detection results.

[0006] Furthermore, the data processing procedure of the multi-scale target perceptron is as follows: the multi-scale feature map is projected onto a unified channel dimension through a 1×1 convolution to obtain the projected feature map; the projected feature map is then subjected to a 1×1 convolution to generate a query matrix, a key matrix, and a value matrix, and a reshaping layer is used to divide the query matrix, key matrix, and value matrix into query fragments, key fragments, and value fragments along the height dimension, respectively; a multilayer perceptron is used to obtain the predicted target score heatmap from the projected feature map, and the predicted target score heatmap is converted into a target perception bias using the sigmoid function; target perception region attention is calculated based on the query fragment, key fragment, value fragment, and target perception bias; based on the target perception region attention, the multi-scale feature map is enhanced through residual connections, layer normalization, and a feedforward network, and the enhanced multi-scale feature map is output.

[0007] Furthermore, when using a multilayer perceptron to obtain the predicted target score heatmap from the projected feature map, a target perception loss is constructed to supervise the obtained predicted target score heatmap; the expression for the target perception loss is:

[0008] in: Perceive loss for the target. for The total number of pixels in For the first The multi-scale feature maps correspond to two-dimensional Gaussian soft labels generated from the ground bounding boxes. =3, 4, 5, Number the pixels. for The first in 1 pixel, It is a logarithmic function. For the sigmoid function, for The first in 1 pixel, For the first Heatmap of predicted target scores corresponding to multi-scale feature maps.

[0009] Furthermore, the attention of the target-aware region is calculated based on the query fragment, key fragment, value fragment, and target-aware bias, and its expression is:

[0010] in: For the first Attention to the target perception region corresponding to the multi-scale feature map. The softmax activation function is used. For the first The query fragment corresponding to the multi-scale feature map. For the first Key segments corresponding to multi-scale feature maps This is the matrix transpose operator. For channel dimension, For learning weights, For the first The target perception bias corresponding to the multi-scale feature map. For the first Value segments corresponding to multi-scale feature maps.

[0011] Furthermore, the data processing procedure of the multi-scale adaptive query selector is as follows: the fused enhanced multi-scale feature map is flattened along the spatial dimension into a feature sequence by an adaptive weight allocator, where each spatial location corresponds to a feature token, which is used as the input of a lightweight classification head and outputs the classification confidence through a fully connected layer; the average of the top-M maximum confidences is calculated based on the classification confidence and determined as the candidate quality; the candidate quality is normalized to obtain the assigned weights, and under the constraint of a fixed total number of queries, the number of queries at each scale is adaptively calculated according to the assigned weights; within each scale, according to the number of queries at each scale, a corresponding number of feature vectors are selected as the target queries for that scale in descending order of classification confidence, and the target queries of all scales are concatenated to generate the initial target query O.

[0012] Furthermore, the candidate quality is normalized to obtain the assigned weights, the expression of which is:

[0013] in: For the first The weights are assigned to the multi-scale feature maps. For the first Candidate quality corresponding to multi-scale feature maps The candidate quality corresponding to the third multi-scale feature map. The candidate quality corresponding to the 4th multi-scale feature map, The candidate quality corresponding to the 5th multi-scale feature map, This is a constant used to prevent division by zero.

[0014] Furthermore, under the constraint of a fixed total number of queries, the number of queries for each scale is adaptively calculated based on the assigned weights, and its expression is:

[0015] in: For the first The number of queries corresponding to each multi-scale feature map. For the first The weights are assigned to the multi-scale feature maps. This represents the total number of queries. This is a round-down operation. This refers to the numbering of the multi-scale feature maps. For the first The weights are assigned to the multi-scale feature maps. For index symbols.

[0016] The beneficial effects of this invention are as follows: (1) By constructing a multi-scale target perceptron, this invention introduces target perception region attention at multiple scales, enabling the ancient character detection model to focus on character regions more accurately, and significantly improving the multi-scale feature representation ability of small characters and fuzzy characters. (2) This invention constructs a multi-scale adaptive query selector and uses an adaptive weight allocator to perform multi-scale adaptive query selection on the fused enhanced multi-scale feature map to generate an initial target query. The selection is combined with the classification confidence, which effectively reduces the probability of background noise being mistakenly selected as the initial target query and improves the detection accuracy. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the ancient character detection method based on multi-scale target perception and adaptive query selection; Figure 2 This is a schematic diagram of the ancient script detection model structure; Figure 3 This is a schematic diagram illustrating the computational process of attention in the target perception region of a multi-scale target perceptron. Detailed Implementation

[0018] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0019] like Figure 1 As shown, the ancient character detection method based on multi-scale target perception and adaptive query selection includes steps S1-S3, as detailed below: S1. Construct an ancient character detection model, which includes a backbone network, a multi-scale object perceptron, a CNN-based cross-scale feature fusion module, a multi-scale adaptive query selector, and a Transformer decoder connected in sequence. Figure 2 As shown; the backbone network is used to extract multi-scale feature maps; the multi-scale target perceptron is used to enhance the multi-scale feature maps through target perception region attention; the CNN-based cross-scale feature fusion module is used to fuse the enhanced multi-scale feature maps; the multi-scale adaptive query selector is used to perform multi-scale adaptive query selection on the fused enhanced multi-scale feature maps through an adaptive weight allocator to generate the initial target query; the Transformer decoder is used to generate ancient character detection results based on the initial target query.

[0020] In an optional embodiment of the present invention, the HGNetv2 network is used as the backbone network. The backbone network receives ancient text images as input and extracts visual features of the ancient text images layer by layer through a series of convolution, pooling, and downsampling operations. The backbone network outputs three feature maps at different scales, denoted as S3, S4, and S5, whose sizes typically correspond to 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. Among them, the high-resolution S3 preserves rich spatial details (such as the edges and textures of small characters), while the low-resolution S5 encodes stronger semantic information (such as the overall region response of characters). This multi-scale output design lays the foundation for subsequent modules to simultaneously capture fine-grained and macroscopic features.

[0021] like Figure 3 As shown, the data processing procedure of the multi-scale target perceptron is as follows: the multi-scale feature map is projected onto a unified channel dimension through a 1×1 convolution to obtain the projected feature map; the projected feature map is then subjected to a 1×1 convolution to generate the query matrix. Key matrix Sum matrix Furthermore, the query matrix, key matrix, and value matrix are divided into query fragments along the height dimension through a reshaping layer. Key fragments Sum value fragment The method involves using a multilayer perceptron to obtain a heatmap of the predicted target score from the projected feature map, and then using a sigmoid function to convert the heatmap into a target perception bias. Attention to the target perception region is calculated based on the query fragment, key fragment, value fragment, and target perception bias. Based on this attention, the multi-scale feature map is enhanced through residual connections, layer normalization, and a feedforward network, outputting the enhanced multi-scale feature map. Specifically, this invention includes four reshaping layer operations, all of which are identical.

[0022] This invention utilizes a multilayer perceptron to obtain a predicted target score heatmap from a projected feature map, and supervises the obtained predicted target score heatmap by constructing a target perception loss; the expression for the target perception loss is:

[0023] in: Perceive loss for the target. for The total number of pixels in For the first The multi-scale feature maps correspond to two-dimensional Gaussian soft labels generated from the ground bounding boxes. =3, 4, 5, Number the pixels. for The first in 1 pixel, It is a logarithmic function. For the sigmoid function, for The first in 1 pixel, For the first Heatmap of predicted target scores corresponding to multi-scale feature maps.

[0024] This invention calculates the attention of the target perception region based on query fragments, key fragments, value fragments, and target perception bias, and its expression is:

[0025] in: For the first Attention to the target perception region corresponding to the multi-scale feature map. The softmax activation function is used. For the first The query fragment corresponding to the multi-scale feature map. For the first Key segments corresponding to multi-scale feature maps This is the matrix transpose operator. For channel dimension, For learning weights, For the first The target perception bias corresponding to the multi-scale feature map. For the first Value segments corresponding to multi-scale feature maps.

[0026] The data processing procedure of the CNN-based cross-scale feature fusion module is as follows: Upsampling, downsampling, and convolution operations are used to adjust the enhanced multi-scale feature maps to a uniform spatial size. Then, channel concatenation and fusion are performed to integrate high-resolution fine-grained details with low-resolution strong semantic information. Cross-scale information interaction is achieved through top-down and bottom-up fusion processes, ultimately outputting three fused enhanced multi-scale feature maps, providing a more robust multi-scale feature representation for the subsequent adaptive query selector.

[0027] The data processing of the multi-scale adaptive query selector is as follows: The fused enhanced multi-scale feature map is flattened along the spatial dimension into a feature sequence using an adaptive weight allocator, where each spatial location corresponds to a feature token. This token is used as input to a lightweight classification head, and a fully connected layer outputs the classification confidence score. The average of the Top-M highest confidence scores is calculated based on the classification confidence score, and this average is used as the candidate quality. The candidate quality is normalized to obtain the assigned weights, and under the constraint of a fixed total number of queries, the number of queries at each scale is adaptively calculated based on the assigned weights. Within each scale, based on the number of queries at each scale and the classification confidence score from high to low, a corresponding number of feature vectors are selected as the target queries for that scale. The target queries from all scales are then concatenated to generate the initial target query O.

[0028] This invention calculates the average of the Top-M highest confidence scores based on classification confidence and determines it as the candidate quality. The expression for this is:

[0029] in: For the first Candidate quality corresponding to multi-scale feature maps For hyperparameters, For the first The first multi-scale feature map corresponding to the first The classification confidence of each feature token.

[0030] This invention normalizes the candidate quality to obtain the allocation weight, the expression of which is:

[0031] in: For the first The weights are assigned to the multi-scale feature maps. For the first Candidate quality corresponding to multi-scale feature maps The candidate quality corresponding to the third multi-scale feature map. The candidate quality corresponding to the 4th multi-scale feature map, The candidate quality corresponding to the 5th multi-scale feature map, This is a constant used to prevent division by zero.

[0032] This invention, under the constraint of a fixed total number of queries, adaptively calculates the number of queries for each scale based on the assigned weights, as expressed in the following expression:

[0033] in: For the first The number of queries corresponding to each multi-scale feature map. For the first The weights are assigned to the multi-scale feature maps. This represents the total number of queries. This is a round-down operation. This refers to the numbering of the multi-scale feature maps. For the first The weights are assigned to the multi-scale feature maps. For index symbols.

[0034] The data processing procedure of the Transformer decoder is as follows: the initial target query O is input into the Transformer decoder, and the ancient character detection result is output. The Transformer decoder consists of 6 sequentially connected decoding layers and a prediction head. Each decoding layer includes a self-attention, cross-attention, and feedforward network connected in sequence. The initial target query is updated layer by layer in the decoder, and the prediction head outputs the character bounding box coordinates, category, and confidence score based on the updated target query.

[0035] S2. Train the ancient character detection model using the ancient character image training set to obtain the trained ancient character detection model.

[0036] In an optional embodiment of the present invention, the ancient character image training set includes multiple ancient character images and corresponding annotation information for each ancient character image, namely, category labels and bounding box location labels. The bounding box location labels are used to characterize the position of the ancient character target in the image and can be represented as center point coordinates, width, and height.

[0037] Before training, the ancient character images in the training set are preprocessed. Preprocessing includes image resizing, pixel normalization, random flipping, and random cropping. These preprocessing operations enhance the diversity of the training samples. Then, the preprocessed ancient character images are input into the ancient character detection model to predict the character targets and their location information. Finally, the training loss is calculated based on the prediction results output by the ancient character detection model and the ground truth annotation information in the ancient character image training set. Backpropagation and parameter updates are then performed on the ancient character detection model based on the training loss until a preset number of training epochs are met, resulting in the trained ancient character detection model.

[0038] During training, the total loss function of the ancient script detection model includes classification loss. Bounding box loss Generalized intersection and comparison loss Fine-grained distribution refinement loss Decoupled distillation focus loss and target perception loss Its expression is:

[0039] in: =1, =5, =2, =0.15, =1.5, =0.8.

[0040] Classification loss The expression used to constrain the consistency between the model's predicted class confidence and the true class label is:

[0041] in: This represents the number of bounding boxes in the image of the ancient characters. The bounding box number in the ancient character image. The first in ancient Chinese character images Category labels for each bounding box. The ancient character image output by the ancient character detection model is the first one. The predicted probability value of each bounding box. This is a balancing factor used to adjust the weights of negative samples. This is the focusing parameter, used to focus on difficult-to-distinguish negative samples.

[0042] Bounding box loss This constraint, used to limit the difference in coordinate position between the predicted bounding box and the true bounding box, making the predicted box closer to the actual ancient text region, is expressed as follows:

[0043] in: The ancient character image output by the ancient character detection model is the first one. The predicted location values ​​of each bounding box The first in ancient Chinese character images The location label of the bounding box.

[0044] Generalized intersection and comparison loss This constraint, used to limit the spatial overlap between the predicted bounding box and the ground truth bounding box, improves the accuracy of the ancient character detection model in locating the boundaries of ancient character targets. Its expression is:

[0045] in: for and Generalized intersection and comparison between them.

[0046] Fine-grained distribution refinement loss The expression used to improve the accuracy of the distribution prediction of the four sides of the bounding box is:

[0047]

[0048]

[0049]

[0050] in, This represents the number of decoder layers in the Transformer decoder. This refers to the decoder layer number in the Transformer decoder. For the first The intersection-union ratio (IoU) of each predicted bounding box and its corresponding ground truth bounding box. To and The weight corresponding to the left adjacent index, For cross-entropy loss, For the first In the layer decoder The probability distribution of the four sides corresponding to each predicted bounding box. To and Adjacent left index, This is a relative offset. To and The weight corresponding to the right adjacent index, To and Adjacent right index, For weighted functions, This is a hyperparameter that controls the upper limit of the function. For the hyperparameters controlling the curvature of the function, It is the maximum index of the discrete interval.

[0051] Decoupled distillation focus loss This method utilizes high-quality prediction distributions to distill constraints on the bounding box distribution learning process and highlights hard-to-learn samples through focus weighting, thereby enhancing the model's localization distribution learning ability. Its expression is:

[0052]

[0053]

[0054] in: For Kullback-Leibler divergence, For temperature parameters, This represents the number of decoder layers in the Transformer decoder. This refers to the decoder layer number in the Transformer decoder. This represents the number of predicted bounding boxes that can be matched with the true bounding boxes. The total number of queries is the number of predicted bounding boxes that cannot be matched with the true bounding boxes. , For the last layer decoder The probability distribution of the four sides corresponding to each predicted bounding box. As the first weight, As the second weight, For the first The classification confidence level corresponding to each predicted bounding box.

[0055] S3. Input the ancient character image to be detected into the trained ancient character detection model to obtain the ancient character detection results.

[0056] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for detecting ancient characters based on multi-scale target perception and adaptive query selection, characterized in that, Includes the following steps: An ancient character detection model is constructed, comprising a backbone network, a multi-scale target perceptron, a CNN-based cross-scale feature fusion module, a multi-scale adaptive query selector, and a Transformer decoder, connected sequentially. The backbone network is used to extract multi-scale feature maps; the multi-scale target perceptron is used to enhance the multi-scale feature maps through target perception region attention; the CNN-based cross-scale feature fusion module is used to fuse the enhanced multi-scale feature maps; and the multi-scale adaptive query selector is used to perform multi-scale adaptive query selection on the fused enhanced multi-scale feature maps through an adaptive weight allocator to generate an initial target query. The Transformer decoder is used to generate ancient character detection results based on the initial target query; The ancient character detection model is trained using a training set of ancient character images to obtain the trained ancient character detection model; The ancient character image to be detected is input into the trained ancient character detection model to obtain the ancient character detection results.

2. The ancient character detection method based on multi-scale target perception and adaptive query selection according to claim 1, characterized in that, The data processing procedure of the multi-scale target perceptron is as follows: the multi-scale feature map is projected onto a unified channel dimension through a 1×1 convolution to obtain the projected feature map; the projected feature map is then subjected to a 1×1 convolution to generate a query matrix, a key matrix, and a value matrix, and a reshaping layer is used to divide the query matrix, key matrix, and value matrix into query segments, key segments, and value segments along the height dimension, respectively; a multilayer perceptron is used to obtain the predicted target score heatmap from the projected feature map, and the Sigmoid function is used to convert the predicted target score heatmap into a target perception bias; Calculate the target perception region attention based on query fragments, key fragments, value fragments, and target perception bias; Based on target perception region attention, the multi-scale feature map is enhanced through residual connections, layer normalization and feedforward network, and the enhanced multi-scale feature map is output.

3. The ancient character detection method based on multi-scale target perception and adaptive query selection according to claim 2, characterized in that, When using a multilayer perceptron to obtain a predicted target score heatmap from the projected feature map, a target perception loss is constructed to supervise the obtained predicted target score heatmap; the expression for the target perception loss is: in: Perceive loss for the target. for The total number of pixels in For the first The multi-scale feature maps correspond to two-dimensional Gaussian soft labels generated from the ground bounding boxes. =3, 4, 5, Number the pixels. for The first in 1 pixel, It is a logarithmic function. For the sigmoid function, for The first in 1 pixel, For the first Heatmap of predicted target scores corresponding to multi-scale feature maps.

4. The ancient character detection method based on multi-scale target perception and adaptive query selection according to claim 2, characterized in that, The attention of the target-aware region is calculated based on the query fragment, key fragment, value fragment, and target-aware bias. The expression is as follows: in: For the first Attention to the target perception region corresponding to the multi-scale feature map. The softmax activation function is used. For the first The query fragment corresponding to the multi-scale feature map For the first Key segments corresponding to multi-scale feature maps This is the matrix transpose operator. For channel dimension, For learning weights, For the first The target perception bias corresponding to the multi-scale feature map. For the first Value segments corresponding to multi-scale feature maps.

5. The ancient character detection method based on multi-scale target perception and adaptive query selection according to claim 1, characterized in that, The data processing of the multi-scale adaptive query selector is as follows: the fused enhanced multi-scale feature map is flattened into a feature sequence along the spatial dimension by an adaptive weight allocator, where each spatial location corresponds to a feature token, which is used as the input of a lightweight classification head, and the classification confidence is output through a fully connected layer. The average of the top-M highest confidence scores is calculated based on the classification confidence score, and this average is determined as the candidate quality. The candidate quality is normalized to obtain the assigned weights, and under the constraint of a fixed total number of queries, the number of queries for each scale is adaptively calculated based on the assigned weights. Within each scale, based on the number of queries at each scale, a corresponding number of feature vectors are selected as the target queries for that scale according to the classification confidence from high to low. The target queries from all scales are then concatenated to generate the initial target query O.

6. The ancient character detection method based on multi-scale target perception and adaptive query selection according to claim 5, characterized in that, The candidate quality is normalized to obtain the assigned weights, and the expression is as follows: in: For the first The weights are assigned to each multi-scale feature map. For the first The candidate quality corresponding to each multi-scale feature map. The candidate quality corresponding to the third multi-scale feature map. The candidate quality corresponding to the 4th multi-scale feature map, The candidate quality corresponding to the 5th multi-scale feature map, This is a constant used to prevent division by zero.

7. The ancient character detection method based on multi-scale target perception and adaptive query selection according to claim 5, characterized in that, Under the constraint of a fixed total number of queries, the number of queries for each scale is adaptively calculated based on the assigned weights, and its expression is: in: For the first The number of queries corresponding to each multi-scale feature map. For the first The weights are assigned to each multi-scale feature map. This represents the total number of queries. This is a round-down operation. This refers to the numbering of the multi-scale feature maps. For the first The weights are assigned to each multi-scale feature map. For index symbols.

Images