Whole graph Chinese character segmentation and recognition method and system

Abstract

The invention belongs to the field of computer vision and artificial intelligence, relates to a whole image Chinese character segmentation recognition method and system, and aims to solve the problems of low accuracy and poor robustness of complex calligraphy image segmentation recognition. The method comprises the following steps: constructing and marking a data set containing multiple books, multiple shooting scenes and multiple quality images; stylized data augmentation is carried out through a cWCT module; training a text area detection model based on the target detection network; training a single character classification model based on a deep convolutional network; text area detection is carried out by adopting a bimodal strategy combining a whole image and a sliding window; and after NMS filtering and spatial sorting, a single character classification model is integrated for identification, and an ordered text is output. The method has the capability of processing complex formats and super thousand long inscription, can accurately analyze complex scenes containing large and small characters, multiple book bodies, noisy points and deformation, and is stable in performance under various imaging conditions such as black and white, color and the like.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method and system for segmenting and recognizing Chinese characters in a whole image. Background Technology

[0002] Chinese calligraphy is an art form with a long history, preserving a vast collection of precious cultural relics such as inscriptions, rubbings, couplets, and plaques. Digital preservation and content identification of these artifacts are of great significance for the research, inheritance, and popularization of historical culture. While optical character recognition (OCR) technology has matured in recognizing standard printed characters with the advancement of artificial intelligence, the automatic recognition of complex, full-page calligraphy works still faces significant challenges.

[0003] The existing technology mainly faces the following challenges:

[0004] 1. High complexity of layout: Calligraphic images such as inscriptions, rubbings, stone scriptures, couplets, and plaques often exhibit irregular layout characteristics, such as slanted rows and columns, misaligned character groups, partial cross-column insertion, and mixed arrangement of paragraphs and inscriptions, making it difficult to accurately analyze their spatial structure by relying solely on traditional row and column division.

[0005] 2. Difficulty in recognizing multiple calligraphic styles within a single work: A single calligraphy work often contains a mixture of various calligraphic styles such as seal script, cursive script, clerical script, regular script, and running script, and contains a large number of variant characters and variant characters. This causes a single recognition model to face great challenges in the same scene due to the large differences in form and style switching, which places high demands on the model's generalization ability and the completeness of the character library.

[0006] 3. Image quality and interference factors: Rubbings of stone inscriptions are often accompanied by background noise such as stone flowers, cracks, and stains; outdoor stone inscriptions are affected by light, shadows, and weathering; perspective distortion and uneven brightness caused by shooting all interfere with the accurate positioning and recognition of text.

[0007] 4. Difficulty in covering the full-image recognition needs of long inscriptions: Although existing multi-character detection and recognition methods are effective in specific image scenarios such as ancient printed texts, relatively regular layouts and short texts, they are often difficult to apply stably to long images such as rubbings of stone inscriptions.

[0008] 5. Difficulty in detecting multi-scale targets and dense small characters: Calligraphic images such as rubbings of stone inscriptions often contain large characters such as the inscription title and small characters in the main text, and even extremely small characters such as annotations. Moreover, different characters vary significantly in stroke thickness, spacing, and contrast. Under conditions of dense arrangement and background noise, candidate boxes are prone to missed detection, overlap, and false merging. The detection process is required to have high recall capability for multi-scale targets and a reliable filtering mechanism to support efficient reading from the whole image to structured text. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention provides a method for segmenting and recognizing Chinese characters in a whole image, employing the following technical solution, including the following steps:

[0010] S1. Construct a dataset containing multiple calligraphic styles, multiple shooting scenes, and multiple quality images, and label the text regions in the calligraphy images;

[0011] S2. The dataset is augmented using the Content-Whitening and Color Transformation module (hereinafter referred to as the cWCT module) to generate diverse training samples;

[0012] S3. Based on the object detection network, train the text region detection model using the augmented dataset;

[0013] S4. Training a single-character classification model based on a deep convolutional network;

[0014] S5. A dual-modal strategy combining full image and sliding window is adopted to perform text region detection using the text region detection model to obtain a set of detection boxes;

[0015] S6. After performing non-maximum suppression (NMS) multi-level filtering on the detection box, cluster it by column and sort it within the column according to its spatial position, integrate the single-character classification model for recognition, and generate an ordered text sequence.

[0016] Preferably, the data augmentation process in step S2 includes basic data augmentation and advanced data augmentation based on a style transfer network. The style transfer network is used to decouple and fuse the content features of the content image and the style features of the style image to generate training samples across calligraphic styles. The style transfer network adopts an invertible residual network based on CAP-VSTNet to achieve decoupling and fusion of the content features of the content image and the style features of the style image, and generates training samples across calligraphic styles through the cWCT module.

[0017] Preferably, in step S5, the text region detection model is based on the YOLO network architecture and adopts a dual-modal detection framework that combines whole-image detection and sliding window detection: whole-image detection is used to obtain candidate text boxes with global structural constraints, and sliding window detection is used to supplement small-scale candidate text boxes under local high-resolution view. The candidate boxes output by the two modalities are uniformly mapped to the original image coordinate system and merged into the same candidate set. Cross-modal global nonmaximum suppression is performed based on the candidate box confidence to realize duplicate box elimination and result fusion, thereby outputting a unified candidate box set.

[0018] Preferably, after model integration, multi-level post-processing and adaptive structural filtering are performed on the candidate box set output in step S5 to obtain a stable final character candidate box set. The multi-level post-processing and adaptive structural filtering include at least the following steps:

[0019] Apply boundary constraints to the candidate box coordinates to ensure they fall within the coordinate range of the original image;

[0020] Minimum candidate boxes smaller than a preset minimum size threshold are removed to suppress image noise and scanning interference;

[0021] An inclusion relationship filtering mechanism based on area hierarchy and confidence level gating is introduced to adaptively filter inner and outer candidate boxes that meet the preset inclusion judgment conditions: if the confidence level of the outer candidate box reaches the preset threshold, the non-subject candidate boxes contained within it are filtered out to avoid misidentifying Chinese radicals, components, and other substructures as independent characters; if the confidence level of the outer candidate box is lower than the preset threshold and there are candidate boxes with higher confidence levels within it, the outer candidate box is deleted and the high-confidence candidate boxes within it are retained, thereby effectively suppressing the false detection and merging of adjacent small characters when the image is blurred.

[0022] Preferably, during the text region cropping process, a coordinate transformation relationship is established between the display coordinate system and the original image pixel coordinate system: the horizontal and vertical scaling factors are determined according to the original image pixel size and the display rendering size, and the candidate box coordinates under the display coordinate system are mapped to the original image pixel coordinate system according to the scaling factors to obtain the original image coordinates for cropping, thereby realizing accurate cropping of the candidate text region and generation of character images.

[0023] Preferably, the adaptive layout analysis in step S5 specifically involves: calculating the center point coordinates of the detection boxes that meet the confidence conditions; adaptively determining the text column segmentation boundary and completing the column division based on the horizontal distribution of the center points using spacing statistics and adjustable fault tolerance coefficients; generating the column order by sorting the center point vertical coordinates from top to bottom within each column; and finally determining the column order according to the preset or recognized layout direction to obtain the reading order of the full text and output the recognized text.

[0024] Preferably, the training optimization of the single-character classification model adopts a combination of regularization training strategy and learning rate scheduling strategy; wherein, a label smoothing mechanism can be introduced when constructing the loss function to alleviate class noise and overfitting, and cosine annealing learning rate scheduling is used during training to make the learning rate decay smoothly with the training process.

[0025] To address the aforementioned technical problems, this invention also provides a whole-image Chinese character segmentation and recognition system, which employs the following technical solution, including:

[0026] The annotation module is used to construct a calligraphy image dataset containing multiple calligraphic styles, multiple shooting scenes, and multiple quality images, and to annotate the text regions in the dataset.

[0027] The processing module is used to perform data augmentation processing on the dataset through the cWCT module to generate diverse training samples;

[0028] The training module is used to train a text region detection model based on an object detection network using an augmented dataset; and to train a single-character classification model based on a deep convolutional network.

[0029] The detection module is used to perform text region detection on the calligraphy image to be recognized by using a dual-modal strategy that combines the whole image and sliding window, and outputs a set of detection boxes.

[0030] The post-processing module is used to perform the following operations sequentially on the detection box set: NMS multi-level filtering; adaptive layout analysis based on the spatial distribution characteristics of the filtered detection boxes to determine the reading order; and sorting the detection boxes according to the determined reading order.

[0031] The recognition and output module is used to call the single-character classification model to recognize the single-character images cropped by the detection box and finally output ordered text.

[0032] To address the aforementioned technical problems, the present invention also provides a computer device that employs the technical solution described below, comprising a memory and a processor. The memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the above-described whole-image Chinese character segmentation and recognition method.

[0033] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium, which employs the technical solution described below. The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the above-described whole-image Chinese character segmentation and recognition method.

[0034] Compared with the prior art, the present invention has the following main advantages:

[0035] (1) Strong ability to analyze complex layouts: For complex layout structures such as the coexistence of large and small characters and the mixing of multiple calligraphic styles in stone inscriptions, this invention achieves high recall and complementary detection of characters of different scales through a dual-modal detection framework that combines whole images and sliding windows. Combined with the fusion deduplication and inclusion relationship filtering mechanism, it effectively avoids missed detection and false detection caused by the overlap and nesting of large and small character frames, and significantly improves the segmentation stability in complex mixed layout scenarios.

[0036] (2) Possesses adaptive layout analysis and intelligent post-processing capabilities: Based on the spatial distribution of detection boxes, this invention performs adaptive column division and sorting, which can accurately infer the reading order of images such as inscriptions from right to left and from top to bottom, and has good adaptability to uneven column spacing, page tilt and shooting distortion. At the same time, by integrating multi-level post-processing and a confidence-based nested box filtering mechanism, it effectively suppresses noise, removes interference boxes, and avoids false detection of radicals or adjacent small characters, thereby outputting clean and structurally correct recognition results in complex layouts.

[0037] (3) Supports automated processing of long inscriptions: This invention constructs an integrated process from image input, joint detection of whole image and sliding window, character cropping, single character recognition to layout analysis and structured text output. It can achieve continuous and stable processing in long inscriptions, epitaphs and rubbings with thousands of characters, significantly reducing the cost of manual character selection and sorting, and improving the efficiency of large-scale digital processing in the field of archaeology and cultural relics.

[0038] (4) High environmental robustness: This invention suppresses false noise detection through adaptive filtering and structured constraint mechanisms, and improves the reliability of the results by using confidence competition and spatial relationship discrimination in the fusion stage. It can effectively deal with common problems in rubbings and photographic images, such as noise interference, edge damage, uneven lighting, local blurring, deformation and perspective distortion. At the same time, this method has good adaptability and stability to various imaging conditions and material backgrounds, including black and white, color, blue background and gray background, and performs reliably on data from different sources.

[0039] (5) Significant advantages in multi-script generalization ability and overall efficiency: This invention trains the detection and recognition model based on multi-script samples and data augmentation strategies such as style transfer, enabling the system to have good generalization ability for various scripts such as seal script, cursive script, clerical script, regular script, and running script. It can also maintain stable reading performance in scenarios where large-character titles and small-character main texts coexist and the differences in character shapes are significant. Compared with the traditional processing flow that relies on manual character segmentation and reading, this invention achieves automated processing from the whole calligraphy image to structured text through whole-image sliding window detection fusion, structured filtering, and reading order restoration. In long text tasks, it can significantly reduce the degree of human intervention and greatly improve the processing efficiency, up to 50 times that of traditional methods. Attached Figure Description

[0040] To more clearly illustrate the solutions in this invention, the accompanying drawings used in the description of the embodiments of this invention will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1This is a flowchart of an embodiment of a whole-image Chinese character segmentation and recognition method according to the present invention;

[0042] Figure 2 This is a schematic diagram illustrating the use of the RoboFlow platform to perform single-word annotation during the dataset construction phase of this invention;

[0043] Figure 3 This is a schematic diagram of style transfer data augmentation based on the cWCT module of the present invention;

[0044] Figure 4 This is a schematic diagram of the dual-modal detection framework using the whole image and sliding window of the present invention; wherein, Figure 4 (a) shows the results of the whole image detection; Figure 4 (b) shows the results of the sliding window detection; Figure 4 (c) shows the results after dual-modal fusion detection;

[0045] Figure 5 This is a schematic diagram of the NMS multi-level filtering mechanism using the present invention; wherein, Figure 5 (a) shows that when the outer large box has a high confidence level, the small boxes with low confidence levels contained inside are filtered out to avoid misdetecting Chinese character components as independent characters; Figure 5 (b) demonstrates that when the outer bounding box has low confidence and there is a smaller bounding box with higher confidence inside, deleting the outer bounding box and retaining the inner bounding box effectively suppresses the false detection and merging of adjacent small characters when the image is blurred.

[0046] Figure 6 This is a schematic diagram illustrating the layout analysis of an image using a whole-image Chinese character segmentation and recognition method according to the present invention.

[0047] Figure 7 This is a schematic diagram illustrating the segmentation of complex layouts and long text images using a whole-image Chinese character segmentation and recognition method according to the present invention; wherein, Figure 7 (a) is a scenario where rows and columns are tilted; Figure 7 (b) is a scenario where both uppercase and lowercase characters coexist; Figure 7 (c) is a scenario where multiple calligraphic styles coexist; Figure 7 (d) is a scene depicting an extremely long inscription;

[0048] Figure 8 This is a schematic diagram illustrating the generalization effect of a whole-image Chinese character segmentation and recognition method of the present invention on images with diverse imaging conditions; wherein, Figure 8 (a) in the image is a scene from a black and white rubbing. Figure 8 (b) in the image represents a scene with significant scanning noise; Figure 8 (c) in the image represents a blue background imaging scene;

[0049] Figure 9 This is a complete processing effect diagram of a whole-image Chinese character segmentation and recognition method using the present invention;

[0050] Figure 10 This is a schematic diagram of the structure of an embodiment of the whole-image Chinese character segmentation and recognition system of the present invention;

[0051] Figure 11 This is a schematic diagram of the structure of an embodiment of the computer device of the present invention. Detailed Implementation

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the specification is for the purpose of describing particular embodiments only and is not intended to limit the invention; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings are used to distinguish different objects and not to describe a particular order.

[0053] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0054] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0055] It should be noted that the whole-image Chinese character segmentation and recognition method provided in the embodiments of the present invention is generally executed by a server / terminal device, and correspondingly, the whole-image Chinese character segmentation and recognition system is generally set in the server / terminal device.

[0056] It should be understood that the number of terminal devices, networks, and servers is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be used.

[0057] Example 1

[0058] Please refer to Figures 1-9 The whole-image Chinese character segmentation and recognition method includes the following steps:

[0059] Step S1: Construct a dataset containing multiple calligraphic styles, multiple shooting scenes, and multiple quality images, and label the text regions in the calligraphy images.

[0060] In this embodiment, the electronic device (e.g., a server / terminal device) on which the whole-image Chinese character segmentation and recognition method runs can receive the whole-image Chinese character segmentation and recognition request via a wired connection or a wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future-developed wireless connection methods.

[0061] In this embodiment, step S1, constructing a dataset containing multiple calligraphic styles, multiple shooting scenes, and multiple quality images, and annotating the text regions in the calligraphy images, may specifically include the following steps:

[0062] S11: Construct a dataset containing multiple book styles, multiple shooting scenes, and multiple quality images.

[0063] Digital images of complete calligraphy works, such as rubbings of stone inscriptions, couplets, plaques, hanging scrolls, and screen panels, are collected from the internet and on-site, ensuring that the images cover various calligraphic styles, including seal script, cursive script, regular script, clerical script, and running script. High-definition scanned images are collected for standard printed scenes, images are collected under controlled lighting conditions for indoor museum collection scenes, and images are collected under natural light and different weather conditions for outdoor scenic stone inscription scenes.

[0064] A calligraphy image dataset was constructed based on collected calligraphy images containing various calligraphic styles.

[0065] S12, Mark the text areas in the calligraphy image.

[0066] The Roboflow annotation platform can be used to accurately annotate calligraphy image datasets. Each character in the image is bounded with a rectangle, and the boundary coordinates of the bounding box are recorded. The annotation format can be YOLO, and each bounding box annotation includes five parameters: category number. Normalized center point x-coordinate Normalized center point ordinate Normalized width and normalized height, where the formula for calculating normalized coordinates is:

[0067] , ,

[0068] in: : The coordinates of the top left and bottom right of the bounding box; Image width and height.

[0069] Save the annotation results as a txt text file with the same name as the image, with each line recording the annotation information of a text box.

[0070] S13, perform quality checks on the labeled dataset.

[0071] Quality checks include screening for erroneous annotations and identifying missing annotations. Screening for erroneous annotations includes checking whether the annotation box exceeds the image boundary, checking whether the aspect ratio of the annotation box is abnormal (aspect ratio > 5 or < 0.2 is considered abnormal), and checking whether there are highly overlapping annotation boxes in the same image (IoU > 0.7 is considered duplicate annotation). Identification of missing annotations is carried out through a combination of manual review and automatic detection. For the detected erroneous and missing annotations, the operation of step S12 is repeated to re-annotate or supplement the annotations.

[0072] S14 assigns a category number to the labeled text characters.

[0073] During the detection phase, all text characters are uniformly labeled as the same category, such as setting the category number to 0; organize the dataset directory structure, create an images directory to store the original image files, and create a labels directory to store the corresponding label txt files, ensuring that the image files and label files have the same name and correspond one-to-one; count the data dataset size, and record information such as the total number of images, the total number of labeled boxes, and the average number of labeled boxes per image.

[0074] S15, the organized dataset is randomly divided into training and test sets in an 8:2 ratio.

[0075] Create a dataSet directory and generate train.txt and val.txt files to record the complete paths of each image in the training and test sets, respectively; ensure that the sample ratio of each book type and scene type is balanced during the partitioning process; generate a dataset configuration file data.yaml to record information such as the training set path, validation set path, number of classes nc, and class names names, where nc is set to 1 and names is set to ['word'], indicating a single-class detection task.

[0076] Step S2 involves performing data augmentation on the dataset using the content-whitening and color transformation module to generate diverse training samples.

[0077] In this embodiment, step S2 may specifically include the following steps:

[0078] S21, adjust the color space of the labeled calligraphy image dataset.

[0079] During the training phase, random perturbation enhancement based on the HSV color space is performed on the input image to simulate the differences in shooting lighting and rubbing material; among them, the random adjustment range of hue H is ±0.015, the random adjustment range of saturation S is ±0.3, and the random adjustment range of brightness V is ±0.3.

[0080] S22, Performs geometric transformation to enhance the calligraphy image.

[0081] To ensure the structural stability of calligraphic characters and avoid excessive deformation, lightweight geometric perturbation enhancement is performed on the input image during the training phase. Specifically, this includes: random rotation angle within ±1°; random translation within ±2% of the image size; random scaling within ±15%; shearing transformation parameter set to 0; and perspective transformation parameter set to 0.

[0082] S23, perform blending enhancement on the calligraphy image.

[0083] During the training phase, mosaic enhancement is applied with a probability of 0.1 to improve the model's robustness to local occlusion, arrangement changes, and dense character regions; copy-paste enhancement is set to a probability of 0, and MixUp enhancement is not enabled.

[0084] S24, Flip and enhance the calligraphy image.

[0085] Considering that the directional nature of calligraphic characters does not hold true for the semantic invariance of characters in the actual shooting scene (up, down, left, right), the horizontal flip probability is set to 0 and the vertical flip probability is set to 0 in this embodiment to maintain the consistency of the calligraphic character direction.

[0086] S25, disable mosaic enhancement in the later stages of training.

[0087] To improve convergence stability in the later stages of training, this embodiment disables mosaic enhancement in the last 15 training rounds, allowing the model to primarily learn character boundaries and local texture features under the true distribution in the later stages.

[0088] S26. Generate cross-calligraphic style transfer samples based on reversible residual networks and cWCT modules.

[0089] In addition to the basic data augmentation mentioned above, this embodiment also introduces advanced augmentation based on a style transfer network to generate training samples across calligraphic styles. The style transfer network is used to decouple and fuse the content features of the content image with the style features of the style image, and the cWCT module completes the whitening-coloring transfer of the feature layer, thereby changing the stroke style and texture distribution while maintaining the character structure.

[0090] Step S26 specifically includes:

[0091] S261, construct the style transfer network and load the pre-trained weights.

[0092] A reversible residual network, RevResNet, is constructed as the backbone network for feature encoding and decoding. The number of network blocks, nBlocks, is [10, 10, 10], the downsampling stride, nStrides, is [1, 2, 2], and the number of channels, nChannels, is [16, 64, 256]. The number of input channels, in... channel The feature expansion factor is 3, the multiplier is 4, and the hidden layer dimension is 3. The number of self-similar propagation steps is 64. Set the value to 1; and load the pre-trained weight file photo_image.pt into the invertible residual network, and set the network to evaluation mode.

[0093] S262 performs scaling preprocessing on the content image and style image and converts them into tensors.

[0094] The content image and style image are scaled proportionally to a maximum side size of 1280 pixels, and the scaling results are made to meet the network downsampling factor alignment requirements. The images are then converted to four-dimensional tensor format and normalized to the range of 0 to 1.

[0095] S263, extract content feature coding and style feature coding.

[0096] The preprocessed content image tensor is input into an invertible residual network and forward propagation is performed to obtain the content feature encoding z_c; the preprocessed style image tensor is input into an invertible residual network and forward propagation is performed to obtain the style feature encoding z_s.

[0097] S264 uses the cWCT module to perform feature fusion to obtain fused features.

[0098] The cWCT module is called to perform whitening-coloring transfer of the feature layer to generate fused features z_cs for content feature encoding z_c and style feature encoding z_s. When it is necessary to control the degree of content preservation, a content weight α_c (between 0 and 1) is introduced to perform interpolation fusion between content statistics and style statistics, thereby controlling the degree of preservation of the original structure by the transfer result.

[0099] S265 performs backpropagation decoding on the fused features and generates a migration image.

[0100] The fused feature z_cs is input into the reversible residual network to perform backpropagation to obtain the transfer image tensor, which is then converted into a high-quality image file and saved. During the saving process, the pixel values ​​are multiplied by 255 and truncated to the range of 0 to 255 to complete the image formatting and disk saving.

[0101] S266, amplifies data and synchronizes annotation files according to a fixed generation strategy.

[0102] For each content image, five style images are randomly selected for style transfer, generating five times the number of cross-script style transfer training samples. The YOLO format annotation files corresponding to the content images are copied and renamed to ensure that the transferred images and annotation files have the same name and correspond one-to-one, thus forming a "style transfer merged dataset".

[0103] In specific implementation, the data augmentation process in step S2 includes basic data augmentation and advanced data augmentation based on a style transfer network. The style transfer network is used to decouple and fuse the content features of the content image and the style features of the style image to generate training samples across calligraphic styles. The style transfer network adopts a reversible residual network based on CAP-VSTNet to achieve the decoupling and fusion of the content features of the content image and the style features of the style image, and generates training samples across calligraphic styles through the cWCT module.

[0104] Basic data augmentation includes geometric and photometric transformations such as random rotation, flipping, color jitter, and random erasure.

[0105] Advanced style transfer augmentation uses the CAP-VSTNet network structure, which encodes features of content and style images through an invertible residual network, and uses the cWCT module to achieve feature decoupling and fusion to generate cross-script training samples.

[0106] The style transfer formula is: ,in, Encoding content features Encoding style features

[0107] The content weight (between 0 and 1) is used to control the degree of content retention.

[0108] Step S3: Based on the object detection network, train a text region detection model using the augmented dataset.

[0109] In this embodiment, step S3 may specifically include the following steps:

[0110] S31, Configure the training dataset and perform a pre-training integrity check.

[0111] Based on the "style transfer merged dataset" obtained in step S2, a dataset configuration file data.yaml is generated, which includes the dataset root directory path path, the training image relative directory train, the validation image relative directory val, the number of categories nc, and the category names names; in this embodiment, the number of categories is set to 1, which means that all text characters are treated as the same detection category.

[0112] Before training begins, the data.yaml file is read and the number of training images and annotation files is counted. The image directory and annotation directory are checked to ensure that the training data is available.

[0113] S32, Initialize the object detection model and load the pre-trained weights.

[0114] The YOLOv12 object detection network was selected as the text region detection model. The corresponding pre-trained weight file (e.g., yolo12s.pt) was loaded for transfer learning, thereby accelerating convergence and improving generalization performance on complex backgrounds.

[0115] S33, set the training resources and input scale parameters.

[0116] Based on the limitations of hardware memory and the high resolution of the entire calligraphy image, the training input image size was set to 1280 pixels, the batch size to 2, and the number of data loading threads to 8.

[0117] S34, set the training rounds and checkpoint saving strategy.

[0118] Set the training rounds to 200; disable the early stop mechanism to allow the training to run for the full 200 rounds; and save the model checkpoint every 10 rounds to facilitate resuming training midway and comparing the effects of different rounds.

[0119] S35, set the optimizer and learning rate scheduling strategy.

[0120] The AdamW optimizer was used for training; the initial learning rate was set to lr0=0.002, the cosine annealing terminal learning rate factor was set to lrf=0.01, and cosine learning rate scheduling was enabled to improve the convergence stability in the later stages of training; at the same time, the weight decay coefficient was set to 0.0005 to reduce the risk of overfitting.

[0121] S36, performs online data augmentation during the training process.

[0122] Enable the online data augmentation strategy during training, specifically including: HSV color perturbation augmentation (hsv_h = 0.015, hsv_s = 0.3, hsv_v = 0.3); lightweight geometric perturbation augmentation (degrees = 1.0, translate = 0.02, scale = 0.15, shear = 0.0, perspective = 0.0); flipping augmentation disabled (fliplr = 0.0, flipud = 0.0); mosaic augmentation enabled with a probability of 0.1 and disabled in the last 15 rounds of training (mosaic = 0.1, close_mosaic = 15); copy paste and MixUp augmentation disabled (copy_paste = 0.0, mixup = 0.0).

[0123] S37, output the training results and conduct verification and evaluation on the best model.

[0124] After training is completed, output the best weight file and the last weight file, and generate a training log and a training curve; then load the best weight model and perform model verification based on the verification configuration to obtain the evaluation metrics for the detection task (including mAP, Precision, and Recall), which are used to quantify the comprehensive performance of the model in the calligraphy character detection task.

[0125] Step S4, train a single-character classification model based on a deep convolutional network.

[0126] In this embodiment, step S4 may specifically include the steps:

[0127] Step S41, construct a single-character classification training dataset and establish a category mapping.

[0128] Store the single-character samples in a directory by category. The data root directory is . / data, where each category corresponds to a character directory. The naming of the character directory satisfies the form of "number - character" (for example, 0430 - yin, 0038 - kou, etc.), and further subdirectories are stored under each character directory according to the font style, including but not limited to subdirectories such as regular script, running script, cursive script, official script, and seal script. The corresponding character image files (*.jpg or *.png) are stored in each font subdirectory.

[0129] All character directories under the root directory of the data are traversed and scanned. The character directory names are parsed by the delimiter to obtain the real character labels (i.e., char_dir.name.split('-')[1]). Each character is assigned a unique category index according to the sorting result of the character directory to obtain the class_to_idx mapping table. At the same time, the sample list samples is recorded, in which each sample contains a tuple of image path and its category index (img_path, class_id).

[0130] Step S42 involves performing integrity verification, transparent background processing, and anomaly tolerance on the training samples.

[0131] During the scanning phase, integrity checks are performed on each image file, and an image decoding verification mechanism is used to remove damaged files, thus avoiding interruptions in the training process due to abnormal samples.

[0132] During the sample reading stage, transparent background normalization is performed on images that may have transparent channels: when the image is in RGBA / LA / P mode, the transparent area is composited onto a white background (RGB value is (255, 255, 255)) to output a uniform RGB three-channel image, so as to ensure the consistency of the input channels of the subsequent convolutional network and reduce the distribution drift caused by transparent pixels.

[0133] In addition, to improve training robustness, when individual samples still show abnormalities during reading or transformation, a limited retry strategy (up to 5 times) is adopted and the sample index is randomly replaced to continue training, thereby avoiding the failure of the entire training due to a single abnormal sample.

[0134] Step S43 introduces a dataset scan cache to reduce the preparation cost of repeated training.

[0135] The samples, class_to_idx, number of classes, number of samples, and other information obtained in step S41 are serialized and written to the cache file dataset_cache_{model_name}.json. When training is performed again and the cache exists, the sample list and class mapping are loaded directly from the cache, skipping the full scan step, thereby shortening the training startup time and ensuring the consistency of the class index.

[0136] Step S44: Perform stratified sampling to divide the dataset to ensure consistent training / validation distribution.

[0137] Based on the complete sample list, a stratified sampling method is used to divide the training set and validation set according to the category label. The validation set ratio is 0.15, the random seed is 42, and the stratify parameter is used to ensure that the proportion of each category in the training set and validation set is consistent, thereby avoiding the deviation of validation accuracy caused by uneven category distribution.

[0138] Subsequently, training dataset objects and validation dataset objects are constructed separately, and the split sample list and the same class_to_idx are injected into the dataset objects to ensure that the label space is completely consistent during the training / validation phases.

[0139] Step S45: Set up data preprocessing and augmentation strategies for training and validation.

[0140] The input images were uniformly scaled to the network's required size of 224×224. During the training phase, further enhancements were applied, including random rotation (angle range of approximately ±8 degrees), brightness and contrast perturbation (ColorJitter), and random erasing (probability 0.1, area ratio 0.02-0.08), to improve the generalization ability for different writing styles, stroke weights, slight rotations, and local defects. Normalization was performed during both the training and validation phases, using ImageNet statistical mean and standard deviation (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) to match the input distribution of the pre-trained convolutional backbone.

[0141] Step S46: Construct a deep convolutional network single-word classification model and configure regularization parameters.

[0142] A convolutional neural network backbone is selected as the feature extractor and classifier for the single-word classification network. In this embodiment, ConvNeXt-Base (corresponding to model_name='convnext_base') is preferably used as the backbone network. The number of classes in the output classification head is set to num_classes through the model creation interface, and explicit regularization is introduced to suppress overfitting, including but not limited to:

[0143] Set drop_rate=0.3 in the network to perform random deactivation;

[0144] Set drop_path_rate=0.2 to perform stochastic depth path dropping.

[0145] When the target model fails to be created, the system can fall back to ResNet152 and add Dropout (0.3) before its fully connected layers before mapping to num_classes to ensure the availability and stability of the training process.

[0146] Step S47: Define the loss function and introduce label smoothing to alleviate the overconfidence problem caused by inter-class similarity.

[0147] For single-character classification tasks, a label-smoothing cross-entropy loss is adopted. The one-hot labels of the true class are adjusted to soft label distribution with a smoothing coefficient ε=0.1, that is, the probability of the true class is 1-ε, and the other classes are evenly distributed ε / num_classes. This reduces the model's overfitting to the training set and the overconfidence output of similar character categories, thereby improving the generalization stability.

[0148] Step S48: Configure the optimizer, learning rate strategy, and mixed precision training mechanism.

[0149] AdamW was used as the optimizer with an initial learning rate of 1e-4, a weight decay coefficient of 1e-4, and a momentum parameter betas = (0.9, 0.999).

[0150] The learning rate scheduling adopts the cosine annealingWarmRestarts strategy, where T_0=20, T_mult=2, and eta_min=1e-6, to achieve periodic annealing and escape local optima through restarting in the later stage, thereby accelerating convergence and improving the final accuracy.

[0151] When the hardware supports CUDA, enable Automatic Mixed Precision (AMP) training. Use the GradScaler to suppress numerical underflow in half precision, improve training throughput and memory utilization, and maintain convergence stability.

[0152] S49 executes training iterations and implements gradient pruning and breakpoint recovery mechanisms.

[0153] The training set after step S44 is divided into 16 data loading processes (num_workers=16) and persistent worker processes (persistent_workers=True) are enabled for batch loading, with the batch size set to 96.

[0154] Train each epoch using the following procedure: Set the model to training mode, iterate through the training set batches, and execute the following for each batch:

[0155] (a) Load the image and labels to the GPU device;

[0156] (b) Output using a mixed-precision forward propagation computation model;

[0157] (c) Calculate the label smoothing cross-entropy loss;

[0158] (d) Perform backpropagation and perform norm clipping (max_norm=1.0) on the gradients of the model parameters to suppress gradient explosion and improve training stability;

[0159] (e) Update model parameters;

[0160] (f) Update the learning rate scheduler state.

[0161] Evaluate model performance on the validation set and calculate validation loss and accuracy. If the current validation accuracy is higher than the historical best accuracy, save the current model as the best model and reset the early stopping counter; otherwise, increment the early stopping counter by 1.

[0162] Training checkpoints are saved every epoch, including model parameters, optimizer state, learning rate scheduler state, mixed precision scaler state, and current best accuracy, to support breakpoint recovery after training interruptions. Training terminates when the early stop counter reaches 10 or when 100 epochs of training have been completed.

[0163] The training process reserves a Mixup enhancement interface (mixup_alpha=0.2), which can linearly mix samples from the same batch with probability and apply weighted loss when enabled. In the specific training configuration shown in this embodiment, Mixup is set to be disabled (use_mixup=False) to keep the training distribution more consistent with the actual character samples.

[0164] S410 extracts model parameters from the saved best model file to obtain the character recognition model with the highest accuracy on the validation set.

[0165] The parameter configurations of the character recognition model after training are summarized in Table 1:

[0166]

[0167] Through the above steps, a single-character image classification model based on a deep convolutional network (preferably ConvNeXt-Base) was trained, enabling the single-character images cropped from upstream to output the corresponding character category. Furthermore, mechanisms such as transparent background normalization, hierarchical partitioning, label smoothing, cosine restart scheduling, AMP, and breakpoint recovery were used to ensure training stability, reproducibility, and engineering usability.

[0168] Step S5: A dual-modal strategy combining full image and sliding window is adopted to detect text regions using a text region detection model, thereby obtaining a set of detection boxes.

[0169] In this embodiment, step S5 may specifically include the following steps:

[0170] S51, read the entire calligraphy image to be detected as a BGR three-channel image matrix I∈R^{H×W×3}, and obtain the image width W and height H.

[0171] S52, Initialize the whole image detection result container , and sliding window detection result container , These are used to store the bounding box coordinates and confidence scores detected by the two channels, respectively.

[0172] S53, set the dual-modal inference parameters.

[0173] Set the inference parameters for whole image detection (channel 1): inference size is 1280 pixels, confidence threshold is 0.20, IoU threshold is 0.5, and maximum number of detectors is 5000;

[0174] Set the sliding window detection (channel 2) inference parameters as follows: inference size is 1280 pixels, confidence threshold is 0.20, IoU threshold is 0.5, and maximum number of detectors is 3000;

[0175] Set the window overlap rate to 20%.

[0176] S54, perform full image detection to obtain the candidate box set for channel 1.

[0177] After scaling the image to a maximum side length of 1280 pixels, input it into the text region detection model trained in step S3, setting the confidence threshold to 0.20, the IoU threshold to 0.5, and the maximum number of detections to 5000; store the bounding box coordinates of the detection results. The corresponding confidence score is stored in .

[0178] S55, perform sliding window detection to obtain the candidate box set for channel 2.

[0179] The sliding window parameters are calculated using a dynamic window strategy, specifically:

[0180] Calculate the number of megapixels in an image 000, 000 and aspect ratio ;

[0181] According to megapixels and Determine the grid division ( ):

[0182] (1) When At that time, if Then set a 3×2 grid with a total of 6 windows. If so, set a 2×3 grid with a total of 6 windows; otherwise, set a 2×2 grid with a total of 4 windows.

[0183] (2) When At that time, if Then set a 3×2 grid with a total of 6 windows. If so, set a 2×3 grid with a total of 6 windows; otherwise, set a 3×3 grid with a total of 9 windows.

[0184] (3) When At that time, if or If so, set the corresponding grid; otherwise, set a 3×3 grid with a total of 9 windows.

[0185] Calculate window parameters based on grid configuration:

[0186] Window size:

[0187] Overlap size:

[0188] Sliding step size:

[0189] S56 performs sliding window cropping on the image and performs detection.

[0190] Starting from coordinates (0, 0), slide along the X and Y axes with a stride as the step size to crop the window area.

[0191] The clipping boundary is: , cropped window image Input the text region detection model trained in step S3, and set the inference size to 1280 pixels, the confidence threshold to 0.20, the IoU threshold to 0.5, and the maximum number of detections to 3000.

[0192] S57 maps the detection box in the sliding window coordinate system to the original image coordinate system.

[0193] The detection results of each window in step S54 are processed, and the coordinates of the detection box in the window coordinate system are... Mapping to the original coordinate system, the calculation formula is:

[0194] .

[0195] Cropping the detection box to the image boundary ensures that: 0 ≤ x1 < x2 ≤ W and 0 ≤ y1 < y2 ≤ H.

[0196] Append the mapped detection box coordinates to The corresponding confidence score is appended to the container. in the container.

[0197] S58, merges the results of whole-image detection and sliding window detection. and get ,merge and

[0198] get ;right Each bounding box in the image is cropped to ensure its coordinates remain within the image boundaries. The resulting...

[0199] and This forms a set of detection boxes, which serve as input for subsequent steps.

[0200] Step S6: After performing non-maximum suppression multi-level filtering on the detection box, cluster it by column and sort it within the column according to its spatial position, integrate the single-character classification model for recognition, and generate an ordered text sequence.

[0201] In this embodiment, step S6 may specifically include the following steps:

[0202] S61, the set of detection boxes obtained in step S5 Perform global NMS processing to remove duplicate detection boxes across channels.

[0203] Set the IoU threshold to 0.40. The detection boxes in the [database] are sorted by confidence score. After sorting in descending order, non-maximum suppression is performed. For any two detection boxes...

[0204] and Calculate the intersection-union ratio:

[0205]

[0206] The intersection area is calculated as follows: When IoU ≥ 0.40, the detection boxes with higher confidence are retained, and the detection boxes with lower confidence are deleted, resulting in the deduplicated detection box set boxes_nms and the corresponding confidence scores_nms.

[0207] S62, Perform size filtering on the detection frames processed in step S61 to remove extremely small detection frames.

[0208] Traversal Each detection box in Calculate its width and height When w < 35 pixels or h < 35 pixels, the bounding box is determined to be a tiny bounding box and deleted, resulting in a size-filtered set of detection boxes. and corresponding confidence level .

[0209] S63 performs inclusion filtering based on area hierarchy and confidence level gating.

[0210] An inclusion relationship filtering mechanism based on area hierarchy and confidence level is introduced to adaptively filter inner and outer candidate boxes that meet preset inclusion criteria: Calculation Area of ​​all detection frames Sort by area in descending order. For each outer candidate box boxi and its inner smaller box boxj, the condition for determining the containment relationship is: the IoU between boxi and boxj is ≥ 0.80, or the distance between the four vertices of boxj and the boundary of boxi is ≤ 10 pixels.

[0211] Set confidence threshold The following intelligent filtering strategy will be implemented:

[0212] (1) If the confidence level of the outer candidate box is Then, it filters out non-subject candidate boxes contained within it to avoid misidentifying Chinese character radicals, components, and other substructures as independent characters.

[0213] (2) If the confidence level of the outer candidate box And it has an internal confidence level. If the candidate box is not clear, the outer candidate box is deleted and the inner high-confidence candidate box is retained, thereby effectively suppressing the false detection and merging of adjacent small characters when the image is blurred;

[0214] (3) If both the outer frame and the inner frame are low confidence levels Then, the confidence scores of the outer bounding box and the maximum confidence scores of the inner bounding box are compared, and the candidate bounding box with the higher confidence score is retained.

[0215] Obtain the set of detection boxes after inclusion relationship filtering. and corresponding confidence level .

[0216] S64 performs adaptive structural filtering based on candidate box area statistics.

[0217] calculate The set of areas of all detection boxes in the middle Calculate the median area Amedian for area A. Set the dynamic threshold to... , where k is a multiplier, and in this embodiment k=4.

[0218] Filtering large-scale candidate box sets Check the layout geometry constraints for each detection box in Blarge:

[0219] (1) Calculate the aspect ratio of the detection frame. ;

[0220] (2) When r > 2.0 and w > 0.3 × W or h > 0.3 × H, the frame is determined to be a cover / title structure area;

[0221] (3) If a cover / title structure exists and the detection results do not present a title-text mixed layout feature, the main bounding box in Blarge is retained first, and small-scale candidate boxes with IoU>0.20 are filtered out.

[0222] When the detection results exhibit layout characteristics of the title and body text (i.e., a clear spatial hierarchical structure), the above filtering is automatically canceled to retain the character boxes in the body text area. This yields the structure-filtered set of detection boxes (boxes_struct) and their corresponding confidence scores (scores_struct).

[0223] S65, apply final confidence filtering to determine the set of valid detection boxes.

[0224] Iterate through boxes_struct and its corresponding scores_struct, keeping only the detection boxes with a confidence score ≥ 0.10, to obtain the final set of valid detection boxes boxes_final and its corresponding detection model confidence score S1.

[0225] S66, batch cropping character regions and performing recognition.

[0226] Based on the bounding box coordinates in `boxes_final`, batch crop character regions from the original image I: The cropped character image is adjusted to 224×224 pixels and input into the character recognition model trained in step S4 for classification and inference.

[0227] The recognition model is based on the ConvNeXt-Base architecture and outputs the Top-K candidate categories and recognition confidence scores for each character. A character error correction mapping table is applied to the recognition results to map easily confused characters to standard glyphs, obtaining the character category label corresponding to each detection box.

[0228] S67 performs adaptive layout analysis and sorting based on the spatial position of the detection box.

[0229] Calculate the center point coordinates of each detection box. , An adaptive clustering algorithm based on spacing statistics is used to automatically sort the data according to the horizontal coordinate.

[0230] (1) Arrange the detection boxes in descending order from right to left according to the x-coordinate of their center points, and calculate the set of lateral spacing between adjacent detection boxes. ;

[0231] (2) Calculate the average spacing avgGap, and set the tolerance factor γ=1.5. When it is determined to be a column split boundary;

[0232] (3) Divide the detection box into several text columns. The columns are numbered from right to left;

[0233] (4) Sort the detection boxes in each column from top to bottom according to the vertical coordinate of the center point, and assign column number and row number to each detection box.

[0234] Traverse each column sequentially from right to left, extracting character categories within each column in vertical order, and concatenating them to generate an ordered text sequence that conforms to the right-to-left and top-to-bottom reading habits of calligraphy works. Output the recognized text along with its corresponding detection box coordinates, column number, row number, and confidence score, completing the text region detection, layout analysis, and recognition process for the entire calligraphy work.

[0235] The beneficial effects of implementing this embodiment are:

[0236] (1) Strong ability to analyze complex layouts: For complex layout structures such as the coexistence of large and small characters and the mixing of multiple calligraphic styles in stone inscriptions, this invention achieves high recall and complementary detection of characters of different scales through a dual-modal detection framework that combines whole images and sliding windows. Combined with the fusion deduplication and inclusion relationship filtering mechanism, it effectively avoids missed detection and false detection caused by the overlap and nesting of large and small character frames, and significantly improves the segmentation stability in complex mixed layout scenarios.

[0237] (2) Possesses adaptive layout analysis and intelligent post-processing capabilities: Based on the spatial distribution of detection boxes, this invention performs adaptive column division and sorting, which can accurately infer the reading order of images such as inscriptions from right to left and from top to bottom, and has good adaptability to uneven column spacing, page tilt and shooting distortion. At the same time, by integrating multi-level post-processing and a confidence-based nested box filtering mechanism, it effectively suppresses noise, removes interference boxes, and avoids false detection of radicals or adjacent small characters, thereby outputting clean and structurally correct recognition results in complex layouts.

[0238] (3) Supports automated processing of long inscriptions: This invention constructs an integrated process from image input, joint detection of whole image and sliding window, character cropping, single character recognition to layout analysis and structured text output. It can achieve continuous and stable processing in long inscriptions, epitaphs and rubbings with thousands of characters, significantly reducing the cost of manual character selection and sorting, and improving the efficiency of large-scale digital processing in the field of archaeology and cultural relics.

[0239] (4) High environmental robustness: This invention suppresses false noise detection through adaptive filtering and structured constraint mechanisms, and improves the reliability of the results by using confidence competition and spatial relationship discrimination in the fusion stage. It can effectively deal with common problems in rubbings and photographic images, such as noise interference, edge damage, uneven lighting, local blurring, deformation and perspective distortion. At the same time, this method has good adaptability and stability to various imaging conditions and material backgrounds, including black and white, color, blue background and gray background, and performs reliably on data from different sources.

[0240] (5) Significant advantages in multi-script generalization ability and overall efficiency: This invention trains the detection and recognition model based on multi-script samples and data augmentation strategies such as style transfer, enabling the system to have good generalization ability for various scripts such as seal script, cursive script, clerical script, regular script, and running script. It can also maintain stable reading performance in scenarios where large-character titles and small-character main texts coexist and the differences in character shapes are significant. Compared with the traditional processing flow that relies on manual character segmentation and reading, this invention achieves automated processing from the whole calligraphy image to structured text through whole-image sliding window detection fusion, structured filtering, and reading order restoration. In long text tasks, it can significantly reduce the degree of human intervention and greatly improve the processing efficiency, up to 50 times that of traditional methods.

[0241] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0242] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0243] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0244] Example 2

[0245] Figure 10 This is a schematic diagram of an embodiment of the whole-image Chinese character segmentation and recognition system of the present invention. Further reference... Figure 10 As a response to the above Figure 1 The present invention provides an embodiment of a whole-image Chinese character segmentation and recognition system, which is implemented by the method shown. Figure 1 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0246] like Figure 10 As shown, the whole-image Chinese character segmentation and recognition system 70 described in this embodiment includes: a labeling module 71, a processing module 72, a training module 73, a detection module 74, a post-processing module 75, and a recognition and output module 76. Wherein:

[0247] The annotation module 71 is used to construct a calligraphy image dataset containing multiple calligraphic styles, multiple shooting scenes and multiple quality images, and to annotate the text regions in the dataset;

[0248] Processing module 72 is used to perform data augmentation processing on the dataset through the cWCT module to generate diverse training samples;

[0249] Training module 73 is used to train a text region detection model based on an object detection network using an augmented dataset; and to train a single character classification model based on a deep convolutional network.

[0250] The detection module 74 is used to perform text region detection on the calligraphy image to be recognized by adopting a dual-modal strategy that combines the whole image and the sliding window, and outputs a set of detection boxes.

[0251] Post-processing module 75 is used to perform the following operations sequentially on the detection box set: NMS multi-level filtering; adaptive layout analysis based on the spatial distribution characteristics of the filtered detection boxes to determine the reading order; and sorting the detection boxes according to the determined reading order.

[0252] The recognition and output module 76 is used to call the single-character classification model to recognize the single-character image cropped by the detection box and finally output ordered text.

[0253] The beneficial effects of implementing this embodiment are: it has end-to-end automated processing capabilities for long inscriptions exceeding 1,000 characters; it has the ability to parse complex layouts that accommodate both large and small characters and mixed script styles; it can adaptively divide the layout and infer the reading order, making it suitable for long vertical inscriptions; it has stronger robustness in real-world data collection environments; and it has multi-script generalization capabilities and overall efficiency advantages.

[0254] Example 3

[0255] To address the aforementioned technical problems, embodiments of the present invention also provide a computer device. Please refer to [link / reference needed]. Figure 11 , Figure 11 This is a basic structural block diagram of the computer device in this embodiment.

[0256] The aforementioned computer device 8 includes a memory 81, a processor 82, and a network interface 83 that are interconnected via a system bus. It should be noted that only the computer device 8 with components 81, 82, and 83 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0257] The aforementioned computer devices can be desktop computers, laptops, handheld computers, and cloud servers, among other computing devices. These devices can facilitate human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0258] The aforementioned memory 81 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the aforementioned memory 81 may be an internal storage unit of the aforementioned computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the aforementioned memory 81 may also be an external storage device of the aforementioned computer device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 8. Of course, the aforementioned memory 81 may also include both the internal storage unit and its external storage device of the aforementioned computer device 8. In this embodiment, the aforementioned memory 81 is typically used to store the operating system and various application software installed on the aforementioned computer device 8, such as computer-readable instructions for a whole-image Chinese character segmentation and recognition method. In addition, the aforementioned memory 81 can also be used to temporarily store various types of data that have been output or will be output.

[0259] In some embodiments, the processor 82 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 82 is typically used to control the overall operation of the computer device 8. In this embodiment, the processor 82 is used to execute computer-readable instructions stored in the memory 81 or to process data, such as executing computer-readable instructions for the whole-image Chinese character segmentation and recognition method.

[0260] The network interface 83 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the computer device 8 and other electronic devices.

[0261] The beneficial effects of implementing this embodiment are: it has end-to-end automated processing capabilities for long inscriptions exceeding 1,000 characters; it has the ability to parse complex layouts that accommodate both large and small characters and mixed script styles; it can adaptively divide the layout and infer the reading order, making it suitable for long vertical inscriptions; it has stronger robustness in real-world data collection environments; and it has multi-script generalization capabilities and overall efficiency advantages.

[0262] Example 4

[0263] The present invention also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the whole-image Chinese character segmentation and recognition method described above.

[0264] The beneficial effects of implementing this embodiment are: it has end-to-end automated processing capabilities for long inscriptions exceeding 1,000 characters; it has the ability to parse complex layouts that accommodate both large and small characters and mixed script styles; it can adaptively divide the layout and infer the reading order, making it suitable for long vertical inscriptions; it has stronger robustness in real-world data collection environments; and it has multi-script generalization capabilities and overall efficiency advantages.

[0265] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0266] Obviously, the embodiments described above are merely some embodiments of the present invention, not all embodiments. The accompanying drawings show preferred embodiments of the present invention, but do not limit the patent scope of the present invention. The present invention can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the patent protection scope of this invention.

Claims

1. A method for segmenting and recognizing Chinese characters in a whole image, characterized in that, Includes the following steps: S1. Construct a dataset containing multiple calligraphic styles, multiple shooting scenes, and multiple quality images, and label the text regions in the calligraphy images; S2. Perform data augmentation on the dataset using the content-whitening and color transformation module to generate diverse training samples; S3. Based on the object detection network, train the text region detection model using the augmented dataset; S4. Training a single-word classification model based on a deep convolutional network; S5. A dual-modal strategy combining full image and sliding window is adopted to perform text region detection using the text region detection model to obtain a set of detection boxes; S6. After performing non-maximum suppression multi-level filtering on the detection box, perform column-based clustering and intra-column sorting based on its spatial position, integrate the single-character classification model for recognition, and generate an ordered text sequence.

2. The whole-image Chinese character segmentation and recognition method according to claim 1, characterized in that, The data augmentation process in step S2 includes basic data augmentation and advanced data augmentation based on a style transfer network. The style transfer network is used to decouple and fuse the content features of the content image and the style features of the style image to generate training samples across calligraphic styles. The style transfer network adopts an invertible residual network based on CAP-VSTNet to achieve the decoupling and fusion of the content features of the content image and the style features of the style image, and generates training samples across calligraphic styles through the cWCT module.

3. The whole-image Chinese character segmentation and recognition method according to claim 1, characterized in that, In step S5, the text region detection model is based on the YOLO network architecture and adopts a dual-modal detection framework that combines whole-image detection and sliding window detection: whole-image detection is used to obtain candidate text boxes with global structural constraints, and sliding window detection is used to supplement small-scale candidate text boxes under local high-resolution view. The candidate boxes output by the two modalities are uniformly mapped to the original image coordinate system and merged into the same candidate set. Cross-modal global nonmaximum suppression is performed based on the candidate box confidence to realize duplicate box elimination and result fusion, thereby outputting a unified candidate box set.

4. The whole-image Chinese character segmentation and recognition method according to claim 1, characterized in that, After model integration, multi-level post-processing and adaptive structural filtering are performed on the candidate box set output in step S5 to obtain a stable final character candidate box set. The multi-level post-processing and adaptive structural filtering include at least the following steps: Apply boundary constraints to the candidate box coordinates to ensure they fall within the coordinate range of the original image; Minimum candidate boxes smaller than a preset minimum size threshold are removed to suppress image noise and scanning interference; An inclusion relationship filtering mechanism based on area hierarchy and confidence level gating is introduced to adaptively filter inner and outer candidate boxes that meet the preset inclusion judgment conditions: if the confidence level of the outer candidate box reaches the preset threshold, the non-subject candidate boxes contained within it are filtered out to avoid misidentifying Chinese radicals, components, and other substructures as independent characters; if the confidence level of the outer candidate box is lower than the preset threshold and there are candidate boxes with higher confidence levels within it, the outer candidate box is deleted and the high-confidence candidate boxes within it are retained, thereby effectively suppressing the false detection and merging of adjacent small characters when the image is blurred.

5. The whole-image Chinese character segmentation and recognition method according to claim 4, characterized in that, During the text region cropping process, a coordinate transformation relationship is established between the display coordinate system and the original image pixel coordinate system: the horizontal and vertical scaling factors are determined according to the original image pixel size and the display rendering size, and the candidate box coordinates under the display coordinate system are mapped to the original image pixel coordinate system according to the scaling factors to obtain the original image coordinates used for cropping, thereby realizing accurate cropping of candidate text regions and generation of character images.

6. The whole-image Chinese character segmentation and recognition method according to claim 1, characterized in that, The adaptive layout analysis in step S6 specifically involves: calculating the center point coordinates of the detection boxes that meet the confidence conditions; adaptively determining the text column segmentation boundary and completing the column division based on the horizontal distribution of the center points using spacing statistics and adjustable fault tolerance coefficients; generating the column order by sorting the center point vertical coordinates from top to bottom within each column; and finally determining the column order according to the preset or recognized layout direction to obtain the reading order of the full text and output the recognized text.

7. The whole-image Chinese character segmentation and recognition method according to claim 1, characterized in that, The training optimization of the single-character classification model adopts a combination of regularization training strategy and learning rate scheduling strategy. Specifically, a label smoothing mechanism can be introduced when constructing the loss function to alleviate class noise and overfitting, and cosine annealing learning rate scheduling is used during training to make the learning rate decay smoothly as the training progresses.

8. A whole-image Chinese character segmentation and recognition system, characterized in that, include: The annotation module is used to construct a calligraphy image dataset containing multiple calligraphic styles, multiple shooting scenes, and multiple quality images, and to annotate the text regions in the dataset. The processing module is used to perform data augmentation processing on the dataset through the cWCT module to generate diverse training samples; The training module is used to train a text region detection model based on an object detection network using an augmented dataset; and to train a single-character classification model based on a deep convolutional network. The detection module is used to perform text region detection on the calligraphy image to be recognized by using a dual-modal strategy that combines the whole image and sliding window, and outputs a set of detection boxes. The post-processing module is used to perform the following operations sequentially on the detection box set: NMS multi-level filtering; adaptive layout analysis based on the spatial distribution characteristics of the filtered detection boxes to determine the reading order; and sorting the detection boxes according to the determined reading order. The recognition and output module is used to call the single-character classification model to recognize the single-character images cropped by the detection box and finally output ordered text.

9. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, characterized in that, When the processor executes the computer-readable instructions, it implements the steps of the whole-image Chinese character segmentation and recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-readable instructions thereon, characterized in that, When the computer-readable instructions are executed by the processor, they implement the steps of the whole-image Chinese character segmentation and recognition method as described in any one of claims 1 to 7.

Images